2024-05-24T11:15:06.746 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-24T11:15:06.747 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/postgres}} --name leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose --archive /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222 -- /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/orig.config.yaml 2024-05-24T11:15:06.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-24T11:15:06.893 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222 branch: reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/postgres}} email: null first_in_suite: false job_id: '7718222' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: reef ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: b9b067bc6d30ec00e391f0fa43f50affe981ba2f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: b9b067bc6d30ec00e391f0fa43f50affe981ba2f kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 630900aeb5fd525af2a375cc2059645127fe426f owner: scheduled_leonidus@teuthology priority: 120 repo: https://github.com/ceph/ceph roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7924 sha1: b9b067bc6d30ec00e391f0fa43f50affe981ba2f sleep_before_teardown: 0 subset: 244/512 suite: fs suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 630900aeb5fd525af2a375cc2059645127fe426f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPwY+Q/LWsWZ2m1Npsm19rPwGG37sM/++l3FxefreIUkJ5pLiXiln6wBt0bIvaITZkljpeB+MW9iAeX4Lx7ac/U= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBuFNL1hn4ISB3vej0uWQ/OLbaQM4VgyD2+nBQUHAE4AhocPDG3ITTTDW2gyJ4/GpxcHnbehe57ht6Gh1cb/kvg= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDGIARQ1mhpyVAUQrOy7uAvSr4qS7EsnG0X3v40fQEwhuyQ1EyepaNccD1TesuCSK6K6NepP1hd4hH1Dop0ordo= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-21_10:46:42 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-24T11:15:06.893 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa; will attempt to use it 2024-05-24T11:15:06.894 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa/tasks 2024-05-24T11:15:06.894 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-24T11:15:06.895 INFO:teuthology.task.internal:Checking packages... 2024-05-24T11:15:06.943 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f' 2024-05-24T11:15:06.943 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-24T11:15:06.943 INFO:teuthology.packaging:ref: None 2024-05-24T11:15:06.943 INFO:teuthology.packaging:tag: None 2024-05-24T11:15:06.943 INFO:teuthology.packaging:branch: reef 2024-05-24T11:15:06.943 INFO:teuthology.packaging:sha1: b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:15:06.944 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2024-05-24T11:15:07.273 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1355.gf1882d80 2024-05-24T11:15:07.274 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-24T11:15:07.283 INFO:teuthology.task.internal:no buildpackages task found 2024-05-24T11:15:07.283 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-24T11:15:07.291 INFO:teuthology.task.internal:Saving configuration 2024-05-24T11:15:07.314 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-24T11:15:07.323 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-24T11:15:07.350 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-24 11:09:09.520947', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPwY+Q/LWsWZ2m1Npsm19rPwGG37sM/++l3FxefreIUkJ5pLiXiln6wBt0bIvaITZkljpeB+MW9iAeX4Lx7ac/U='} 2024-05-24T11:15:07.376 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-24 11:09:09.522206', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBuFNL1hn4ISB3vej0uWQ/OLbaQM4VgyD2+nBQUHAE4AhocPDG3ITTTDW2gyJ4/GpxcHnbehe57ht6Gh1cb/kvg='} 2024-05-24T11:15:07.396 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-24 11:09:09.523364', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDGIARQ1mhpyVAUQrOy7uAvSr4qS7EsnG0X3v40fQEwhuyQ1EyepaNccD1TesuCSK6K6NepP1hd4hH1Dop0ordo='} 2024-05-24T11:15:07.396 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-24T11:15:07.403 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-24T11:15:07.404 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-24T11:15:07.404 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-24T11:15:07.404 INFO:teuthology.run_tasks:Running task console_log... 2024-05-24T11:15:07.492 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe6c4bc5360>, signals=[15]) 2024-05-24T11:15:07.493 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-24T11:15:07.499 INFO:teuthology.task.internal:Opening connections... 2024-05-24T11:15:07.499 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:15:07.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-24T11:15:07.572 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-05-24T11:15:07.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-24T11:15:07.652 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-05-24T11:15:07.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-24T11:15:07.732 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-24T11:15:07.740 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-05-24T11:15:07.760 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-05-24T11:15:07.760 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-05-24T11:15:07.820 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Red Hat Enterprise Linux" 2024-05-24T11:15:07.820 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8.6 (Ootpa)" 2024-05-24T11:15:07.820 INFO:teuthology.orchestra.run.smithi039.stdout:ID="rhel" 2024-05-24T11:15:07.820 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="fedora" 2024-05-24T11:15:07.820 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8.6" 2024-05-24T11:15:07.820 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2024-05-24T11:15:07.820 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-24T11:15:07.820 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-05-24T11:15:07.820 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-24T11:15:07.821 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.redhat.com/" 2024-05-24T11:15:07.821 INFO:teuthology.orchestra.run.smithi039.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-24T11:15:07.821 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-24T11:15:07.821 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:15:07.821 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-24T11:15:07.821 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-24T11:15:07.821 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-24T11:15:07.821 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-24T11:15:07.822 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-05-24T11:15:07.845 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-05-24T11:15:07.867 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-05-24T11:15:07.867 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-05-24T11:15:07.928 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Red Hat Enterprise Linux" 2024-05-24T11:15:07.928 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8.6 (Ootpa)" 2024-05-24T11:15:07.928 INFO:teuthology.orchestra.run.smithi053.stdout:ID="rhel" 2024-05-24T11:15:07.928 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="fedora" 2024-05-24T11:15:07.928 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8.6" 2024-05-24T11:15:07.928 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2024-05-24T11:15:07.928 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-24T11:15:07.928 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-05-24T11:15:07.929 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-24T11:15:07.929 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.redhat.com/" 2024-05-24T11:15:07.929 INFO:teuthology.orchestra.run.smithi053.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-24T11:15:07.929 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-24T11:15:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:15:07.929 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-24T11:15:07.929 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-24T11:15:07.929 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-24T11:15:07.929 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-24T11:15:07.929 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-05-24T11:15:07.953 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-05-24T11:15:07.975 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-05-24T11:15:07.975 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="Red Hat Enterprise Linux" 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="8.6 (Ootpa)" 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:ID="rhel" 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="fedora" 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="8.6" 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el8" 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-24T11:15:08.035 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://www.redhat.com/" 2024-05-24T11:15:08.036 INFO:teuthology.orchestra.run.smithi137.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-24T11:15:08.036 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-24T11:15:08.036 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:15:08.036 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-24T11:15:08.036 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-24T11:15:08.036 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-24T11:15:08.036 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-24T11:15:08.036 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-05-24T11:15:08.061 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-24T11:15:08.071 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-24T11:15:08.076 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-24T11:15:08.076 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-05-24T11:15:08.078 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-05-24T11:15:08.080 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-05-24T11:15:08.099 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-24T11:15:08.106 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-24T11:15:08.106 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-05-24T11:15:08.138 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-05-24T11:15:08.141 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-05-24T11:15:08.175 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-24T11:15:08.258 INFO:teuthology.run_tasks:Running task kernel... 2024-05-24T11:15:08.272 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-05-24T11:15:08.272 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-05-24T11:15:08.273 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-24T11:15:08.274 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-24T11:15:08.274 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-24T11:15:08.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T11:15:08.294 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-05-24T11:15:08.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T11:15:08.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T11:15:08.295 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-05-24T11:15:08.296 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-05-24T11:15:08.354 INFO:teuthology.orchestra.run.smithi137.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:08.354 INFO:teuthology.task.kernel:Running kernel on smithi137: 4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:08.354 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum install -y kernel 2024-05-24T11:15:08.356 INFO:teuthology.orchestra.run.smithi053.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:08.356 INFO:teuthology.orchestra.run.smithi039.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:08.356 INFO:teuthology.task.kernel:Running kernel on smithi053: 4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:08.357 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum install -y kernel 2024-05-24T11:15:08.357 INFO:teuthology.task.kernel:Running kernel on smithi039: 4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:08.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum install -y kernel 2024-05-24T11:15:09.869 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2024-05-24T11:15:09.869 INFO:teuthology.orchestra.run.smithi137.stdout:Unable to read consumer identity 2024-05-24T11:15:09.880 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2024-05-24T11:15:09.880 INFO:teuthology.orchestra.run.smithi039.stdout:Unable to read consumer identity 2024-05-24T11:15:09.936 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-24T11:15:09.936 INFO:teuthology.orchestra.run.smithi053.stdout:Unable to read consumer identity 2024-05-24T11:15:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:15:10.165 INFO:teuthology.orchestra.run.smithi039.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-24T11:15:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:15:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:15:10.168 INFO:teuthology.orchestra.run.smithi137.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-24T11:15:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:15:10.228 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:15:10.228 INFO:teuthology.orchestra.run.smithi053.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-24T11:15:10.228 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:15:11.921 INFO:teuthology.orchestra.run.smithi137.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2024-05-24T11:15:11.979 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-05-24T11:15:11.985 INFO:teuthology.orchestra.run.smithi039.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2024-05-24T11:15:13.032 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 12 MB/s | 14 MB 00:01 2024-05-24T11:15:13.324 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 10 MB/s | 14 MB 00:01 2024-05-24T11:15:13.757 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 14 MB 00:01 2024-05-24T11:15:17.097 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 505 kB/s | 24 kB 00:00 2024-05-24T11:15:17.686 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 454 kB/s | 24 kB 00:00 2024-05-24T11:15:17.858 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-24T11:15:17.858 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-24T11:15:17.940 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-24T11:15:17.941 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-05-24T11:15:17.941 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-24T11:15:18.077 DEBUG:teuthology.orchestra.run.smithi137:> echo no | sudo yum reinstall kernel || true 2024-05-24T11:15:18.230 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 441 kB/s | 24 kB 00:00 2024-05-24T11:15:18.411 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2024-05-24T11:15:18.412 INFO:teuthology.orchestra.run.smithi137.stdout:Unable to read consumer identity 2024-05-24T11:15:18.564 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-24T11:15:18.565 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-24T11:15:18.670 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-05-24T11:15:18.671 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-05-24T11:15:18.672 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-05-24T11:15:18.740 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:15:18.740 INFO:teuthology.orchestra.run.smithi137.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-24T11:15:18.740 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:15:18.807 DEBUG:teuthology.orchestra.run.smithi039:> echo no | sudo yum reinstall kernel || true 2024-05-24T11:15:19.049 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-24T11:15:19.050 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-24T11:15:19.052 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 May 2024 11:15:17 AM UTC. 2024-05-24T11:15:19.120 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-24T11:15:19.121 INFO:teuthology.orchestra.run.smithi137.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-24T11:15:19.122 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-05-24T11:15:19.122 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-24T11:15:19.129 INFO:teuthology.orchestra.run.smithi137.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-24T11:15:19.170 INFO:teuthology.orchestra.run.smithi137.stderr:Error: No packages marked for reinstall. 2024-05-24T11:15:19.209 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2024-05-24T11:15:19.209 INFO:teuthology.orchestra.run.smithi039.stdout:Unable to read consumer identity 2024-05-24T11:15:19.215 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum reinstall -y kernel || true 2024-05-24T11:15:19.284 DEBUG:teuthology.orchestra.run.smithi053:> echo no | sudo yum reinstall kernel || true 2024-05-24T11:15:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:15:19.523 INFO:teuthology.orchestra.run.smithi039.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-24T11:15:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:15:19.547 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2024-05-24T11:15:19.547 INFO:teuthology.orchestra.run.smithi137.stdout:Unable to read consumer identity 2024-05-24T11:15:19.672 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-24T11:15:19.672 INFO:teuthology.orchestra.run.smithi053.stdout:Unable to read consumer identity 2024-05-24T11:15:19.850 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 May 2024 11:15:17 AM UTC. 2024-05-24T11:15:19.861 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:15:19.861 INFO:teuthology.orchestra.run.smithi137.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-24T11:15:19.861 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:15:19.886 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:15:19.886 INFO:teuthology.orchestra.run.smithi053.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-24T11:15:19.886 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:15:19.946 INFO:teuthology.orchestra.run.smithi039.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-24T11:15:19.947 INFO:teuthology.orchestra.run.smithi039.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-24T11:15:19.985 INFO:teuthology.orchestra.run.smithi039.stderr:Error: No packages marked for reinstall. 2024-05-24T11:15:20.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum reinstall -y kernel || true 2024-05-24T11:15:20.169 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:03 ago on Fri 24 May 2024 11:15:17 AM UTC. 2024-05-24T11:15:20.237 INFO:teuthology.orchestra.run.smithi137.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-24T11:15:20.238 INFO:teuthology.orchestra.run.smithi137.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-24T11:15:20.271 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 May 2024 11:15:18 AM UTC. 2024-05-24T11:15:20.273 INFO:teuthology.orchestra.run.smithi137.stderr:Error: No packages marked for reinstall. 2024-05-24T11:15:20.325 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-05-24T11:15:20.356 INFO:teuthology.orchestra.run.smithi053.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-24T11:15:20.364 INFO:teuthology.orchestra.run.smithi053.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-24T11:15:20.387 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2024-05-24T11:15:20.387 INFO:teuthology.orchestra.run.smithi039.stdout:Unable to read consumer identity 2024-05-24T11:15:20.408 INFO:teuthology.orchestra.run.smithi053.stderr:Error: No packages marked for reinstall. 2024-05-24T11:15:20.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum reinstall -y kernel || true 2024-05-24T11:15:20.495 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:20.495 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:20.496 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-24T11:15:20.496 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-24T11:15:20.496 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-24T11:15:20.531 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-05-24T11:15:20.543 DEBUG:teuthology.parallel:result is None 2024-05-24T11:15:20.634 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:15:20.635 INFO:teuthology.orchestra.run.smithi039.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-24T11:15:20.635 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:15:20.846 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-24T11:15:20.847 INFO:teuthology.orchestra.run.smithi053.stdout:Unable to read consumer identity 2024-05-24T11:15:20.954 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:03 ago on Fri 24 May 2024 11:15:17 AM UTC. 2024-05-24T11:15:21.025 INFO:teuthology.orchestra.run.smithi039.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-24T11:15:21.026 INFO:teuthology.orchestra.run.smithi039.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-24T11:15:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:Error: No packages marked for reinstall. 2024-05-24T11:15:21.068 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:15:21.068 INFO:teuthology.orchestra.run.smithi053.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-24T11:15:21.068 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:15:21.098 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q kernel | sort -rV | head -n 1 2024-05-24T11:15:21.276 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:21.276 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:21.277 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-24T11:15:21.277 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-24T11:15:21.277 DEBUG:teuthology.orchestra.run.smithi039:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-24T11:15:21.313 INFO:teuthology.orchestra.run.smithi039.stdout:ttyS1 2024-05-24T11:15:21.327 DEBUG:teuthology.parallel:result is None 2024-05-24T11:15:21.429 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:03 ago on Fri 24 May 2024 11:15:18 AM UTC. 2024-05-24T11:15:21.504 INFO:teuthology.orchestra.run.smithi053.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-24T11:15:21.504 INFO:teuthology.orchestra.run.smithi053.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-24T11:15:21.532 INFO:teuthology.orchestra.run.smithi053.stderr:Error: No packages marked for reinstall. 2024-05-24T11:15:21.590 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-05-24T11:15:21.766 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:21.766 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-24T11:15:21.767 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-24T11:15:21.767 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-24T11:15:21.767 DEBUG:teuthology.orchestra.run.smithi053:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-24T11:15:21.806 INFO:teuthology.orchestra.run.smithi053.stdout:ttyS1 2024-05-24T11:15:21.826 DEBUG:teuthology.parallel:result is None 2024-05-24T11:15:21.826 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-24T11:15:21.834 INFO:teuthology.task.internal:Creating test directory... 2024-05-24T11:15:21.834 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-24T11:15:21.836 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-24T11:15:21.869 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-24T11:15:21.889 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-24T11:15:21.899 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-24T11:15:21.912 INFO:teuthology.task.internal:Creating archive directory... 2024-05-24T11:15:21.912 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-24T11:15:21.915 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-24T11:15:21.930 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-24T11:15:21.980 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-24T11:15:21.990 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-24T11:15:21.990 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-24T11:15:21.992 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-24T11:15:22.009 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-24T11:15:22.037 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-24T11:15:22.046 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-24T11:15:22.048 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-24T11:15:22.051 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-24T11:15:22.057 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-24T11:15:22.065 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-24T11:15:22.067 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-24T11:15:22.074 INFO:teuthology.task.internal:Configuring sudo... 2024-05-24T11:15:22.075 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-24T11:15:22.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-24T11:15:22.079 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-24T11:15:22.129 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-24T11:15:22.140 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-24T11:15:22.140 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-24T11:15:22.145 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-24T11:15:22.154 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-24T11:15:22.188 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-24T11:15:22.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-24T11:15:22.302 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-24T11:15:22.387 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-24T11:15:22.428 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:15:22.429 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-24T11:15:22.495 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-24T11:15:22.543 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-24T11:15:22.607 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-24T11:15:22.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-24T11:15:22.705 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:15:22.705 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-24T11:15:22.779 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-24T11:15:22.826 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-24T11:15:22.882 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-24T11:15:22.969 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-24T11:15:23.014 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:15:23.014 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-24T11:15:23.083 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-05-24T11:15:23.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-05-24T11:15:23.088 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-05-24T11:15:23.129 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-24T11:15:23.140 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-24T11:15:23.165 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-24T11:15:23.574 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-24T11:15:23.583 INFO:teuthology.task.internal:Starting timer... 2024-05-24T11:15:23.583 INFO:teuthology.run_tasks:Running task pcp... 2024-05-24T11:15:23.599 INFO:teuthology.run_tasks:Running task selinux... 2024-05-24T11:15:23.613 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-05-24T11:15:23.613 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2024-05-24T11:15:23.692 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2024-05-24T11:15:23.694 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-05-24T11:15:23.754 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-05-24T11:15:23.756 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-05-24T11:15:23.841 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-05-24T11:15:23.843 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-24T11:15:23.843 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2024-05-24T11:15:23.880 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2024-05-24T11:15:23.881 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-05-24T11:15:23.904 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-05-24T11:15:23.905 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-05-24T11:15:23.923 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-05-24T11:15:23.923 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive'} 2024-05-24T11:15:23.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-24T11:15:23.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T11:15:23.952 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-24T11:15:23.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T11:15:23.982 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-24T11:15:24.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T11:15:24.009 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-24T11:15:24.009 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2024-05-24T11:15:24.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-05-24T11:15:24.083 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-05-24T11:15:24.128 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-24T11:15:24.137 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-24T11:15:24.148 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-24T11:15:24.148 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi053.front.sepia.ceph.com,smithi137.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-24T11:26:07.534 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com')] 2024-05-24T11:26:07.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-05-24T11:26:07.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-24T11:26:07.623 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-05-24T11:26:07.697 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-05-24T11:26:07.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-24T11:26:07.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-24T11:26:07.777 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-05-24T11:26:07.851 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-05-24T11:26:07.851 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-05-24T11:26:07.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-24T11:26:07.927 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-05-24T11:26:07.995 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-05-24T11:26:07.997 INFO:teuthology.run_tasks:Running task clock... 2024-05-24T11:26:08.007 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-24T11:26:08.007 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-24T11:26:08.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-24T11:26:08.010 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-24T11:26:08.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-24T11:26:08.012 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-24T11:26:08.012 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-24T11:26:08.045 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-24T11:26:08.051 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-24T11:26:08.062 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-24T11:26:08.068 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-24T11:26:08.070 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-24T11:26:08.083 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-24T11:26:08.133 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2024-05-24T11:26:08.147 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2024-05-24T11:26:08.151 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-05-24T11:26:08.161 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-05-24T11:26:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-24T11:26:08.167 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-05-24T11:26:08.172 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-05-24T11:26:08.177 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-24T11:26:08.185 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-24T11:26:08.187 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-24T11:26:08.201 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-24T11:26:08.202 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-24T11:26:08.257 INFO:teuthology.orchestra.run.smithi039.stderr:bash: ntpq: command not found 2024-05-24T11:26:08.262 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-24T11:26:08.262 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-05-24T11:26:08.262 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.262 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.262 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.262 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.268 INFO:teuthology.orchestra.run.smithi137.stderr:bash: ntpq: command not found 2024-05-24T11:26:08.268 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2024-05-24T11:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-24T11:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-05-24T11:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.272 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.273 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-24T11:26:08.273 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-05-24T11:26:08.273 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.273 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.274 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.274 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T11:26:08.274 INFO:teuthology.run_tasks:Running task pexec... 2024-05-24T11:26:08.283 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-24T11:26:08.283 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:26:08.283 DEBUG:teuthology.orchestra.run.smithi039:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-24T11:26:08.284 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi053.front.sepia.ceph.com 2024-05-24T11:26:08.284 DEBUG:teuthology.orchestra.run.smithi053:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-24T11:26:08.284 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi137.front.sepia.ceph.com 2024-05-24T11:26:08.284 DEBUG:teuthology.orchestra.run.smithi137:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-24T11:26:08.689 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2024-05-24T11:26:08.699 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2024-05-24T11:26:08.749 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-24T11:26:10.503 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-05-24T11:26:10.602 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-05-24T11:26:10.654 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 20 kB/s | 4.1 kB 00:00 2024-05-24T11:26:10.882 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 28 kB/s | 4.5 kB 00:00 2024-05-24T11:26:10.965 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 kB/s | 4.5 kB 00:00 2024-05-24T11:26:11.391 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-05-24T11:26:14.110 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-24T11:26:14.112 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-24T11:26:14.112 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-05-24T11:26:14.112 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-24T11:26:14.112 INFO:teuthology.orchestra.run.smithi137.stdout:Resetting modules: 2024-05-24T11:26:14.112 INFO:teuthology.orchestra.run.smithi137.stdout: container-tools 2024-05-24T11:26:14.112 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:26:14.112 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-24T11:26:14.112 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-05-24T11:26:14.112 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:26:14.384 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-05-24T11:26:14.385 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-05-24T11:26:14.385 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-05-24T11:26:14.385 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-05-24T11:26:14.386 INFO:teuthology.orchestra.run.smithi039.stdout:Resetting modules: 2024-05-24T11:26:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: container-tools 2024-05-24T11:26:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:26:14.386 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-05-24T11:26:14.386 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-05-24T11:26:14.386 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:26:14.823 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-24T11:26:14.825 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-05-24T11:26:14.825 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-05-24T11:26:14.825 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-05-24T11:26:14.825 INFO:teuthology.orchestra.run.smithi053.stdout:Resetting modules: 2024-05-24T11:26:14.825 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools 2024-05-24T11:26:14.825 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:26:14.825 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-05-24T11:26:14.825 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-05-24T11:26:14.825 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:26:21.936 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-24T11:26:22.389 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2024-05-24T11:26:24.242 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-05-24T11:26:24.655 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 24 kB/s | 4.5 kB 00:00 2024-05-24T11:26:26.091 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-05-24T11:26:26.573 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2024-05-24T11:26:27.818 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-24T11:26:27.989 INFO:teuthology.orchestra.run.smithi137.stdout:========================================================================================================== 2024-05-24T11:26:27.989 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-24T11:26:27.989 INFO:teuthology.orchestra.run.smithi137.stdout:========================================================================================================== 2024-05-24T11:26:27.989 INFO:teuthology.orchestra.run.smithi137.stdout:Installing group/module packages: 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-24T11:26:27.990 INFO:teuthology.orchestra.run.smithi137.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout:Installing module profiles: 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout: container-tools/common 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling module streams: 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout: container-tools rhel8 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout:========================================================================================================== 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout:Install 13 Packages 2024-05-24T11:26:27.991 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:26:27.992 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 26 M 2024-05-24T11:26:27.992 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 87 M 2024-05-24T11:26:27.992 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-24T11:26:28.140 INFO:teuthology.orchestra.run.smithi137.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 635 kB/s | 94 kB 00:00 2024-05-24T11:26:28.165 INFO:teuthology.orchestra.run.smithi137.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 143 kB/s | 25 kB 00:00 2024-05-24T11:26:28.190 INFO:teuthology.orchestra.run.smithi137.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 198 kB/s | 39 kB 00:00 2024-05-24T11:26:28.274 INFO:teuthology.orchestra.run.smithi137.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 306 kB/s | 41 kB 00:00 2024-05-24T11:26:28.333 INFO:teuthology.orchestra.run.smithi137.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.0 MB/s | 1.0 MB 00:00 2024-05-24T11:26:28.517 INFO:teuthology.orchestra.run.smithi137.stdout:(6/13): buildah-1.31.5-1.module+el8.9.0+21697+6 27 MB/s | 8.8 MB 00:00 2024-05-24T11:26:28.542 INFO:teuthology.orchestra.run.smithi137.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.1 MB/s | 239 kB 00:00 2024-05-24T11:26:28.567 INFO:teuthology.orchestra.run.smithi137.stdout:(8/13): cockpit-podman-75-1.module+el8.9.0+2169 2.5 MB/s | 739 kB 00:00 2024-05-24T11:26:28.663 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-05-24T11:26:28.742 INFO:teuthology.orchestra.run.smithi137.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 16 MB/s | 3.7 MB 00:00 2024-05-24T11:26:28.759 INFO:teuthology.orchestra.run.smithi137.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 707 kB/s | 154 kB 00:00 2024-05-24T11:26:28.868 INFO:teuthology.orchestra.run.smithi137.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 20 MB/s | 2.5 MB 00:00 2024-05-24T11:26:28.901 INFO:teuthology.orchestra.run.smithi137.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 348 kB/s | 49 kB 00:00 2024-05-24T11:26:29.018 INFO:teuthology.orchestra.run.smithi137.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 18 MB/s | 8.1 MB 00:00 2024-05-24T11:26:29.019 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-24T11:26:29.019 INFO:teuthology.orchestra.run.smithi137.stdout:Total 25 MB/s | 26 MB 00:01 2024-05-24T11:26:29.054 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-05-24T11:26:29.303 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-24T11:26:29.324 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-24T11:26:29.324 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-24T11:26:29.354 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-24T11:26:29.555 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-24T11:26:29.564 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-24T11:26:29.888 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-24T11:26:30.305 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-24T11:26:30.787 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-24T11:26:30.981 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-24T11:26:31.126 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-24T11:26:31.269 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-24T11:26:31.448 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-24T11:26:31.505 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 28 kB/s | 4.5 kB 00:00 2024-05-24T11:26:31.640 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-24T11:26:31.909 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-24T11:26:32.305 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2024-05-24T11:26:32.494 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-05-24T11:26:32.672 INFO:teuthology.orchestra.run.smithi039.stdout:========================================================================================================== 2024-05-24T11:26:32.672 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-05-24T11:26:32.672 INFO:teuthology.orchestra.run.smithi039.stdout:========================================================================================================== 2024-05-24T11:26:32.672 INFO:teuthology.orchestra.run.smithi039.stdout:Installing group/module packages: 2024-05-24T11:26:32.672 INFO:teuthology.orchestra.run.smithi039.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-24T11:26:32.672 INFO:teuthology.orchestra.run.smithi039.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout:Installing weak dependencies: 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-24T11:26:32.673 INFO:teuthology.orchestra.run.smithi039.stdout:Installing module profiles: 2024-05-24T11:26:32.674 INFO:teuthology.orchestra.run.smithi039.stdout: container-tools/common 2024-05-24T11:26:32.674 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:26:32.674 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling module streams: 2024-05-24T11:26:32.674 INFO:teuthology.orchestra.run.smithi039.stdout: container-tools rhel8 2024-05-24T11:26:32.674 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:26:32.674 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-05-24T11:26:32.674 INFO:teuthology.orchestra.run.smithi039.stdout:========================================================================================================== 2024-05-24T11:26:32.674 INFO:teuthology.orchestra.run.smithi039.stdout:Install 13 Packages 2024-05-24T11:26:32.674 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:26:32.675 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 26 M 2024-05-24T11:26:32.675 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 87 M 2024-05-24T11:26:32.675 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-05-24T11:26:32.797 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-24T11:26:32.846 INFO:teuthology.orchestra.run.smithi039.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 231 kB/s | 39 kB 00:00 2024-05-24T11:26:32.871 INFO:teuthology.orchestra.run.smithi039.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 126 kB/s | 25 kB 00:00 2024-05-24T11:26:32.896 INFO:teuthology.orchestra.run.smithi039.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 424 kB/s | 94 kB 00:00 2024-05-24T11:26:32.972 INFO:teuthology.orchestra.run.smithi039.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 327 kB/s | 41 kB 00:00 2024-05-24T11:26:33.022 INFO:teuthology.orchestra.run.smithi039.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.7 MB/s | 1.0 MB 00:00 2024-05-24T11:26:33.053 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-24T11:26:33.072 INFO:teuthology.orchestra.run.smithi039.stdout:(6/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 4.7 MB/s | 239 kB 00:00 2024-05-24T11:26:33.147 INFO:teuthology.orchestra.run.smithi039.stdout:(7/13): cockpit-podman-75-1.module+el8.9.0+2169 4.1 MB/s | 739 kB 00:00 2024-05-24T11:26:33.298 INFO:teuthology.orchestra.run.smithi039.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 22 MB/s | 8.8 MB 00:00 2024-05-24T11:26:33.365 INFO:teuthology.orchestra.run.smithi039.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 13 MB/s | 3.7 MB 00:00 2024-05-24T11:26:33.382 INFO:teuthology.orchestra.run.smithi039.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 655 kB/s | 154 kB 00:00 2024-05-24T11:26:33.435 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-24T11:26:33.532 INFO:teuthology.orchestra.run.smithi039.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 15 MB/s | 2.5 MB 00:00 2024-05-24T11:26:33.649 INFO:teuthology.orchestra.run.smithi039.stdout:(12/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 23 MB/s | 8.1 MB 00:00 2024-05-24T11:26:33.674 INFO:teuthology.orchestra.run.smithi039.stdout:(13/13): udica-0.2.6-20.module+el8.9.0+21697+6a 169 kB/s | 49 kB 00:00 2024-05-24T11:26:33.675 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-05-24T11:26:33.675 INFO:teuthology.orchestra.run.smithi039.stdout:Total 26 MB/s | 26 MB 00:01 2024-05-24T11:26:33.967 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-05-24T11:26:33.990 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-05-24T11:26:33.990 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-05-24T11:26:34.234 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-05-24T11:26:34.244 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-05-24T11:26:34.301 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-24T11:26:34.514 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-24T11:26:34.696 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-24T11:26:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-05-24T11:26:35.124 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-24T11:26:35.124 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-24T11:26:35.125 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-24T11:26:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-24T11:26:35.686 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-24T11:26:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-24T11:26:35.874 INFO:teuthology.orchestra.run.smithi053.stdout:========================================================================================================== 2024-05-24T11:26:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-05-24T11:26:35.875 INFO:teuthology.orchestra.run.smithi053.stdout:========================================================================================================== 2024-05-24T11:26:35.875 INFO:teuthology.orchestra.run.smithi053.stdout:Installing group/module packages: 2024-05-24T11:26:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-24T11:26:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-24T11:26:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-24T11:26:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-24T11:26:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-24T11:26:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-24T11:26:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-24T11:26:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-24T11:26:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-24T11:26:35.877 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-05-24T11:26:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-24T11:26:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-24T11:26:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-24T11:26:35.878 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-05-24T11:26:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-24T11:26:35.878 INFO:teuthology.orchestra.run.smithi053.stdout:Installing module profiles: 2024-05-24T11:26:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools/common 2024-05-24T11:26:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:26:35.878 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2024-05-24T11:26:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools rhel8 2024-05-24T11:26:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:26:35.879 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-05-24T11:26:35.879 INFO:teuthology.orchestra.run.smithi053.stdout:========================================================================================================== 2024-05-24T11:26:35.879 INFO:teuthology.orchestra.run.smithi053.stdout:Install 13 Packages 2024-05-24T11:26:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:26:35.880 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 26 M 2024-05-24T11:26:35.881 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 87 M 2024-05-24T11:26:35.881 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-05-24T11:26:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-24T11:26:36.093 INFO:teuthology.orchestra.run.smithi053.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 115 kB/s | 25 kB 00:00 2024-05-24T11:26:36.118 INFO:teuthology.orchestra.run.smithi053.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 164 kB/s | 39 kB 00:00 2024-05-24T11:26:36.152 INFO:teuthology.orchestra.run.smithi053.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 343 kB/s | 94 kB 00:00 2024-05-24T11:26:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-24T11:26:36.185 INFO:teuthology.orchestra.run.smithi053.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 445 kB/s | 41 kB 00:00 2024-05-24T11:26:36.261 INFO:teuthology.orchestra.run.smithi053.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 7.1 MB/s | 1.0 MB 00:00 2024-05-24T11:26:36.353 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-24T11:26:36.397 INFO:teuthology.orchestra.run.smithi053.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 3.4 MB/s | 739 kB 00:00 2024-05-24T11:26:36.436 INFO:teuthology.orchestra.run.smithi053.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.3 MB/s | 239 kB 00:00 2024-05-24T11:26:36.604 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-24T11:26:36.628 INFO:teuthology.orchestra.run.smithi053.stdout:(8/13): python3-podman-4.6.0-2.module+el8.9.0+2 802 kB/s | 154 kB 00:00 2024-05-24T11:26:36.745 INFO:teuthology.orchestra.run.smithi053.stdout:(9/13): buildah-1.31.5-1.module+el8.9.0+21697+6 15 MB/s | 8.8 MB 00:00 2024-05-24T11:26:36.849 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-24T11:26:36.862 INFO:teuthology.orchestra.run.smithi053.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 8.0 MB/s | 3.7 MB 00:00 2024-05-24T11:26:36.946 INFO:teuthology.orchestra.run.smithi053.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 13 MB/s | 2.5 MB 00:00 2024-05-24T11:26:37.004 INFO:teuthology.orchestra.run.smithi053.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 349 kB/s | 49 kB 00:00 2024-05-24T11:26:37.280 INFO:teuthology.orchestra.run.smithi053.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 13 MB/s | 8.1 MB 00:00 2024-05-24T11:26:37.280 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-05-24T11:26:37.280 INFO:teuthology.orchestra.run.smithi053.stdout:Total 18 MB/s | 26 MB 00:01 2024-05-24T11:26:37.593 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-05-24T11:26:37.617 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-05-24T11:26:37.617 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-05-24T11:26:37.781 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-24T11:26:37.874 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-05-24T11:26:37.884 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-05-24T11:26:38.062 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-24T11:26:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-24T11:26:38.686 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-05-24T11:26:39.213 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-24T11:26:39.411 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-24T11:26:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-24T11:26:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-24T11:26:39.654 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-24T11:26:39.749 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-24T11:26:39.868 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-24T11:26:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-24T11:26:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-24T11:26:40.298 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-24T11:26:40.508 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-24T11:26:40.509 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-24T11:26:40.509 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-24T11:26:40.520 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-24T11:26:41.577 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-24T11:26:41.936 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-24T11:26:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-24T11:26:43.316 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-24T11:26:43.526 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-24T11:26:43.731 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-24T11:26:44.179 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-24T11:26:44.179 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-24T11:26:44.179 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-24T11:26:44.179 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-24T11:26:44.179 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-24T11:26:44.180 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-24T11:26:44.180 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-24T11:26:44.180 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-24T11:26:44.180 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-24T11:26:44.180 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-24T11:26:44.180 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-24T11:26:44.180 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-24T11:26:44.180 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-24T11:26:47.341 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-24T11:26:47.341 INFO:teuthology.orchestra.run.smithi137.stdout:Installed products updated. 2024-05-24T11:26:47.914 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:47.915 INFO:teuthology.orchestra.run.smithi137.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:47.916 INFO:teuthology.orchestra.run.smithi137.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-24T11:26:47.916 INFO:teuthology.orchestra.run.smithi137.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-24T11:26:47.916 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:26:47.916 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-24T11:26:48.085 DEBUG:teuthology.parallel:result is None 2024-05-24T11:26:52.467 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-24T11:26:52.468 INFO:teuthology.orchestra.run.smithi039.stdout:Installed products updated. 2024-05-24T11:26:53.105 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:26:53.105 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-05-24T11:26:53.105 INFO:teuthology.orchestra.run.smithi039.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:53.105 INFO:teuthology.orchestra.run.smithi039.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:53.105 INFO:teuthology.orchestra.run.smithi039.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-24T11:26:53.105 INFO:teuthology.orchestra.run.smithi039.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-24T11:26:53.105 INFO:teuthology.orchestra.run.smithi039.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:53.105 INFO:teuthology.orchestra.run.smithi039.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:53.105 INFO:teuthology.orchestra.run.smithi039.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-24T11:26:53.106 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-24T11:26:53.106 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-24T11:26:53.106 INFO:teuthology.orchestra.run.smithi039.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:53.106 INFO:teuthology.orchestra.run.smithi039.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:53.106 INFO:teuthology.orchestra.run.smithi039.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-24T11:26:53.106 INFO:teuthology.orchestra.run.smithi039.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-24T11:26:53.106 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:26:53.106 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-05-24T11:26:53.274 DEBUG:teuthology.parallel:result is None 2024-05-24T11:26:56.491 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-24T11:26:56.492 INFO:teuthology.orchestra.run.smithi053.stdout:Installed products updated. 2024-05-24T11:26:57.152 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:26:57.152 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-05-24T11:26:57.152 INFO:teuthology.orchestra.run.smithi053.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:26:57.153 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-24T11:26:57.338 DEBUG:teuthology.parallel:result is None 2024-05-24T11:26:57.338 INFO:teuthology.run_tasks:Running task install... 2024-05-24T11:26:57.347 DEBUG:teuthology.task.install:project ceph 2024-05-24T11:26:57.347 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f'}} 2024-05-24T11:26:57.347 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f'} 2024-05-24T11:26:57.347 INFO:teuthology.task.install:Using flavor: default 2024-05-24T11:26:57.353 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-24T11:26:57.353 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-24T11:26:57.353 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f', 'tag': None, 'wait_for_package': False} 2024-05-24T11:26:57.354 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:26:57.356 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f', 'tag': None, 'wait_for_package': False} 2024-05-24T11:26:57.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:26:57.357 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f', 'tag': None, 'wait_for_package': False} 2024-05-24T11:26:57.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:26:57.508 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/ 2024-05-24T11:26:57.508 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1349.gb9b067bc 2024-05-24T11:26:57.513 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/ 2024-05-24T11:26:57.513 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1349.gb9b067bc 2024-05-24T11:26:57.580 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/ 2024-05-24T11:26:57.580 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1349.gb9b067bc 2024-05-24T11:26:57.629 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-24T11:26:57.630 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:26:57.630 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-24T11:26:57.664 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-24T11:26:57.665 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-24T11:26:57.680 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-24T11:26:57.680 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:26:57.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-24T11:26:57.688 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-24T11:26:57.688 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:26:57.688 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-24T11:26:57.716 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-24T11:26:57.716 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-24T11:26:57.723 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-24T11:26:57.724 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-24T11:26:57.736 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-24T11:26:57.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-24T11:26:57.796 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-24T11:26:57.817 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-24T11:26:57.879 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-24T11:26:57.880 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-24T11:26:57.886 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-05-24T11:26:57.887 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-05-24T11:26:57.913 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-05-24T11:26:57.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-05-24T11:26:57.947 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2024-05-24T11:26:57.949 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2024-05-24T11:26:58.266 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2024-05-24T11:26:58.350 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2024-05-24T11:26:58.350 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-24T11:26:59.669 INFO:teuthology.orchestra.run.smithi039.stdout:47 files removed 2024-05-24T11:26:59.724 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-24T11:27:00.052 INFO:teuthology.orchestra.run.smithi053.stdout:47 files removed 2024-05-24T11:27:00.080 INFO:teuthology.orchestra.run.smithi137.stdout:47 files removed 2024-05-24T11:27:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2024-05-24T11:27:00.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-24T11:27:00.137 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-24T11:27:00.475 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2024-05-24T11:27:00.492 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2024-05-24T11:27:01.897 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 221 kB/s | 80 kB 00:00 2024-05-24T11:27:02.231 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 38 kB/s | 11 kB 00:00 2024-05-24T11:27:02.268 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 243 kB/s | 80 kB 00:00 2024-05-24T11:27:02.323 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 218 kB/s | 80 kB 00:00 2024-05-24T11:27:02.548 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 5.1 kB/s | 1.5 kB 00:00 2024-05-24T11:27:02.582 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 51 kB/s | 11 kB 00:00 2024-05-24T11:27:02.669 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-05-24T11:27:02.824 INFO:teuthology.orchestra.run.smithi039.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-05-24T11:27:03.061 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2024-05-24T11:27:03.066 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 3.3 kB/s | 1.5 kB 00:00 2024-05-24T11:27:03.251 INFO:teuthology.orchestra.run.smithi137.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-24T11:27:03.278 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-05-24T11:27:05.104 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 14 MB 00:01 2024-05-24T11:27:05.893 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 14 MB 00:02 2024-05-24T11:27:05.991 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 14 MB 00:03 2024-05-24T11:27:09.596 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 442 kB/s | 24 kB 00:00 2024-05-24T11:27:10.035 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 410 kB/s | 24 kB 00:00 2024-05-24T11:27:10.283 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 432 kB/s | 24 kB 00:00 2024-05-24T11:27:11.056 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 61 MB 00:01 2024-05-24T11:27:11.386 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 53 MB/s | 67 MB 00:01 2024-05-24T11:27:11.977 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 67 MB 00:01 2024-05-24T11:27:22.545 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 61 MB 00:01 2024-05-24T11:27:23.373 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 8.4 MB/s | 8.8 MB 00:01 2024-05-24T11:27:23.628 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 61 MB 00:01 2024-05-24T11:27:26.533 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 67 MB 00:01 2024-05-24T11:27:33.678 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-05-24T11:27:35.079 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-24T11:27:40.784 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-24T11:27:40.785 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-24T11:27:40.803 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-24T11:27:40.804 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-24T11:27:40.806 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-24T11:27:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-24T11:27:40.810 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-24T11:27:40.814 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-24T11:27:40.815 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-24T11:27:41.043 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-05-24T11:27:41.057 INFO:teuthology.orchestra.run.smithi137.stdout:=================================================================================================================================== 2024-05-24T11:27:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-05-24T11:27:41.058 INFO:teuthology.orchestra.run.smithi137.stdout:=================================================================================================================================== 2024-05-24T11:27:41.058 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-05-24T11:27:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-24T11:27:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-24T11:27:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 6.4 k 2024-05-24T11:27:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 5.5 M 2024-05-24T11:27:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 859 k 2024-05-24T11:27:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 165 k 2024-05-24T11:27:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 1.6 M 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 138 k 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 3.5 M 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 7.4 M 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 49 k 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 13 M 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 47 M 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 263 k 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 173 k 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 261 k 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 18 k 2024-05-24T11:27:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 31 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 781 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 139 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 197 k 2024-05-24T11:27:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 371 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 365 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 107 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 95 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 3.5 M 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 181 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-24T11:27:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 3.7 M 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 3.6 M 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-24T11:27:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 22 M 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 24 k 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 2.3 M 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 246 k 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 4.8 M 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 18 M 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 16 k 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 24 k 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-24T11:27:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 142 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 473 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 6.6 M 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-24T11:27:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-24T11:27:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 45 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 129 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-24T11:27:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-24T11:27:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-24T11:27:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling module streams: 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql 10 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:27:41.072 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-05-24T11:27:41.073 INFO:teuthology.orchestra.run.smithi137.stdout:=================================================================================================================================== 2024-05-24T11:27:41.073 INFO:teuthology.orchestra.run.smithi137.stdout:Install 130 Packages 2024-05-24T11:27:41.073 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 16 Packages 2024-05-24T11:27:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:27:41.073 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 215 M 2024-05-24T11:27:41.073 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-05-24T11:27:41.321 INFO:teuthology.orchestra.run.smithi137.stdout:(1/146): ceph-18.2.2-1349.gb9b067bc.el8.x86_64. 52 kB/s | 6.4 kB 00:00 2024-05-24T11:27:41.613 INFO:teuthology.orchestra.run.smithi137.stdout:(2/146): ceph-fuse-18.2.2-1349.gb9b067bc.el8.x8 2.9 MB/s | 859 kB 00:00 2024-05-24T11:27:41.721 INFO:teuthology.orchestra.run.smithi137.stdout:(3/146): ceph-immutable-object-cache-18.2.2-134 1.5 MB/s | 165 kB 00:00 2024-05-24T11:27:42.555 INFO:teuthology.orchestra.run.smithi137.stdout:(4/146): ceph-common-18.2.2-1349.gb9b067bc.el8. 16 MB/s | 22 MB 00:01 2024-05-24T11:27:42.596 INFO:teuthology.orchestra.run.smithi137.stdout:(5/146): ceph-mds-18.2.2-1349.gb9b067bc.el8.x86 2.6 MB/s | 2.3 MB 00:00 2024-05-24T11:27:42.620 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-24T11:27:42.621 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-24T11:27:42.640 INFO:teuthology.orchestra.run.smithi039.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-24T11:27:42.642 INFO:teuthology.orchestra.run.smithi039.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-24T11:27:42.643 INFO:teuthology.orchestra.run.smithi039.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-24T11:27:42.645 INFO:teuthology.orchestra.run.smithi039.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-24T11:27:42.647 INFO:teuthology.orchestra.run.smithi039.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-24T11:27:42.651 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-24T11:27:42.653 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-24T11:27:42.697 INFO:teuthology.orchestra.run.smithi137.stdout:(6/146): ceph-mgr-18.2.2-1349.gb9b067bc.el8.x86 11 MB/s | 1.6 MB 00:00 2024-05-24T11:27:42.772 INFO:teuthology.orchestra.run.smithi137.stdout:(7/146): ceph-base-18.2.2-1349.gb9b067bc.el8.x8 3.5 MB/s | 5.5 MB 00:01 2024-05-24T11:27:42.888 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-05-24T11:27:42.903 INFO:teuthology.orchestra.run.smithi039.stdout:=================================================================================================================================== 2024-05-24T11:27:42.903 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout:=================================================================================================================================== 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 6.4 k 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 5.5 M 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 859 k 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 165 k 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 1.6 M 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 138 k 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 3.5 M 2024-05-24T11:27:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 7.4 M 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 49 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 13 M 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 47 M 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 263 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 173 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 261 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 18 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 31 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 781 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 139 k 2024-05-24T11:27:42.905 INFO:teuthology.orchestra.run.smithi039.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 197 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 371 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 365 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 107 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 95 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 3.5 M 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 181 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-24T11:27:42.906 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 3.7 M 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 3.6 M 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-24T11:27:42.907 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 22 M 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 24 k 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 2.3 M 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 246 k 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 4.8 M 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 18 M 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 16 k 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 24 k 2024-05-24T11:27:42.908 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 142 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 473 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-24T11:27:42.909 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 6.6 M 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-24T11:27:42.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 45 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 129 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-24T11:27:42.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-24T11:27:42.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-24T11:27:42.913 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-24T11:27:42.914 INFO:teuthology.orchestra.run.smithi039.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling module streams: 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql 10 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout:=================================================================================================================================== 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout:Install 130 Packages 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 16 Packages 2024-05-24T11:27:42.915 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:27:42.916 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 215 M 2024-05-24T11:27:42.916 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-05-24T11:27:43.479 INFO:teuthology.orchestra.run.smithi039.stdout:(1/146): ceph-18.2.2-1349.gb9b067bc.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2024-05-24T11:27:43.797 INFO:teuthology.orchestra.run.smithi137.stdout:(8/146): ceph-osd-18.2.2-1349.gb9b067bc.el8.x86 17 MB/s | 18 MB 00:01 2024-05-24T11:27:43.891 INFO:teuthology.orchestra.run.smithi137.stdout:(9/146): ceph-mon-18.2.2-1349.gb9b067bc.el8.x86 3.7 MB/s | 4.8 MB 00:01 2024-05-24T11:27:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:(2/146): ceph-fuse-18.2.2-1349.gb9b067bc.el8.x8 2.0 MB/s | 859 kB 00:00 2024-05-24T11:27:43.914 INFO:teuthology.orchestra.run.smithi137.stdout:(10/146): ceph-selinux-18.2.2-1349.gb9b067bc.el 208 kB/s | 24 kB 00:00 2024-05-24T11:27:43.936 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-24T11:27:43.937 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-24T11:27:43.958 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-24T11:27:43.959 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-24T11:27:43.961 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-24T11:27:43.962 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-24T11:27:43.965 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-24T11:27:43.970 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-24T11:27:43.971 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-24T11:27:43.988 INFO:teuthology.orchestra.run.smithi039.stdout:(3/146): ceph-immutable-object-cache-18.2.2-134 1.8 MB/s | 165 kB 00:00 2024-05-24T11:27:44.015 INFO:teuthology.orchestra.run.smithi137.stdout:(11/146): cephfs-mirror-18.2.2-1349.gb9b067bc.e 2.5 MB/s | 261 kB 00:00 2024-05-24T11:27:44.090 INFO:teuthology.orchestra.run.smithi137.stdout:(12/146): libcephfs-devel-18.2.2-1349.gb9b067bc 413 kB/s | 31 kB 00:00 2024-05-24T11:27:44.228 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout:=================================================================================================================================== 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout:=================================================================================================================================== 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 6.4 k 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 5.5 M 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 859 k 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 165 k 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 1.6 M 2024-05-24T11:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 138 k 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 3.5 M 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 7.4 M 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 49 k 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 13 M 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 47 M 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 263 k 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 173 k 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 261 k 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 18 k 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-24T11:27:44.246 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 31 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 781 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 139 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 197 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 371 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 365 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 107 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 95 k 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 3.5 M 2024-05-24T11:27:44.247 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 181 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 3.7 M 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 3.6 M 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-24T11:27:44.248 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 22 M 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 24 k 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 2.3 M 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 246 k 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 4.8 M 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 18 M 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1349.gb9b067bc.el8 ceph-noarch 16 k 2024-05-24T11:27:44.249 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 24 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 142 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-24T11:27:44.250 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 473 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 6.6 M 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-24T11:27:44.251 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 45 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:18.2.2-1349.gb9b067bc.el8 ceph 129 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-24T11:27:44.252 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-24T11:27:44.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-24T11:27:44.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-24T11:27:44.255 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql 10 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout:=================================================================================================================================== 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout:Install 130 Packages 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 16 Packages 2024-05-24T11:27:44.256 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:27:44.257 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 215 M 2024-05-24T11:27:44.257 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-05-24T11:27:44.488 INFO:teuthology.orchestra.run.smithi039.stdout:(4/146): ceph-base-18.2.2-1349.gb9b067bc.el8.x8 4.0 MB/s | 5.5 MB 00:01 2024-05-24T11:27:44.547 INFO:teuthology.orchestra.run.smithi039.stdout:(5/146): ceph-mds-18.2.2-1349.gb9b067bc.el8.x86 4.1 MB/s | 2.3 MB 00:00 2024-05-24T11:27:44.575 INFO:teuthology.orchestra.run.smithi053.stdout:(1/146): ceph-18.2.2-1349.gb9b067bc.el8.x86_64. 27 kB/s | 6.4 kB 00:00 2024-05-24T11:27:44.674 INFO:teuthology.orchestra.run.smithi137.stdout:(13/146): libcephfs2-18.2.2-1349.gb9b067bc.el8. 1.3 MB/s | 781 kB 00:00 2024-05-24T11:27:44.774 INFO:teuthology.orchestra.run.smithi137.stdout:(14/146): libcephsqlite-18.2.2-1349.gb9b067bc.e 1.4 MB/s | 142 kB 00:00 2024-05-24T11:27:44.814 INFO:teuthology.orchestra.run.smithi039.stdout:(6/146): ceph-mgr-18.2.2-1349.gb9b067bc.el8.x86 4.8 MB/s | 1.6 MB 00:00 2024-05-24T11:27:44.857 INFO:teuthology.orchestra.run.smithi137.stdout:(15/146): librados-devel-18.2.2-1349.gb9b067bc. 1.6 MB/s | 139 kB 00:00 2024-05-24T11:27:45.375 INFO:teuthology.orchestra.run.smithi137.stdout:(16/146): ceph-radosgw-18.2.2-1349.gb9b067bc.el 5.0 MB/s | 13 MB 00:02 2024-05-24T11:27:45.400 INFO:teuthology.orchestra.run.smithi137.stdout:(17/146): libradosstriper1-18.2.2-1349.gb9b067b 874 kB/s | 473 kB 00:00 2024-05-24T11:27:45.406 INFO:teuthology.orchestra.run.smithi039.stdout:(7/146): ceph-mon-18.2.2-1349.gb9b067bc.el8.x86 5.6 MB/s | 4.8 MB 00:00 2024-05-24T11:27:45.442 INFO:teuthology.orchestra.run.smithi137.stdout:(18/146): python3-ceph-argparse-18.2.2-1349.gb9 1.0 MB/s | 45 kB 00:00 2024-05-24T11:27:45.495 INFO:teuthology.orchestra.run.smithi053.stdout:(2/146): ceph-fuse-18.2.2-1349.gb9b067bc.el8.x8 934 kB/s | 859 kB 00:00 2024-05-24T11:27:45.550 INFO:teuthology.orchestra.run.smithi137.stdout:(19/146): python3-ceph-common-18.2.2-1349.gb9b0 1.2 MB/s | 129 kB 00:00 2024-05-24T11:27:45.642 INFO:teuthology.orchestra.run.smithi137.stdout:(20/146): python3-cephfs-18.2.2-1349.gb9b067bc. 2.1 MB/s | 197 kB 00:00 2024-05-24T11:27:45.684 INFO:teuthology.orchestra.run.smithi053.stdout:(3/146): ceph-immutable-object-cache-18.2.2-134 882 kB/s | 165 kB 00:00 2024-05-24T11:27:45.734 INFO:teuthology.orchestra.run.smithi137.stdout:(21/146): python3-rados-18.2.2-1349.gb9b067bc.e 4.0 MB/s | 371 kB 00:00 2024-05-24T11:27:45.834 INFO:teuthology.orchestra.run.smithi137.stdout:(22/146): python3-rbd-18.2.2-1349.gb9b067bc.el8 3.6 MB/s | 365 kB 00:00 2024-05-24T11:27:45.943 INFO:teuthology.orchestra.run.smithi137.stdout:(23/146): python3-rgw-18.2.2-1349.gb9b067bc.el8 986 kB/s | 107 kB 00:00 2024-05-24T11:27:46.045 INFO:teuthology.orchestra.run.smithi137.stdout:(24/146): rbd-fuse-18.2.2-1349.gb9b067bc.el8.x8 932 kB/s | 95 kB 00:00 2024-05-24T11:27:46.712 INFO:teuthology.orchestra.run.smithi137.stdout:(25/146): rbd-mirror-18.2.2-1349.gb9b067bc.el8. 5.2 MB/s | 3.5 MB 00:00 2024-05-24T11:27:46.796 INFO:teuthology.orchestra.run.smithi137.stdout:(26/146): rbd-nbd-18.2.2-1349.gb9b067bc.el8.x86 2.1 MB/s | 181 kB 00:00 2024-05-24T11:27:46.904 INFO:teuthology.orchestra.run.smithi137.stdout:(27/146): librgw2-18.2.2-1349.gb9b067bc.el8.x86 4.3 MB/s | 6.6 MB 00:01 2024-05-24T11:27:46.930 INFO:teuthology.orchestra.run.smithi137.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1349.g 182 kB/s | 24 kB 00:00 2024-05-24T11:27:46.980 INFO:teuthology.orchestra.run.smithi137.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1349.gb9b067b 1.8 MB/s | 138 kB 00:00 2024-05-24T11:27:47.367 INFO:teuthology.orchestra.run.smithi053.stdout:(4/146): ceph-mds-18.2.2-1349.gb9b067bc.el8.x86 1.3 MB/s | 2.3 MB 00:01 2024-05-24T11:27:47.630 INFO:teuthology.orchestra.run.smithi137.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1349.gb9b06 5.0 MB/s | 3.5 MB 00:00 2024-05-24T11:27:47.714 INFO:teuthology.orchestra.run.smithi137.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1349.gb9 2.9 MB/s | 246 kB 00:00 2024-05-24T11:27:47.789 INFO:teuthology.orchestra.run.smithi137.stdout:(32/146): ceph-mgr-rook-18.2.2-1349.gb9b067bc.e 658 kB/s | 49 kB 00:00 2024-05-24T11:27:47.856 INFO:teuthology.orchestra.run.smithi137.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1349.gb 245 kB/s | 16 kB 00:00 2024-05-24T11:27:47.948 INFO:teuthology.orchestra.run.smithi137.stdout:(34/146): ceph-volume-18.2.2-1349.gb9b067bc.el8 2.8 MB/s | 263 kB 00:00 2024-05-24T11:27:48.032 INFO:teuthology.orchestra.run.smithi137.stdout:(35/146): cephadm-18.2.2-1349.gb9b067bc.el8.noa 2.0 MB/s | 173 kB 00:00 2024-05-24T11:27:48.098 INFO:teuthology.orchestra.run.smithi137.stdout:(36/146): cephfs-top-18.2.2-1349.gb9b067bc.el8. 272 kB/s | 18 kB 00:00 2024-05-24T11:27:48.257 INFO:teuthology.orchestra.run.smithi137.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.9 MB/s | 306 kB 00:00 2024-05-24T11:27:48.276 INFO:teuthology.orchestra.run.smithi053.stdout:(5/146): ceph-base-18.2.2-1349.gb9b067bc.el8.x8 1.4 MB/s | 5.5 MB 00:03 2024-05-24T11:27:48.299 INFO:teuthology.orchestra.run.smithi137.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 1.4 MB/s | 59 kB 00:00 2024-05-24T11:27:48.349 INFO:teuthology.orchestra.run.smithi137.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2024-05-24T11:27:48.389 INFO:teuthology.orchestra.run.smithi039.stdout:(8/146): ceph-radosgw-18.2.2-1349.gb9b067bc.el8 4.4 MB/s | 13 MB 00:02 2024-05-24T11:27:48.425 INFO:teuthology.orchestra.run.smithi137.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 6.2 MB/s | 476 kB 00:00 2024-05-24T11:27:48.443 INFO:teuthology.orchestra.run.smithi053.stdout:(6/146): ceph-mgr-18.2.2-1349.gb9b067bc.el8.x86 1.5 MB/s | 1.6 MB 00:01 2024-05-24T11:27:48.467 INFO:teuthology.orchestra.run.smithi137.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2024-05-24T11:27:48.483 INFO:teuthology.orchestra.run.smithi039.stdout:(9/146): ceph-selinux-18.2.2-1349.gb9b067bc.el8 260 kB/s | 24 kB 00:00 2024-05-24T11:27:48.500 INFO:teuthology.orchestra.run.smithi137.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 5.4 MB/s | 185 kB 00:00 2024-05-24T11:27:48.542 INFO:teuthology.orchestra.run.smithi137.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 792 kB/s | 33 kB 00:00 2024-05-24T11:27:48.576 INFO:teuthology.orchestra.run.smithi137.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 492 kB/s | 16 kB 00:00 2024-05-24T11:27:48.618 INFO:teuthology.orchestra.run.smithi137.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 4.1 MB/s | 173 kB 00:00 2024-05-24T11:27:48.668 INFO:teuthology.orchestra.run.smithi137.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2024-05-24T11:27:48.710 INFO:teuthology.orchestra.run.smithi137.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2024-05-24T11:27:48.744 INFO:teuthology.orchestra.run.smithi137.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 334 kB/s | 11 kB 00:00 2024-05-24T11:27:48.819 INFO:teuthology.orchestra.run.smithi137.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2024-05-24T11:27:48.833 INFO:teuthology.orchestra.run.smithi039.stdout:(10/146): ceph-common-18.2.2-1349.gb9b067bc.el8 3.8 MB/s | 22 MB 00:05 2024-05-24T11:27:48.886 INFO:teuthology.orchestra.run.smithi137.stdout:(50/146): python3-kubernetes-11.0.0-6.el8.noarc 18 MB/s | 1.2 MB 00:00 2024-05-24T11:27:48.928 INFO:teuthology.orchestra.run.smithi137.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2024-05-24T11:27:48.984 INFO:teuthology.orchestra.run.smithi039.stdout:(11/146): cephfs-mirror-18.2.2-1349.gb9b067bc.e 1.7 MB/s | 261 kB 00:00 2024-05-24T11:27:49.037 INFO:teuthology.orchestra.run.smithi137.stdout:(52/146): ceph-mgr-diskprediction-local-18.2.2- 3.6 MB/s | 7.4 MB 00:02 2024-05-24T11:27:49.062 INFO:teuthology.orchestra.run.smithi137.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 439 kB/s | 59 kB 00:00 2024-05-24T11:27:49.084 INFO:teuthology.orchestra.run.smithi039.stdout:(12/146): libcephfs-devel-18.2.2-1349.gb9b067bc 310 kB/s | 31 kB 00:00 2024-05-24T11:27:49.113 INFO:teuthology.orchestra.run.smithi137.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 5.6 MB/s | 283 kB 00:00 2024-05-24T11:27:49.155 INFO:teuthology.orchestra.run.smithi137.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 515 kB/s | 60 kB 00:00 2024-05-24T11:27:49.213 INFO:teuthology.orchestra.run.smithi137.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2024-05-24T11:27:49.247 INFO:teuthology.orchestra.run.smithi137.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 367 kB/s | 34 kB 00:00 2024-05-24T11:27:49.289 INFO:teuthology.orchestra.run.smithi137.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2024-05-24T11:27:49.314 INFO:teuthology.orchestra.run.smithi137.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2024-05-24T11:27:49.339 INFO:teuthology.orchestra.run.smithi137.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2024-05-24T11:27:49.342 INFO:teuthology.orchestra.run.smithi039.stdout:(13/146): libcephfs2-18.2.2-1349.gb9b067bc.el8. 3.0 MB/s | 781 kB 00:00 2024-05-24T11:27:49.365 INFO:teuthology.orchestra.run.smithi137.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 476 kB/s | 24 kB 00:00 2024-05-24T11:27:49.390 INFO:teuthology.orchestra.run.smithi137.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 584 kB/s | 29 kB 00:00 2024-05-24T11:27:49.415 INFO:teuthology.orchestra.run.smithi137.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 550 kB/s | 27 kB 00:00 2024-05-24T11:27:49.432 INFO:teuthology.orchestra.run.smithi137.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 6.0 MB/s | 255 kB 00:00 2024-05-24T11:27:49.434 INFO:teuthology.orchestra.run.smithi039.stdout:(14/146): libcephsqlite-18.2.2-1349.gb9b067bc.e 1.5 MB/s | 142 kB 00:00 2024-05-24T11:27:49.474 INFO:teuthology.orchestra.run.smithi137.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2024-05-24T11:27:49.491 INFO:teuthology.orchestra.run.smithi137.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2024-05-24T11:27:49.493 INFO:teuthology.orchestra.run.smithi053.stdout:(7/146): ceph-mon-18.2.2-1349.gb9b067bc.el8.x86 4.0 MB/s | 4.8 MB 00:01 2024-05-24T11:27:49.510 INFO:teuthology.orchestra.run.smithi039.stdout:(15/146): librados-devel-18.2.2-1349.gb9b067bc. 1.8 MB/s | 139 kB 00:00 2024-05-24T11:27:49.516 INFO:teuthology.orchestra.run.smithi137.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2024-05-24T11:27:49.541 INFO:teuthology.orchestra.run.smithi137.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2024-05-24T11:27:49.559 INFO:teuthology.orchestra.run.smithi137.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2024-05-24T11:27:49.624 INFO:teuthology.orchestra.run.smithi137.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2024-05-24T11:27:49.727 INFO:teuthology.orchestra.run.smithi039.stdout:(16/146): ceph-osd-18.2.2-1349.gb9b067bc.el8.x8 3.7 MB/s | 18 MB 00:04 2024-05-24T11:27:49.760 INFO:teuthology.orchestra.run.smithi039.stdout:(17/146): libradosstriper1-18.2.2-1349.gb9b067b 1.8 MB/s | 473 kB 00:00 2024-05-24T11:27:49.784 INFO:teuthology.orchestra.run.smithi137.stdout:(71/146): python3-jwt-1.6.1-2.el8.noarch.rpm 277 kB/s | 43 kB 00:00 2024-05-24T11:27:49.809 INFO:teuthology.orchestra.run.smithi137.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 71 kB/s | 18 kB 00:00 2024-05-24T11:27:49.844 INFO:teuthology.orchestra.run.smithi039.stdout:(18/146): python3-ceph-argparse-18.2.2-1349.gb9 536 kB/s | 45 kB 00:00 2024-05-24T11:27:49.918 INFO:teuthology.orchestra.run.smithi137.stdout:(73/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.1 MB/s | 155 kB 00:00 2024-05-24T11:27:49.935 INFO:teuthology.orchestra.run.smithi039.stdout:(19/146): python3-ceph-common-18.2.2-1349.gb9b0 1.4 MB/s | 129 kB 00:00 2024-05-24T11:27:49.943 INFO:teuthology.orchestra.run.smithi137.stdout:(74/146): python3-pycparser-2.14-14.el8.noarch. 817 kB/s | 109 kB 00:00 2024-05-24T11:27:49.951 INFO:teuthology.orchestra.run.smithi053.stdout:(8/146): ceph-common-18.2.2-1349.gb9b067bc.el8. 3.9 MB/s | 22 MB 00:05 2024-05-24T11:27:50.002 INFO:teuthology.orchestra.run.smithi137.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 512 kB/s | 43 kB 00:00 2024-05-24T11:27:50.018 INFO:teuthology.orchestra.run.smithi053.stdout:(9/146): ceph-selinux-18.2.2-1349.gb9b067bc.el8 368 kB/s | 24 kB 00:00 2024-05-24T11:27:50.027 INFO:teuthology.orchestra.run.smithi039.stdout:(20/146): python3-cephfs-18.2.2-1349.gb9b067bc. 2.1 MB/s | 197 kB 00:00 2024-05-24T11:27:50.053 INFO:teuthology.orchestra.run.smithi137.stdout:(76/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 172 kB/s | 19 kB 00:00 2024-05-24T11:27:50.111 INFO:teuthology.orchestra.run.smithi137.stdout:(77/146): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2024-05-24T11:27:50.136 INFO:teuthology.orchestra.run.smithi137.stdout:(78/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 327 kB/s | 44 kB 00:00 2024-05-24T11:27:50.178 INFO:teuthology.orchestra.run.smithi039.stdout:(21/146): python3-rados-18.2.2-1349.gb9b067bc.e 2.4 MB/s | 371 kB 00:00 2024-05-24T11:27:50.220 INFO:teuthology.orchestra.run.smithi137.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2024-05-24T11:27:50.262 INFO:teuthology.orchestra.run.smithi137.stdout:(80/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2024-05-24T11:27:50.329 INFO:teuthology.orchestra.run.smithi137.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2024-05-24T11:27:50.337 INFO:teuthology.orchestra.run.smithi039.stdout:(22/146): python3-rbd-18.2.2-1349.gb9b067bc.el8 2.3 MB/s | 365 kB 00:00 2024-05-24T11:27:50.388 INFO:teuthology.orchestra.run.smithi137.stdout:(82/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 383 kB/s | 48 kB 00:00 2024-05-24T11:27:50.420 INFO:teuthology.orchestra.run.smithi039.stdout:(23/146): python3-rgw-18.2.2-1349.gb9b067bc.el8 1.3 MB/s | 107 kB 00:00 2024-05-24T11:27:50.479 INFO:teuthology.orchestra.run.smithi137.stdout:(83/146): libsepol-devel-2.9-3.el8.x86_64.rpm 579 kB/s | 87 kB 00:00 2024-05-24T11:27:50.496 INFO:teuthology.orchestra.run.smithi137.stdout:(84/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 531 kB/s | 58 kB 00:00 2024-05-24T11:27:50.512 INFO:teuthology.orchestra.run.smithi039.stdout:(24/146): rbd-fuse-18.2.2-1349.gb9b067bc.el8.x8 1.0 MB/s | 95 kB 00:00 2024-05-24T11:27:50.630 INFO:teuthology.orchestra.run.smithi137.stdout:(85/146): libcom_err-devel-1.45.6-4.el8.x86_64. 257 kB/s | 39 kB 00:00 2024-05-24T11:27:50.655 INFO:teuthology.orchestra.run.smithi137.stdout:(86/146): fuse-2.9.7-17.el8.x86_64.rpm 524 kB/s | 83 kB 00:00 2024-05-24T11:27:50.747 INFO:teuthology.orchestra.run.smithi137.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 5.4 MB/s | 645 kB 00:00 2024-05-24T11:27:50.806 INFO:teuthology.orchestra.run.smithi137.stdout:(88/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2024-05-24T11:27:50.864 INFO:teuthology.orchestra.run.smithi137.stdout:(89/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2024-05-24T11:27:50.915 INFO:teuthology.orchestra.run.smithi137.stdout:(90/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 669 kB/s | 73 kB 00:00 2024-05-24T11:27:50.999 INFO:teuthology.orchestra.run.smithi137.stdout:(91/146): elfutils-libelf-devel-0.189-3.el8.x86 458 kB/s | 62 kB 00:00 2024-05-24T11:27:51.021 INFO:teuthology.orchestra.run.smithi039.stdout:(25/146): librgw2-18.2.2-1349.gb9b067bc.el8.x86 5.1 MB/s | 6.6 MB 00:01 2024-05-24T11:27:51.049 INFO:teuthology.orchestra.run.smithi137.stdout:(92/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.1 MB/s | 562 kB 00:00 2024-05-24T11:27:51.088 INFO:teuthology.orchestra.run.smithi039.stdout:(26/146): rbd-nbd-18.2.2-1349.gb9b067bc.el8.x86 2.7 MB/s | 181 kB 00:00 2024-05-24T11:27:51.188 INFO:teuthology.orchestra.run.smithi039.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1349.g 242 kB/s | 24 kB 00:00 2024-05-24T11:27:51.272 INFO:teuthology.orchestra.run.smithi039.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1349.gb9b067b 1.6 MB/s | 138 kB 00:00 2024-05-24T11:27:51.299 INFO:teuthology.orchestra.run.smithi137.stdout:(93/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 7.8 MB/s | 2.3 MB 00:00 2024-05-24T11:27:51.324 INFO:teuthology.orchestra.run.smithi137.stdout:(94/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 171 kB/s | 47 kB 00:00 2024-05-24T11:27:51.450 INFO:teuthology.orchestra.run.smithi137.stdout:(95/146): python3-cryptography-3.2.1-7.el8_9.x8 3.6 MB/s | 559 kB 00:00 2024-05-24T11:27:51.475 INFO:teuthology.orchestra.run.smithi137.stdout:(96/146): NetworkManager-initscripts-updown-1.4 960 kB/s | 144 kB 00:00 2024-05-24T11:27:51.482 INFO:teuthology.orchestra.run.smithi039.stdout:(29/146): rbd-mirror-18.2.2-1349.gb9b067bc.el8. 3.6 MB/s | 3.5 MB 00:00 2024-05-24T11:27:51.608 INFO:teuthology.orchestra.run.smithi137.stdout:(97/146): python3-jsonpointer-1.10-11.el8.noarc 150 kB/s | 20 kB 00:00 2024-05-24T11:27:51.634 INFO:teuthology.orchestra.run.smithi137.stdout:(98/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 543 kB/s | 100 kB 00:00 2024-05-24T11:27:51.751 INFO:teuthology.orchestra.run.smithi137.stdout:(99/146): python3-prettytable-0.7.2-14.el8.noar 310 kB/s | 44 kB 00:00 2024-05-24T11:27:51.776 INFO:teuthology.orchestra.run.smithi137.stdout:(100/146): python3-pyasn1-modules-0.3.7-6.el8.n 776 kB/s | 110 kB 00:00 2024-05-24T11:27:51.884 INFO:teuthology.orchestra.run.smithi137.stdout:(101/146): python3-jsonpatch-1.21-2.el8.noarch. 199 kB/s | 27 kB 00:00 2024-05-24T11:27:51.902 INFO:teuthology.orchestra.run.smithi137.stdout:(102/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.0 MB/s | 126 kB 00:00 2024-05-24T11:27:52.044 INFO:teuthology.orchestra.run.smithi137.stdout:(103/146): bison-3.0.4-10.el8.x86_64.rpm 4.8 MB/s | 688 kB 00:00 2024-05-24T11:27:52.098 INFO:teuthology.orchestra.run.smithi137.stdout:(104/146): python3-werkzeug-0.12.2-4.el8.noarch 2.1 MB/s | 457 kB 00:00 2024-05-24T11:27:52.138 INFO:teuthology.orchestra.run.smithi039.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1349.gb9b06 4.1 MB/s | 3.5 MB 00:00 2024-05-24T11:27:52.202 INFO:teuthology.orchestra.run.smithi137.stdout:(105/146): libtool-2.4.6-25.el8.x86_64.rpm 6.7 MB/s | 709 kB 00:00 2024-05-24T11:27:52.222 INFO:teuthology.orchestra.run.smithi039.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1349.gb9 2.9 MB/s | 246 kB 00:00 2024-05-24T11:27:52.236 INFO:teuthology.orchestra.run.smithi137.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2024-05-24T11:27:52.289 INFO:teuthology.orchestra.run.smithi039.stdout:(32/146): ceph-mgr-rook-18.2.2-1349.gb9b067bc.e 741 kB/s | 49 kB 00:00 2024-05-24T11:27:52.353 INFO:teuthology.orchestra.run.smithi137.stdout:(107/146): python3-markupsafe-0.23-19.el8.x86_6 258 kB/s | 39 kB 00:00 2024-05-24T11:27:52.393 INFO:teuthology.orchestra.run.smithi039.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1349.gb 158 kB/s | 16 kB 00:00 2024-05-24T11:27:52.403 INFO:teuthology.orchestra.run.smithi137.stdout:(108/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.5 MB/s | 259 kB 00:00 2024-05-24T11:27:52.515 INFO:teuthology.orchestra.run.smithi039.stdout:(34/146): ceph-volume-18.2.2-1349.gb9b067bc.el8 2.1 MB/s | 263 kB 00:00 2024-05-24T11:27:52.537 INFO:teuthology.orchestra.run.smithi137.stdout:(109/146): uuid-1.6.2-43.el8.x86_64.rpm 348 kB/s | 64 kB 00:00 2024-05-24T11:27:52.562 INFO:teuthology.orchestra.run.smithi137.stdout:(110/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 652 kB/s | 103 kB 00:00 2024-05-24T11:27:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:(35/146): cephadm-18.2.2-1349.gb9b067bc.el8.noa 2.5 MB/s | 173 kB 00:00 2024-05-24T11:27:52.621 INFO:teuthology.orchestra.run.smithi137.stdout:(111/146): python3-jinja2-2.10.1-3.el8.noarch.r 9.0 MB/s | 538 kB 00:00 2024-05-24T11:27:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:(36/146): cephfs-top-18.2.2-1349.gb9b067bc.el8. 311 kB/s | 18 kB 00:00 2024-05-24T11:27:52.677 INFO:teuthology.orchestra.run.smithi053.stdout:(10/146): ceph-radosgw-18.2.2-1349.gb9b067bc.el 4.1 MB/s | 13 MB 00:03 2024-05-24T11:27:52.760 INFO:teuthology.orchestra.run.smithi053.stdout:(11/146): cephfs-mirror-18.2.2-1349.gb9b067bc.e 3.1 MB/s | 261 kB 00:00 2024-05-24T11:27:52.830 INFO:teuthology.orchestra.run.smithi137.stdout:(112/146): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2024-05-24T11:27:52.844 INFO:teuthology.orchestra.run.smithi053.stdout:(12/146): libcephfs-devel-18.2.2-1349.gb9b067bc 372 kB/s | 31 kB 00:00 2024-05-24T11:27:52.988 INFO:teuthology.orchestra.run.smithi137.stdout:(113/146): python3-numpy-1.14.3-10.el8.x86_64.r 10 MB/s | 3.7 MB 00:00 2024-05-24T11:27:53.005 INFO:teuthology.orchestra.run.smithi137.stdout:(114/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2024-05-24T11:27:53.061 INFO:teuthology.orchestra.run.smithi053.stdout:(13/146): libcephfs2-18.2.2-1349.gb9b067bc.el8. 3.5 MB/s | 781 kB 00:00 2024-05-24T11:27:53.136 INFO:teuthology.orchestra.run.smithi053.stdout:(14/146): libcephsqlite-18.2.2-1349.gb9b067bc.e 1.9 MB/s | 142 kB 00:00 2024-05-24T11:27:53.147 INFO:teuthology.orchestra.run.smithi137.stdout:(115/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2024-05-24T11:27:53.164 INFO:teuthology.orchestra.run.smithi137.stdout:(116/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.3 MB/s | 225 kB 00:00 2024-05-24T11:27:53.211 INFO:teuthology.orchestra.run.smithi053.stdout:(15/146): librados-devel-18.2.2-1349.gb9b067bc. 1.8 MB/s | 139 kB 00:00 2024-05-24T11:27:53.232 INFO:teuthology.orchestra.run.smithi039.stdout:(37/146): ceph-mgr-diskprediction-local-18.2.2- 4.2 MB/s | 7.4 MB 00:01 2024-05-24T11:27:53.353 INFO:teuthology.orchestra.run.smithi053.stdout:(16/146): libradosstriper1-18.2.2-1349.gb9b067b 3.3 MB/s | 473 kB 00:00 2024-05-24T11:27:53.389 INFO:teuthology.orchestra.run.smithi137.stdout:(117/146): openblas-threads-0.3.15-6.el8.x86_64 22 MB/s | 5.2 MB 00:00 2024-05-24T11:27:53.415 INFO:teuthology.orchestra.run.smithi137.stdout:(118/146): python3-mako-1.0.6-14.el8.noarch.rpm 627 kB/s | 157 kB 00:00 2024-05-24T11:27:53.508 INFO:teuthology.orchestra.run.smithi039.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 353 kB/s | 306 kB 00:00 2024-05-24T11:27:53.550 INFO:teuthology.orchestra.run.smithi137.stdout:(119/146): automake-1.16.1-8.el8.noarch.rpm 5.2 MB/s | 714 kB 00:00 2024-05-24T11:27:53.632 INFO:teuthology.orchestra.run.smithi137.stdout:(120/146): openblas-0.3.15-6.el8.x86_64.rpm 21 MB/s | 5.0 MB 00:00 2024-05-24T11:27:53.708 INFO:teuthology.orchestra.run.smithi039.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 376 kB/s | 75 kB 00:00 2024-05-24T11:27:53.800 INFO:teuthology.orchestra.run.smithi039.stdout:(40/146): liboath-2.6.2-3.el8.x86_64.rpm 104 kB/s | 59 kB 00:00 2024-05-24T11:27:54.049 INFO:teuthology.orchestra.run.smithi137.stdout:(121/146): ceph-test-18.2.2-1349.gb9b067bc.el8. 4.6 MB/s | 47 MB 00:10 2024-05-24T11:27:54.066 INFO:teuthology.orchestra.run.smithi137.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 383 kB/s | 198 kB 00:00 2024-05-24T11:27:54.091 INFO:teuthology.orchestra.run.smithi137.stdout:(123/146): jq-1.6-7.el8.x86_64.rpm 441 kB/s | 202 kB 00:00 2024-05-24T11:27:54.109 INFO:teuthology.orchestra.run.smithi039.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 141 kB/s | 44 kB 00:00 2024-05-24T11:27:54.191 INFO:teuthology.orchestra.run.smithi137.stdout:(124/146): python3-pytz-2017.2-11.el8.noarch.rp 383 kB/s | 54 kB 00:00 2024-05-24T11:27:54.242 INFO:teuthology.orchestra.run.smithi137.stdout:(125/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-24T11:27:54.375 INFO:teuthology.orchestra.run.smithi137.stdout:(126/146): postgresql-10.23-4.module+el8.9.0+21 8.2 MB/s | 1.5 MB 00:00 2024-05-24T11:27:54.417 INFO:teuthology.orchestra.run.smithi137.stdout:(127/146): postgresql-contrib-10.23-4.module+el 4.5 MB/s | 812 kB 00:00 2024-05-24T11:27:54.526 INFO:teuthology.orchestra.run.smithi039.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 582 kB/s | 476 kB 00:00 2024-05-24T11:27:54.543 INFO:teuthology.orchestra.run.smithi039.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 427 kB/s | 185 kB 00:00 2024-05-24T11:27:54.578 INFO:teuthology.orchestra.run.smithi053.stdout:(17/146): librgw2-18.2.2-1349.gb9b067bc.el8.x86 5.3 MB/s | 6.6 MB 00:01 2024-05-24T11:27:54.601 INFO:teuthology.orchestra.run.smithi137.stdout:(128/146): python3-scipy-1.0.0-21.module+el8.9. 26 MB/s | 14 MB 00:00 2024-05-24T11:27:54.626 INFO:teuthology.orchestra.run.smithi137.stdout:(129/146): python3-packaging-16.8-9.el8.noarch. 231 kB/s | 48 kB 00:00 2024-05-24T11:27:54.643 INFO:teuthology.orchestra.run.smithi039.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 283 kB/s | 33 kB 00:00 2024-05-24T11:27:54.668 INFO:teuthology.orchestra.run.smithi039.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2024-05-24T11:27:54.687 INFO:teuthology.orchestra.run.smithi053.stdout:(18/146): python3-ceph-argparse-18.2.2-1349.gb9 413 kB/s | 45 kB 00:00 2024-05-24T11:27:54.760 INFO:teuthology.orchestra.run.smithi137.stdout:(130/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-05-24T11:27:54.771 INFO:teuthology.orchestra.run.smithi053.stdout:(19/146): python3-ceph-common-18.2.2-1349.gb9b0 1.5 MB/s | 129 kB 00:00 2024-05-24T11:27:54.863 INFO:teuthology.orchestra.run.smithi053.stdout:(20/146): python3-cephfs-18.2.2-1349.gb9b067bc. 2.1 MB/s | 197 kB 00:00 2024-05-24T11:27:54.936 INFO:teuthology.orchestra.run.smithi039.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 591 kB/s | 173 kB 00:00 2024-05-24T11:27:55.005 INFO:teuthology.orchestra.run.smithi053.stdout:(21/146): python3-rados-18.2.2-1349.gb9b067bc.e 2.6 MB/s | 371 kB 00:00 2024-05-24T11:27:55.152 INFO:teuthology.orchestra.run.smithi039.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 375 kB/s | 81 kB 00:00 2024-05-24T11:27:55.163 INFO:teuthology.orchestra.run.smithi053.stdout:(22/146): python3-rbd-18.2.2-1349.gb9b067bc.el8 2.3 MB/s | 365 kB 00:00 2024-05-24T11:27:55.196 INFO:teuthology.orchestra.run.smithi137.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 532 kB/s | 232 kB 00:00 2024-05-24T11:27:55.244 INFO:teuthology.orchestra.run.smithi039.stdout:(48/146): python3-cherrypy-18.4.0-1.el8.noarch. 668 kB/s | 384 kB 00:00 2024-05-24T11:27:55.246 INFO:teuthology.orchestra.run.smithi053.stdout:(23/146): python3-rgw-18.2.2-1349.gb9b067bc.el8 1.3 MB/s | 107 kB 00:00 2024-05-24T11:27:55.269 INFO:teuthology.orchestra.run.smithi039.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2024-05-24T11:27:55.339 INFO:teuthology.orchestra.run.smithi053.stdout:(24/146): rbd-fuse-18.2.2-1349.gb9b067bc.el8.x8 1.0 MB/s | 95 kB 00:00 2024-05-24T11:27:55.353 INFO:teuthology.orchestra.run.smithi039.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-05-24T11:27:55.393 INFO:teuthology.orchestra.run.smithi137.stdout:(132/146): librados2-18.2.2-1349.gb9b067bc.el8. 4.7 MB/s | 3.7 MB 00:00 2024-05-24T11:27:55.427 INFO:teuthology.orchestra.run.smithi137.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.3 MB/s | 303 kB 00:00 2024-05-24T11:27:55.478 INFO:teuthology.orchestra.run.smithi039.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 390 kB/s | 49 kB 00:00 2024-05-24T11:27:55.582 INFO:teuthology.orchestra.run.smithi137.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.4 MB/s | 854 kB 00:00 2024-05-24T11:27:55.619 INFO:teuthology.orchestra.run.smithi137.stdout:(135/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.6 MB/s | 711 kB 00:00 2024-05-24T11:27:55.703 INFO:teuthology.orchestra.run.smithi137.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 12 MB/s | 1.5 MB 00:00 2024-05-24T11:27:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 261 kB/s | 59 kB 00:00 2024-05-24T11:27:55.778 INFO:teuthology.orchestra.run.smithi137.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.0 MB/s | 333 kB 00:00 2024-05-24T11:27:55.829 INFO:teuthology.orchestra.run.smithi137.stdout:(138/146): librbd1-18.2.2-1349.gb9b067bc.el8.x8 3.0 MB/s | 3.6 MB 00:01 2024-05-24T11:27:55.870 INFO:teuthology.orchestra.run.smithi137.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.4 MB/s | 1.1 MB 00:00 2024-05-24T11:27:55.904 INFO:teuthology.orchestra.run.smithi137.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 221 kB 00:00 2024-05-24T11:27:55.920 INFO:teuthology.orchestra.run.smithi039.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 278 kB/s | 60 kB 00:00 2024-05-24T11:27:55.946 INFO:teuthology.orchestra.run.smithi137.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 254 kB 00:00 2024-05-24T11:27:56.010 INFO:teuthology.orchestra.run.smithi137.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 237 kB 00:00 2024-05-24T11:27:56.047 INFO:teuthology.orchestra.run.smithi137.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.2 MB/s | 180 kB 00:00 2024-05-24T11:27:56.114 INFO:teuthology.orchestra.run.smithi137.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 592 kB/s | 99 kB 00:00 2024-05-24T11:27:56.197 INFO:teuthology.orchestra.run.smithi137.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 13 MB/s | 2.5 MB 00:00 2024-05-24T11:27:56.222 INFO:teuthology.orchestra.run.smithi137.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 587 kB/s | 103 kB 00:00 2024-05-24T11:27:56.225 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-05-24T11:27:56.226 INFO:teuthology.orchestra.run.smithi137.stdout:Total 14 MB/s | 215 MB 00:15 2024-05-24T11:27:56.247 INFO:teuthology.orchestra.run.smithi053.stdout:(25/146): ceph-test-18.2.2-1349.gb9b067bc.el8.x 7.6 MB/s | 47 MB 00:06 2024-05-24T11:27:56.413 INFO:teuthology.orchestra.run.smithi039.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 575 kB/s | 283 kB 00:00 2024-05-24T11:27:56.456 INFO:teuthology.orchestra.run.smithi053.stdout:(26/146): ceph-osd-18.2.2-1349.gb9b067bc.el8.x8 2.3 MB/s | 18 MB 00:08 2024-05-24T11:27:56.472 INFO:teuthology.orchestra.run.smithi053.stdout:(27/146): rbd-nbd-18.2.2-1349.gb9b067bc.el8.x86 807 kB/s | 181 kB 00:00 2024-05-24T11:27:56.521 INFO:teuthology.orchestra.run.smithi039.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2024-05-24T11:27:56.531 INFO:teuthology.orchestra.run.smithi053.stdout:(28/146): rbd-mirror-18.2.2-1349.gb9b067bc.el8. 2.9 MB/s | 3.5 MB 00:01 2024-05-24T11:27:56.556 INFO:teuthology.orchestra.run.smithi053.stdout:(29/146): ceph-grafana-dashboards-18.2.2-1349.g 242 kB/s | 24 kB 00:00 2024-05-24T11:27:56.582 INFO:teuthology.orchestra.run.smithi053.stdout:(30/146): ceph-mgr-cephadm-18.2.2-1349.gb9b067b 1.2 MB/s | 138 kB 00:00 2024-05-24T11:27:56.632 INFO:teuthology.orchestra.run.smithi053.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1349.gb9 4.8 MB/s | 246 kB 00:00 2024-05-24T11:27:56.638 INFO:teuthology.orchestra.run.smithi039.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 289 kB/s | 34 kB 00:00 2024-05-24T11:27:56.674 INFO:teuthology.orchestra.run.smithi053.stdout:(32/146): ceph-mgr-rook-18.2.2-1349.gb9b067bc.e 1.2 MB/s | 49 kB 00:00 2024-05-24T11:27:56.716 INFO:teuthology.orchestra.run.smithi053.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1349.gb 391 kB/s | 16 kB 00:00 2024-05-24T11:27:56.775 INFO:teuthology.orchestra.run.smithi053.stdout:(34/146): ceph-volume-18.2.2-1349.gb9b067bc.el8 4.4 MB/s | 263 kB 00:00 2024-05-24T11:27:56.817 INFO:teuthology.orchestra.run.smithi053.stdout:(35/146): cephadm-18.2.2-1349.gb9b067bc.el8.noa 4.1 MB/s | 173 kB 00:00 2024-05-24T11:27:56.858 INFO:teuthology.orchestra.run.smithi053.stdout:(36/146): cephfs-top-18.2.2-1349.gb9b067bc.el8. 435 kB/s | 18 kB 00:00 2024-05-24T11:27:57.197 INFO:teuthology.orchestra.run.smithi039.stdout:(57/146): ceph-test-18.2.2-1349.gb9b067bc.el8.x 5.4 MB/s | 47 MB 00:08 2024-05-24T11:27:57.222 INFO:teuthology.orchestra.run.smithi039.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 337 kB/s | 196 kB 00:00 2024-05-24T11:27:57.256 INFO:teuthology.orchestra.run.smithi039.stdout:(59/146): python3-kubernetes-11.0.0-6.el8.noarc 601 kB/s | 1.2 MB 00:01 2024-05-24T11:27:57.267 INFO:teuthology.orchestra.run.smithi053.stdout:(37/146): ceph-mgr-dashboard-18.2.2-1349.gb9b06 4.8 MB/s | 3.5 MB 00:00 2024-05-24T11:27:57.300 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-05-24T11:27:57.331 INFO:teuthology.orchestra.run.smithi039.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2024-05-24T11:27:57.364 INFO:teuthology.orchestra.run.smithi039.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2024-05-24T11:27:57.392 INFO:teuthology.orchestra.run.smithi053.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 574 kB/s | 306 kB 00:00 2024-05-24T11:27:57.440 INFO:teuthology.orchestra.run.smithi039.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2024-05-24T11:27:57.501 INFO:teuthology.orchestra.run.smithi053.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 693 kB/s | 75 kB 00:00 2024-05-24T11:27:57.506 INFO:teuthology.orchestra.run.smithi039.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 194 kB/s | 27 kB 00:00 2024-05-24T11:27:57.618 INFO:teuthology.orchestra.run.smithi053.stdout:(40/146): liboath-2.6.2-3.el8.x86_64.rpm 168 kB/s | 59 kB 00:00 2024-05-24T11:27:57.693 INFO:teuthology.orchestra.run.smithi053.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.4 MB/s | 476 kB 00:00 2024-05-24T11:27:57.696 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-05-24T11:27:57.696 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-05-24T11:27:57.718 INFO:teuthology.orchestra.run.smithi053.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 435 kB/s | 44 kB 00:00 2024-05-24T11:27:57.798 INFO:teuthology.orchestra.run.smithi039.stdout:(64/146): python3-rsa-4.9-2.el8.noarch.rpm 107 kB/s | 64 kB 00:00 2024-05-24T11:27:57.819 INFO:teuthology.orchestra.run.smithi053.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2024-05-24T11:27:57.832 INFO:teuthology.orchestra.run.smithi039.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 651 kB/s | 255 kB 00:00 2024-05-24T11:27:57.835 INFO:teuthology.orchestra.run.smithi053.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2024-05-24T11:27:57.916 INFO:teuthology.orchestra.run.smithi039.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 614 kB/s | 251 kB 00:00 2024-05-24T11:27:57.919 INFO:teuthology.orchestra.run.smithi053.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2024-05-24T11:27:57.991 INFO:teuthology.orchestra.run.smithi039.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 318 kB/s | 61 kB 00:00 2024-05-24T11:27:58.025 INFO:teuthology.orchestra.run.smithi039.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 444 kB/s | 85 kB 00:00 2024-05-24T11:27:58.028 INFO:teuthology.orchestra.run.smithi053.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 899 kB/s | 173 kB 00:00 2024-05-24T11:27:58.050 INFO:teuthology.orchestra.run.smithi039.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 171 kB/s | 23 kB 00:00 2024-05-24T11:27:58.053 INFO:teuthology.orchestra.run.smithi053.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2024-05-24T11:27:58.100 INFO:teuthology.orchestra.run.smithi039.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 925 kB/s | 70 kB 00:00 2024-05-24T11:27:58.137 INFO:teuthology.orchestra.run.smithi053.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 749 kB/s | 81 kB 00:00 2024-05-24T11:27:58.162 INFO:teuthology.orchestra.run.smithi053.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2024-05-24T11:27:58.237 INFO:teuthology.orchestra.run.smithi053.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2024-05-24T11:27:58.259 INFO:teuthology.orchestra.run.smithi039.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 86 kB/s | 18 kB 00:00 2024-05-24T11:27:58.284 INFO:teuthology.orchestra.run.smithi039.stdout:(72/146): python3-jwt-1.6.1-2.el8.noarch.rpm 243 kB/s | 43 kB 00:00 2024-05-24T11:27:58.346 INFO:teuthology.orchestra.run.smithi053.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2024-05-24T11:27:58.409 INFO:teuthology.orchestra.run.smithi039.stdout:(73/146): python3-pycparser-2.14-14.el8.noarch. 873 kB/s | 109 kB 00:00 2024-05-24T11:27:58.435 INFO:teuthology.orchestra.run.smithi039.stdout:(74/146): python3-oauthlib-2.1.0-1.el8.noarch.r 884 kB/s | 155 kB 00:00 2024-05-24T11:27:58.446 INFO:teuthology.orchestra.run.smithi053.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 4.1 MB/s | 1.2 MB 00:00 2024-05-24T11:27:58.471 INFO:teuthology.orchestra.run.smithi053.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 468 kB/s | 59 kB 00:00 2024-05-24T11:27:58.543 INFO:teuthology.orchestra.run.smithi039.stdout:(75/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 173 kB/s | 19 kB 00:00 2024-05-24T11:27:58.555 INFO:teuthology.orchestra.run.smithi053.stdout:(54/146): ceph-mgr-diskprediction-local-18.2.2- 3.7 MB/s | 7.4 MB 00:01 2024-05-24T11:27:58.580 INFO:teuthology.orchestra.run.smithi053.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 450 kB/s | 60 kB 00:00 2024-05-24T11:27:58.605 INFO:teuthology.orchestra.run.smithi053.stdout:(56/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2024-05-24T11:27:58.644 INFO:teuthology.orchestra.run.smithi039.stdout:(76/146): python3-requests-oauthlib-1.0.0-1.el8 182 kB/s | 43 kB 00:00 2024-05-24T11:27:58.689 INFO:teuthology.orchestra.run.smithi053.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2024-05-24T11:27:58.748 INFO:teuthology.orchestra.run.smithi053.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2024-05-24T11:27:58.769 INFO:teuthology.orchestra.run.smithi039.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 195 kB/s | 44 kB 00:00 2024-05-24T11:27:58.798 INFO:teuthology.orchestra.run.smithi053.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 591 kB/s | 64 kB 00:00 2024-05-24T11:27:58.853 INFO:teuthology.orchestra.run.smithi039.stdout:(78/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2024-05-24T11:27:58.856 INFO:teuthology.orchestra.run.smithi053.stdout:(60/146): python3-portend-2.6-1.el8.noarch.rpm 52 kB/s | 16 kB 00:00 2024-05-24T11:27:58.882 INFO:teuthology.orchestra.run.smithi053.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2024-05-24T11:27:58.898 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-05-24T11:27:58.903 INFO:teuthology.orchestra.run.smithi039.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2024-05-24T11:27:58.907 INFO:teuthology.orchestra.run.smithi053.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 219 kB/s | 24 kB 00:00 2024-05-24T11:27:58.909 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-05-24T11:27:58.974 INFO:teuthology.orchestra.run.smithi053.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 249 kB/s | 29 kB 00:00 2024-05-24T11:27:58.978 INFO:teuthology.orchestra.run.smithi039.stdout:(80/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2024-05-24T11:27:58.999 INFO:teuthology.orchestra.run.smithi053.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 234 kB/s | 27 kB 00:00 2024-05-24T11:27:59.025 INFO:teuthology.orchestra.run.smithi053.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 2.1 MB/s | 255 kB 00:00 2024-05-24T11:27:59.062 INFO:teuthology.orchestra.run.smithi039.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.2 MB/s | 200 kB 00:00 2024-05-24T11:27:59.108 INFO:teuthology.orchestra.run.smithi053.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 565 kB/s | 61 kB 00:00 2024-05-24T11:27:59.142 INFO:teuthology.orchestra.run.smithi053.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 729 kB/s | 85 kB 00:00 2024-05-24T11:27:59.145 INFO:teuthology.orchestra.run.smithi039.stdout:(82/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 289 kB/s | 48 kB 00:00 2024-05-24T11:27:59.196 INFO:teuthology.orchestra.run.smithi039.stdout:(83/146): libsepol-devel-2.9-3.el8.x86_64.rpm 651 kB/s | 87 kB 00:00 2024-05-24T11:27:59.217 INFO:teuthology.orchestra.run.smithi053.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2024-05-24T11:27:59.221 INFO:teuthology.orchestra.run.smithi039.stdout:(84/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-05-24T11:27:59.276 INFO:teuthology.orchestra.run.smithi053.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2024-05-24T11:27:59.301 INFO:teuthology.orchestra.run.smithi053.stdout:(70/146): python3-webob-1.8.5-1.el8.1.noarch.rp 769 kB/s | 251 kB 00:00 2024-05-24T11:27:59.355 INFO:teuthology.orchestra.run.smithi039.stdout:(85/146): fuse-2.9.7-17.el8.x86_64.rpm 623 kB/s | 83 kB 00:00 2024-05-24T11:27:59.380 INFO:teuthology.orchestra.run.smithi039.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 210 kB/s | 39 kB 00:00 2024-05-24T11:27:59.384 INFO:teuthology.orchestra.run.smithi053.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 7.2 MB/s | 1.7 MB 00:00 2024-05-24T11:27:59.435 INFO:teuthology.orchestra.run.smithi053.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 330 kB/s | 44 kB 00:00 2024-05-24T11:27:59.460 INFO:teuthology.orchestra.run.smithi053.stdout:(73/146): python3-jsonpointer-1.10-11.el8.noarc 109 kB/s | 20 kB 00:00 2024-05-24T11:27:59.539 INFO:teuthology.orchestra.run.smithi039.stdout:(87/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2024-05-24T11:27:59.597 INFO:teuthology.orchestra.run.smithi039.stdout:(88/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.6 MB/s | 645 kB 00:00 2024-05-24T11:27:59.610 INFO:teuthology.orchestra.run.smithi053.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 152 kB/s | 27 kB 00:00 2024-05-24T11:27:59.636 INFO:teuthology.orchestra.run.smithi053.stdout:(75/146): python3-pyasn1-modules-0.3.7-6.el8.no 439 kB/s | 110 kB 00:00 2024-05-24T11:27:59.647 INFO:teuthology.orchestra.run.smithi039.stdout:(89/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2024-05-24T11:27:59.686 INFO:teuthology.orchestra.run.smithi053.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 560 kB/s | 126 kB 00:00 2024-05-24T11:27:59.731 INFO:teuthology.orchestra.run.smithi039.stdout:(90/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 543 kB/s | 73 kB 00:00 2024-05-24T11:27:59.736 INFO:teuthology.orchestra.run.smithi053.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.6 MB/s | 457 kB 00:00 2024-05-24T11:27:59.770 INFO:teuthology.orchestra.run.smithi053.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2024-05-24T11:27:59.820 INFO:teuthology.orchestra.run.smithi053.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2024-05-24T11:27:59.839 INFO:teuthology.orchestra.run.smithi039.stdout:(91/146): elfutils-libelf-devel-0.189-3.el8.x86 322 kB/s | 62 kB 00:00 2024-05-24T11:27:59.854 INFO:teuthology.orchestra.run.smithi053.stdout:(80/146): libtool-2.4.6-25.el8.x86_64.rpm 5.9 MB/s | 709 kB 00:00 2024-05-24T11:27:59.879 INFO:teuthology.orchestra.run.smithi053.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 357 kB/s | 39 kB 00:00 2024-05-24T11:27:59.904 INFO:teuthology.orchestra.run.smithi053.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2024-05-24T11:27:59.906 INFO:teuthology.orchestra.run.smithi039.stdout:(92/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.1 MB/s | 562 kB 00:00 2024-05-24T11:27:59.982 INFO:teuthology.orchestra.run.smithi039.stdout:(93/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 624 kB/s | 47 kB 00:00 2024-05-24T11:27:59.996 INFO:teuthology.orchestra.run.smithi053.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 450 kB/s | 64 kB 00:00 2024-05-24T11:28:00.021 INFO:teuthology.orchestra.run.smithi053.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 728 kB/s | 103 kB 00:00 2024-05-24T11:28:00.057 INFO:teuthology.orchestra.run.smithi039.stdout:(94/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-24T11:28:00.163 INFO:teuthology.orchestra.run.smithi053.stdout:(85/146): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2024-05-24T11:28:00.180 INFO:teuthology.orchestra.run.smithi039.stdout:(95/146): python3-cryptography-3.2.1-7.el8_9.x8 2.8 MB/s | 559 kB 00:00 2024-05-24T11:28:00.197 INFO:teuthology.orchestra.run.smithi053.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.6 MB/s | 538 kB 00:00 2024-05-24T11:28:00.233 INFO:teuthology.orchestra.run.smithi039.stdout:(96/146): NetworkManager-initscripts-updown-1.4 823 kB/s | 144 kB 00:00 2024-05-24T11:28:00.297 INFO:teuthology.orchestra.run.smithi053.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2024-05-24T11:28:00.331 INFO:teuthology.orchestra.run.smithi053.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.9 MB/s | 323 kB 00:00 2024-05-24T11:28:00.333 INFO:teuthology.orchestra.run.smithi039.stdout:(97/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 652 kB/s | 100 kB 00:00 2024-05-24T11:28:00.358 INFO:teuthology.orchestra.run.smithi039.stdout:(98/146): python3-jsonpointer-1.10-11.el8.noarc 159 kB/s | 20 kB 00:00 2024-05-24T11:28:00.373 INFO:teuthology.orchestra.run.smithi053.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2024-05-24T11:28:00.406 INFO:teuthology.orchestra.run.smithi053.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2024-05-24T11:28:00.434 INFO:teuthology.orchestra.run.smithi039.stdout:(99/146): python3-prettytable-0.7.2-14.el8.noar 439 kB/s | 44 kB 00:00 2024-05-24T11:28:00.492 INFO:teuthology.orchestra.run.smithi039.stdout:(100/146): python3-pyasn1-modules-0.3.7-6.el8.n 824 kB/s | 110 kB 00:00 2024-05-24T11:28:00.526 INFO:teuthology.orchestra.run.smithi039.stdout:(101/146): thrift-0.13.0-2.el8.x86_64.rpm 701 kB/s | 1.7 MB 00:02 2024-05-24T11:28:00.573 INFO:teuthology.orchestra.run.smithi053.stdout:(91/146): openblas-threads-0.3.15-6.el8.x86_64. 21 MB/s | 5.2 MB 00:00 2024-05-24T11:28:00.576 INFO:teuthology.orchestra.run.smithi039.stdout:(102/146): python3-jsonpatch-1.21-2.el8.noarch. 186 kB/s | 27 kB 00:00 2024-05-24T11:28:00.598 INFO:teuthology.orchestra.run.smithi053.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 696 kB/s | 157 kB 00:00 2024-05-24T11:28:00.643 INFO:teuthology.orchestra.run.smithi039.stdout:(103/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 840 kB/s | 126 kB 00:00 2024-05-24T11:28:00.718 INFO:teuthology.orchestra.run.smithi039.stdout:(104/146): python3-werkzeug-0.12.2-4.el8.noarch 2.3 MB/s | 457 kB 00:00 2024-05-24T11:28:00.724 INFO:teuthology.orchestra.run.smithi053.stdout:(93/146): openblas-0.3.15-6.el8.x86_64.rpm 16 MB/s | 5.0 MB 00:00 2024-05-24T11:28:00.766 INFO:teuthology.orchestra.run.smithi053.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 3.6 MB/s | 714 kB 00:00 2024-05-24T11:28:00.794 INFO:teuthology.orchestra.run.smithi039.stdout:(105/146): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2024-05-24T11:28:00.836 INFO:teuthology.orchestra.run.smithi039.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 2.6 MB/s | 688 kB 00:00 2024-05-24T11:28:00.858 INFO:teuthology.orchestra.run.smithi053.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 764 kB/s | 198 kB 00:00 2024-05-24T11:28:00.883 INFO:teuthology.orchestra.run.smithi053.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-24T11:28:00.894 INFO:teuthology.orchestra.run.smithi039.stdout:(107/146): python3-markupsafe-0.23-19.el8.x86_6 387 kB/s | 39 kB 00:00 2024-05-24T11:28:00.931 INFO:teuthology.orchestra.run.smithi053.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 330 kB/s | 54 kB 00:00 2024-05-24T11:28:00.945 INFO:teuthology.orchestra.run.smithi039.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 3.1 MB/s | 709 kB 00:00 2024-05-24T11:28:00.970 INFO:teuthology.orchestra.run.smithi039.stdout:(109/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2024-05-24T11:28:01.008 INFO:teuthology.orchestra.run.smithi053.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-24T11:28:01.079 INFO:teuthology.orchestra.run.smithi039.stdout:(110/146): uuid-1.6.2-43.el8.x86_64.rpm 348 kB/s | 64 kB 00:00 2024-05-24T11:28:01.104 INFO:teuthology.orchestra.run.smithi039.stdout:(111/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 652 kB/s | 103 kB 00:00 2024-05-24T11:28:01.192 INFO:teuthology.orchestra.run.smithi053.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 5.8 MB/s | 1.5 MB 00:00 2024-05-24T11:28:01.234 INFO:teuthology.orchestra.run.smithi053.stdout:(100/146): postgresql-contrib-10.23-4.module+el 3.5 MB/s | 812 kB 00:00 2024-05-24T11:28:01.237 INFO:teuthology.orchestra.run.smithi039.stdout:(112/146): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2024-05-24T11:28:01.271 INFO:teuthology.orchestra.run.smithi039.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.7 MB/s | 538 kB 00:00 2024-05-24T11:28:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:(114/146): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2024-05-24T11:28:01.418 INFO:teuthology.orchestra.run.smithi053.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 24 MB/s | 14 MB 00:00 2024-05-24T11:28:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2024-05-24T11:28:01.443 INFO:teuthology.orchestra.run.smithi053.stdout:(102/146): python3-packaging-16.8-9.el8.noarch. 231 kB/s | 48 kB 00:00 2024-05-24T11:28:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:(116/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.3 MB/s | 225 kB 00:00 2024-05-24T11:28:01.510 INFO:teuthology.orchestra.run.smithi053.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 195 kB/s | 18 kB 00:00 2024-05-24T11:28:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:(117/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-05-24T11:28:01.593 INFO:teuthology.orchestra.run.smithi053.stdout:(104/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-05-24T11:28:01.669 INFO:teuthology.orchestra.run.smithi053.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 189 kB/s | 43 kB 00:00 2024-05-24T11:28:01.694 INFO:teuthology.orchestra.run.smithi053.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 1.1 MB/s | 109 kB 00:00 2024-05-24T11:28:01.747 INFO:teuthology.orchestra.run.smithi039.stdout:(118/146): openblas-threads-0.3.15-6.el8.x86_64 16 MB/s | 5.2 MB 00:00 2024-05-24T11:28:01.764 INFO:teuthology.orchestra.run.smithi039.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 495 kB/s | 157 kB 00:00 2024-05-24T11:28:01.820 INFO:teuthology.orchestra.run.smithi053.stdout:(107/146): python3-oauthlib-2.1.0-1.el8.noarch. 499 kB/s | 155 kB 00:00 2024-05-24T11:28:01.853 INFO:teuthology.orchestra.run.smithi053.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 232 kB/s | 43 kB 00:00 2024-05-24T11:28:01.856 INFO:teuthology.orchestra.run.smithi039.stdout:(120/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-05-24T11:28:01.878 INFO:teuthology.orchestra.run.smithi053.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 102 kB/s | 19 kB 00:00 2024-05-24T11:28:01.881 INFO:teuthology.orchestra.run.smithi039.stdout:(121/146): libpq-13.11-1.el8.x86_64.rpm 1.7 MB/s | 198 kB 00:00 2024-05-24T11:28:01.917 INFO:teuthology.orchestra.run.smithi039.stdout:(122/146): automake-1.16.1-8.el8.noarch.rpm 4.2 MB/s | 714 kB 00:00 2024-05-24T11:28:01.953 INFO:teuthology.orchestra.run.smithi053.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 331 kB/s | 44 kB 00:00 2024-05-24T11:28:01.995 INFO:teuthology.orchestra.run.smithi053.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.2 MB/s | 605 kB 00:00 2024-05-24T11:28:02.015 INFO:teuthology.orchestra.run.smithi039.stdout:(123/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-24T11:28:02.029 INFO:teuthology.orchestra.run.smithi053.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2024-05-24T11:28:02.040 INFO:teuthology.orchestra.run.smithi039.stdout:(124/146): python3-pytz-2017.2-11.el8.noarch.rp 341 kB/s | 54 kB 00:00 2024-05-24T11:28:02.054 INFO:teuthology.orchestra.run.smithi053.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.1 MB/s | 220 kB 00:00 2024-05-24T11:28:02.088 INFO:teuthology.orchestra.run.smithi053.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 2.1 MB/s | 200 kB 00:00 2024-05-24T11:28:02.149 INFO:teuthology.orchestra.run.smithi039.stdout:(125/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-05-24T11:28:02.171 INFO:teuthology.orchestra.run.smithi053.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 745 kB/s | 87 kB 00:00 2024-05-24T11:28:02.197 INFO:teuthology.orchestra.run.smithi053.stdout:(116/146): keyutils-libs-devel-1.5.10-9.el8.x86 287 kB/s | 48 kB 00:00 2024-05-24T11:28:02.248 INFO:teuthology.orchestra.run.smithi053.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 359 kB/s | 58 kB 00:00 2024-05-24T11:28:02.297 INFO:teuthology.orchestra.run.smithi053.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 308 kB/s | 39 kB 00:00 2024-05-24T11:28:02.322 INFO:teuthology.orchestra.run.smithi053.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 662 kB/s | 83 kB 00:00 2024-05-24T11:28:02.325 INFO:teuthology.orchestra.run.smithi039.stdout:(126/146): postgresql-10.23-4.module+el8.9.0+21 5.3 MB/s | 1.5 MB 00:00 2024-05-24T11:28:02.375 INFO:teuthology.orchestra.run.smithi039.stdout:(127/146): postgresql-contrib-10.23-4.module+el 3.5 MB/s | 812 kB 00:00 2024-05-24T11:28:02.414 INFO:teuthology.orchestra.run.smithi053.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.8 MB/s | 645 kB 00:00 2024-05-24T11:28:02.440 INFO:teuthology.orchestra.run.smithi053.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2024-05-24T11:28:02.465 INFO:teuthology.orchestra.run.smithi053.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2024-05-24T11:28:02.515 INFO:teuthology.orchestra.run.smithi053.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 724 kB/s | 73 kB 00:00 2024-05-24T11:28:02.567 INFO:teuthology.orchestra.run.smithi039.stdout:(128/146): python3-scipy-1.0.0-21.module+el8.9. 21 MB/s | 14 MB 00:00 2024-05-24T11:28:02.582 INFO:teuthology.orchestra.run.smithi053.stdout:(124/146): elfutils-libelf-devel-0.189-3.el8.x8 434 kB/s | 62 kB 00:00 2024-05-24T11:28:02.600 INFO:teuthology.orchestra.run.smithi039.stdout:(129/146): python3-packaging-16.8-9.el8.noarch. 214 kB/s | 48 kB 00:00 2024-05-24T11:28:02.616 INFO:teuthology.orchestra.run.smithi053.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.6 MB/s | 562 kB 00:00 2024-05-24T11:28:02.708 INFO:teuthology.orchestra.run.smithi053.stdout:(126/146): openssl-devel-1.1.1k-12.el8_9.x86_64 12 MB/s | 2.3 MB 00:00 2024-05-24T11:28:02.717 INFO:teuthology.orchestra.run.smithi039.stdout:(130/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-05-24T11:28:02.758 INFO:teuthology.orchestra.run.smithi053.stdout:(127/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 268 kB/s | 47 kB 00:00 2024-05-24T11:28:02.792 INFO:teuthology.orchestra.run.smithi053.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 3.1 MB/s | 559 kB 00:00 2024-05-24T11:28:02.834 INFO:teuthology.orchestra.run.smithi039.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.9 MB/s | 232 kB 00:00 2024-05-24T11:28:02.842 INFO:teuthology.orchestra.run.smithi053.stdout:(129/146): NetworkManager-initscripts-updown-1. 1.1 MB/s | 144 kB 00:00 2024-05-24T11:28:02.909 INFO:teuthology.orchestra.run.smithi053.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 663 kB/s | 100 kB 00:00 2024-05-24T11:28:03.035 INFO:teuthology.orchestra.run.smithi039.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2024-05-24T11:28:03.109 INFO:teuthology.orchestra.run.smithi053.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.1 MB/s | 232 kB 00:00 2024-05-24T11:28:03.110 INFO:teuthology.orchestra.run.smithi039.stdout:(133/146): librbd1-18.2.2-1349.gb9b067bc.el8.x8 7.0 MB/s | 3.6 MB 00:00 2024-05-24T11:28:03.210 INFO:teuthology.orchestra.run.smithi039.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.8 MB/s | 854 kB 00:00 2024-05-24T11:28:03.259 INFO:teuthology.orchestra.run.smithi053.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.0 MB/s | 303 kB 00:00 2024-05-24T11:28:03.260 INFO:teuthology.orchestra.run.smithi039.stdout:(135/146): librados2-18.2.2-1349.gb9b067bc.el8. 5.3 MB/s | 3.7 MB 00:00 2024-05-24T11:28:03.294 INFO:teuthology.orchestra.run.smithi039.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.8 MB/s | 711 kB 00:00 2024-05-24T11:28:03.353 INFO:teuthology.orchestra.run.smithi039.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 10 MB/s | 1.5 MB 00:00 2024-05-24T11:28:03.394 INFO:teuthology.orchestra.run.smithi039.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.4 MB/s | 333 kB 00:00 2024-05-24T11:28:03.401 INFO:teuthology.orchestra.run.smithi053.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2024-05-24T11:28:03.495 INFO:teuthology.orchestra.run.smithi039.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.3 MB/s | 1.1 MB 00:00 2024-05-24T11:28:03.520 INFO:teuthology.orchestra.run.smithi039.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.0 MB/s | 254 kB 00:00 2024-05-24T11:28:03.545 INFO:teuthology.orchestra.run.smithi039.stdout:(141/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.1 MB/s | 221 kB 00:00 2024-05-24T11:28:03.551 INFO:teuthology.orchestra.run.smithi053.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.6 MB/s | 711 kB 00:00 2024-05-24T11:28:03.587 INFO:teuthology.orchestra.run.smithi039.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 2.5 MB/s | 237 kB 00:00 2024-05-24T11:28:03.645 INFO:teuthology.orchestra.run.smithi039.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.4 MB/s | 180 kB 00:00 2024-05-24T11:28:03.668 INFO:teuthology.orchestra.run.smithi053.stdout:(135/146): librbd1-18.2.2-1349.gb9b067bc.el8.x8 4.3 MB/s | 3.6 MB 00:00 2024-05-24T11:28:03.704 INFO:teuthology.orchestra.run.smithi039.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 623 kB/s | 99 kB 00:00 2024-05-24T11:28:03.777 INFO:teuthology.orchestra.run.smithi053.stdout:(136/146): librados2-18.2.2-1349.gb9b067bc.el8. 3.8 MB/s | 3.7 MB 00:00 2024-05-24T11:28:03.805 INFO:teuthology.orchestra.run.smithi039.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 11 MB/s | 2.5 MB 00:00 2024-05-24T11:28:03.811 INFO:teuthology.orchestra.run.smithi053.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.3 MB/s | 333 kB 00:00 2024-05-24T11:28:03.821 INFO:teuthology.orchestra.run.smithi039.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 586 kB/s | 103 kB 00:00 2024-05-24T11:28:03.825 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-05-24T11:28:03.825 INFO:teuthology.orchestra.run.smithi039.stdout:Total 10 MB/s | 215 MB 00:20 2024-05-24T11:28:03.861 INFO:teuthology.orchestra.run.smithi053.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-05-24T11:28:03.918 INFO:teuthology.orchestra.run.smithi053.stdout:(139/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 4.0 MB/s | 1.5 MB 00:00 2024-05-24T11:28:03.953 INFO:teuthology.orchestra.run.smithi053.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 221 kB 00:00 2024-05-24T11:28:03.978 INFO:teuthology.orchestra.run.smithi053.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 254 kB 00:00 2024-05-24T11:28:04.045 INFO:teuthology.orchestra.run.smithi053.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.8 MB/s | 237 kB 00:00 2024-05-24T11:28:04.071 INFO:teuthology.orchestra.run.smithi053.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.5 MB/s | 180 kB 00:00 2024-05-24T11:28:04.096 INFO:teuthology.orchestra.run.smithi053.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 846 kB/s | 99 kB 00:00 2024-05-24T11:28:04.204 INFO:teuthology.orchestra.run.smithi053.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 16 MB/s | 2.5 MB 00:00 2024-05-24T11:28:04.229 INFO:teuthology.orchestra.run.smithi053.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 649 kB/s | 103 kB 00:00 2024-05-24T11:28:04.232 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-05-24T11:28:04.233 INFO:teuthology.orchestra.run.smithi053.stdout:Total 11 MB/s | 215 MB 00:19 2024-05-24T11:28:04.606 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-05-24T11:28:04.740 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-24T11:28:04.891 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-05-24T11:28:04.985 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-24T11:28:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-24T11:28:05.318 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-05-24T11:28:05.318 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-05-24T11:28:05.336 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-05-24T11:28:05.370 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-24T11:28:05.558 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-24T11:28:05.784 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-05-24T11:28:05.784 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-05-24T11:28:06.033 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-24T11:28:06.233 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-24T11:28:06.294 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-24T11:28:06.513 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-24T11:28:06.586 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-05-24T11:28:06.597 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-05-24T11:28:06.691 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-24T11:28:06.892 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-24T11:28:06.993 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-24T11:28:07.108 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-05-24T11:28:07.119 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-05-24T11:28:07.216 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-24T11:28:07.624 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-24T11:28:07.824 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 9/162 2024-05-24T11:28:08.213 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 9/162 2024-05-24T11:28:08.383 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 10/162 2024-05-24T11:28:08.535 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 10/162 2024-05-24T11:28:08.708 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 11/162 2024-05-24T11:28:08.789 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 11/162 2024-05-24T11:28:09.126 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:18.2.2-1349.gb9b067bc.el8.x86_64 12/162 2024-05-24T11:28:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-24T11:28:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-24T11:28:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-24T11:28:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-24T11:28:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-24T11:28:10.529 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-24T11:28:10.753 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-24T11:28:10.932 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 19/162 2024-05-24T11:28:11.023 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 19/162 2024-05-24T11:28:11.174 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86 20/162 2024-05-24T11:28:11.279 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86 20/162 2024-05-24T11:28:11.469 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-24T11:28:11.727 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-24T11:28:11.963 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-24T11:28:12.140 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-24T11:28:12.267 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-24T11:28:12.360 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-05-24T11:28:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-24T11:28:12.502 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-24T11:28:12.902 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-24T11:28:12.997 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-05-24T11:28:13.040 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-24T11:28:13.130 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-24T11:28:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-24T11:28:13.202 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-24T11:28:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-24T11:28:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-24T11:28:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-24T11:28:13.591 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-24T11:28:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-24T11:28:13.775 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-24T11:28:13.797 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-24T11:28:13.805 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-24T11:28:13.868 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-24T11:28:13.953 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-24T11:28:13.965 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-24T11:28:14.017 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-24T11:28:14.086 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-24T11:28:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-24T11:28:14.277 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-24T11:28:14.282 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-24T11:28:14.481 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-24T11:28:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-24T11:28:14.587 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-24T11:28:14.634 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-24T11:28:14.661 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-24T11:28:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-24T11:28:14.789 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-24T11:28:14.913 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-24T11:28:14.922 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-24T11:28:15.136 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-24T11:28:15.137 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-24T11:28:15.221 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-24T11:28:15.383 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-24T11:28:15.455 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 9/162 2024-05-24T11:28:15.575 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-24T11:28:15.599 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:18.2.2-1349.gb9b067bc.el8. 31/162 2024-05-24T11:28:15.734 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:18.2.2-1349.gb9b067bc.el 32/162 2024-05-24T11:28:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 9/162 2024-05-24T11:28:15.892 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 9/162 2024-05-24T11:28:16.011 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:18.2.2-1349.gb9b067bc.el8.x86_6 33/162 2024-05-24T11:28:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 10/162 2024-05-24T11:28:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-24T11:28:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 10/162 2024-05-24T11:28:16.233 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 9/162 2024-05-24T11:28:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 10/162 2024-05-24T11:28:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 11/162 2024-05-24T11:28:16.516 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 10/162 2024-05-24T11:28:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 11/162 2024-05-24T11:28:16.711 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 11/162 2024-05-24T11:28:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 11/162 2024-05-24T11:28:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:18.2.2-1349.gb9b067bc.el8.x86_64 12/162 2024-05-24T11:28:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-24T11:28:17.180 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:18.2.2-1349.gb9b067bc.el8.x86_64 12/162 2024-05-24T11:28:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-24T11:28:17.453 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-24T11:28:17.514 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-24T11:28:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-24T11:28:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-24T11:28:17.933 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-24T11:28:17.934 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-24T11:28:18.001 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-24T11:28:18.135 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-24T11:28:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-24T11:28:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-24T11:28:18.564 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-24T11:28:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-24T11:28:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-24T11:28:18.841 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-24T11:28:18.986 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 19/162 2024-05-24T11:28:18.988 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 19/162 2024-05-24T11:28:19.085 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 19/162 2024-05-24T11:28:19.100 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 19/162 2024-05-24T11:28:19.244 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86 20/162 2024-05-24T11:28:19.249 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-24T11:28:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86 20/162 2024-05-24T11:28:19.313 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86 20/162 2024-05-24T11:28:19.355 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86 20/162 2024-05-24T11:28:19.477 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-24T11:28:19.552 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-24T11:28:19.569 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-24T11:28:19.603 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-24T11:28:19.744 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-24T11:28:19.792 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-24T11:28:19.796 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-24T11:28:20.033 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-24T11:28:20.153 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-24T11:28:20.250 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-24T11:28:20.305 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-24T11:28:20.432 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-24T11:28:20.461 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 41/162 2024-05-24T11:28:20.471 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-24T11:28:20.598 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-24T11:28:20.613 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-24T11:28:20.621 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:18.2.2-1349.gb9b067bc.el8.x86_6 42/162 2024-05-24T11:28:20.839 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-24T11:28:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-24T11:28:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-24T11:28:21.181 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-24T11:28:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-24T11:28:21.298 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-24T11:28:21.426 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-24T11:28:21.444 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-24T11:28:21.649 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-24T11:28:21.659 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-24T11:28:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-24T11:28:22.218 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-24T11:28:22.238 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-24T11:28:22.291 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-24T11:28:22.451 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-24T11:28:22.463 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-24T11:28:22.565 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-24T11:28:22.590 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-24T11:28:22.628 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-24T11:28:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-16.8-9.el8.noarch 49/162 2024-05-24T11:28:22.801 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-24T11:28:22.989 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-05-24T11:28:23.071 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-05-24T11:28:23.110 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-24T11:28:23.207 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-24T11:28:23.357 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : jq-1.6-7.el8.x86_64 51/162 2024-05-24T11:28:23.396 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-24T11:28:23.494 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-24T11:28:23.589 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.1-8.el8.noarch 52/162 2024-05-24T11:28:23.648 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-24T11:28:23.689 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-24T11:28:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-43.el8.x86_64 53/162 2024-05-24T11:28:23.873 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:18.2.2-1349.gb9b067bc.el8. 31/162 2024-05-24T11:28:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 53/162 2024-05-24T11:28:23.952 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-24T11:28:24.016 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:18.2.2-1349.gb9b067bc.el 32/162 2024-05-24T11:28:24.070 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 54/162 2024-05-24T11:28:24.266 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:18.2.2-1349.gb9b067bc.el8. 31/162 2024-05-24T11:28:24.271 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 55/162 2024-05-24T11:28:24.292 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:18.2.2-1349.gb9b067bc.el8.x86_6 33/162 2024-05-24T11:28:24.414 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 56/162 2024-05-24T11:28:24.434 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:18.2.2-1349.gb9b067bc.el 32/162 2024-05-24T11:28:24.634 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-24T11:28:24.719 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:18.2.2-1349.gb9b067bc.el8.x86_6 33/162 2024-05-24T11:28:24.828 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-24T11:28:24.938 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-24T11:28:25.035 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-24T11:28:25.191 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-24T11:28:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-24T11:28:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-24T11:28:25.683 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-24T11:28:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-24T11:28:25.978 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-24T11:28:26.138 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-24T11:28:26.193 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-24T11:28:26.335 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-24T11:28:26.367 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-24T11:28:26.558 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-24T11:28:26.817 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-24T11:28:26.957 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-24T11:28:27.187 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-24T11:28:27.242 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-24T11:28:27.503 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-24T11:28:27.713 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-24T11:28:27.919 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-24T11:28:28.176 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-24T11:28:28.309 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-24T11:28:28.325 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-24T11:28:28.451 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-24T11:28:28.638 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-24T11:28:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-24T11:28:28.701 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-24T11:28:28.865 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-24T11:28:28.922 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-24T11:28:28.995 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-24T11:28:29.059 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-24T11:28:29.133 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-24T11:28:29.252 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-24T11:28:29.347 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-24T11:28:29.359 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 41/162 2024-05-24T11:28:29.466 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-24T11:28:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-24T11:28:29.511 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:18.2.2-1349.gb9b067bc.el8.x86_6 42/162 2024-05-24T11:28:29.668 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 41/162 2024-05-24T11:28:29.714 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-24T11:28:29.718 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-24T11:28:29.798 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-24T11:28:29.862 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:18.2.2-1349.gb9b067bc.el8.x86_6 42/162 2024-05-24T11:28:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-24T11:28:29.989 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-24T11:28:30.049 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-24T11:28:30.053 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-24T11:28:30.154 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-24T11:28:30.203 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-24T11:28:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-24T11:28:30.353 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-24T11:28:30.404 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-24T11:28:30.457 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-24T11:28:30.513 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-24T11:28:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-24T11:28:30.687 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-24T11:28:30.726 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-24T11:28:30.764 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-24T11:28:30.882 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-24T11:28:30.992 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-24T11:28:31.112 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-24T11:28:31.186 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-24T11:28:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-24T11:28:31.383 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-24T11:28:31.527 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-24T11:28:31.552 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-24T11:28:31.573 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-24T11:28:31.585 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-24T11:28:31.739 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-24T11:28:31.919 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-24T11:28:31.986 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-24T11:28:31.987 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-24T11:28:32.014 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-24T11:28:32.175 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-24T11:28:32.216 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-packaging-16.8-9.el8.noarch 49/162 2024-05-24T11:28:32.317 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-24T11:28:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-24T11:28:32.415 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-05-24T11:28:32.493 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-24T11:28:32.528 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 50/162 2024-05-24T11:28:32.565 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-24T11:28:32.794 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : jq-1.6-7.el8.x86_64 51/162 2024-05-24T11:28:32.808 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-24T11:28:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-24T11:28:33.039 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : automake-1.16.1-8.el8.noarch 52/162 2024-05-24T11:28:33.079 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-24T11:28:33.173 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-24T11:28:33.245 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : uuid-1.6.2-43.el8.x86_64 53/162 2024-05-24T11:28:33.261 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-24T11:28:33.323 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 53/162 2024-05-24T11:28:33.373 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-24T11:28:33.521 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-24T11:28:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-24T11:28:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 54/162 2024-05-24T11:28:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-24T11:28:33.870 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 55/162 2024-05-24T11:28:34.089 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 56/162 2024-05-24T11:28:34.198 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-24T11:28:34.266 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-24T11:28:34.504 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-24T11:28:34.533 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-24T11:28:34.594 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-24T11:28:34.709 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-24T11:28:34.855 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-24T11:28:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-24T11:28:34.899 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-24T11:28:35.052 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-24T11:28:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-24T11:28:35.119 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-24T11:28:35.292 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-24T11:28:35.412 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-24T11:28:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-24T11:28:35.534 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-24T11:28:35.662 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-24T11:28:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-24T11:28:35.811 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-24T11:28:35.886 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-24T11:28:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-24T11:28:36.170 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-24T11:28:36.319 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-24T11:28:36.353 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-24T11:28:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-24T11:28:36.578 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-24T11:28:36.668 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-24T11:28:36.798 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-24T11:28:36.828 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 103/162 2024-05-24T11:28:36.904 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 103/162 2024-05-24T11:28:36.987 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-24T11:28:37.014 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-05-24T11:28:37.104 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 104/162 2024-05-24T11:28:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-24T11:28:37.294 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-24T11:28:37.333 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-24T11:28:37.612 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-24T11:28:37.637 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-24T11:28:37.870 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-24T11:28:37.878 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-24T11:28:38.050 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-24T11:28:38.081 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-24T11:28:38.112 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-24T11:28:38.315 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-24T11:28:38.342 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-24T11:28:38.526 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-24T11:28:38.569 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-24T11:28:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-24T11:28:38.821 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-24T11:28:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-24T11:28:39.047 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-24T11:28:39.095 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-24T11:28:39.275 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-24T11:28:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-24T11:28:39.580 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-24T11:28:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-24T11:28:39.745 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-24T11:28:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-24T11:28:39.984 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-24T11:28:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-24T11:28:40.282 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-24T11:28:40.302 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-24T11:28:40.543 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-24T11:28:40.549 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-24T11:28:40.707 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-24T11:28:40.779 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 105/162 2024-05-24T11:28:40.790 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-24T11:28:40.890 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-24T11:28:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-24T11:28:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 105/162 2024-05-24T11:28:41.148 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-24T11:28:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-24T11:28:41.508 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-24T11:28:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 105/162 2024-05-24T11:28:41.630 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-24T11:28:41.742 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-24T11:28:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-24T11:28:41.896 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 106/162 2024-05-24T11:28:41.937 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-24T11:28:41.981 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 106/162 2024-05-24T11:28:42.105 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-24T11:28:42.136 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 107/162 2024-05-24T11:28:42.147 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-24T11:28:42.325 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-24T11:28:42.334 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-24T11:28:42.526 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-24T11:28:42.560 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-24T11:28:42.782 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-24T11:28:42.787 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-24T11:28:42.983 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-24T11:28:43.230 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-24T11:28:43.468 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-24T11:28:43.703 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-24T11:28:43.981 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-24T11:28:44.017 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-24T11:28:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-24T11:28:44.580 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-24T11:28:44.908 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-24T11:28:45.157 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-24T11:28:45.177 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-24T11:28:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-24T11:28:45.483 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-24T11:28:45.720 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-24T11:28:46.023 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-24T11:28:46.226 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-24T11:28:46.295 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-24T11:28:46.495 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-24T11:28:46.607 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 103/162 2024-05-24T11:28:46.684 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 103/162 2024-05-24T11:28:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 104/162 2024-05-24T11:28:47.790 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-24T11:28:47.984 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 103/162 2024-05-24T11:28:48.046 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 103/162 2024-05-24T11:28:48.235 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 104/162 2024-05-24T11:28:51.076 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 105/162 2024-05-24T11:28:51.299 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 105/162 2024-05-24T11:28:51.962 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 105/162 2024-05-24T11:28:52.205 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 105/162 2024-05-24T11:28:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 106/162 2024-05-24T11:28:53.109 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 106/162 2024-05-24T11:28:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 107/162 2024-05-24T11:28:53.310 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 105/162 2024-05-24T11:28:53.945 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 105/162 2024-05-24T11:28:54.255 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 106/162 2024-05-24T11:28:54.367 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 106/162 2024-05-24T11:28:54.509 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 107/162 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 107/162 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /dev 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /run 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /dev 2024-05-24T11:29:01.812 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:29:02.150 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 108/162 2024-05-24T11:29:03.785 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 108/162 2024-05-24T11:29:03.985 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 109/162 2024-05-24T11:29:04.203 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 109/162 2024-05-24T11:29:04.400 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 110/162 2024-05-24T11:29:04.874 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 110/162 2024-05-24T11:29:05.033 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 111/162 2024-05-24T11:29:05.051 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 111/162 2024-05-24T11:29:06.814 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 112/162 2024-05-24T11:29:06.953 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 112/162 2024-05-24T11:29:07.064 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1349.gb9b067bc.e 113/162 2024-05-24T11:29:07.274 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1349.gb9b067bc. 114/162 2024-05-24T11:29:07.526 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noa 115/162 2024-05-24T11:29:08.557 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noa 115/162 2024-05-24T11:29:08.800 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.n 116/162 2024-05-24T11:29:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.n 116/162 2024-05-24T11:29:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b 117/162 2024-05-24T11:29:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b 117/162 2024-05-24T11:29:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1349.gb9b067bc.el 118/162 2024-05-24T11:29:10.625 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 119/162 2024-05-24T11:29:10.664 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 119/162 2024-05-24T11:29:10.823 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 120/162 2024-05-24T11:29:10.835 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 120/162 2024-05-24T11:29:12.122 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:18.2.2-1349.gb9b067bc.el8.x86_64 121/162 2024-05-24T11:29:12.341 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 122/162 2024-05-24T11:29:12.376 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 122/162 2024-05-24T11:29:12.532 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1349.gb9b06 123/162 2024-05-24T11:29:12.575 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1349.gb9b06 123/162 2024-05-24T11:29:12.781 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 124/162 2024-05-24T11:29:13.129 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 124/162 2024-05-24T11:29:13.315 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 125/162 2024-05-24T11:29:14.723 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 107/162 2024-05-24T11:29:14.723 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2024-05-24T11:29:14.723 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2024-05-24T11:29:14.723 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /dev 2024-05-24T11:29:14.723 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /run 2024-05-24T11:29:14.724 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2024-05-24T11:29:14.724 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2024-05-24T11:29:14.724 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2024-05-24T11:29:14.724 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2024-05-24T11:29:14.724 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /dev 2024-05-24T11:29:14.724 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:29:15.115 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 108/162 2024-05-24T11:29:15.312 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 107/162 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /run 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2024-05-24T11:29:15.313 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:29:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 108/162 2024-05-24T11:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 108/162 2024-05-24T11:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 109/162 2024-05-24T11:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 109/162 2024-05-24T11:29:17.456 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 110/162 2024-05-24T11:29:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 108/162 2024-05-24T11:29:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 109/162 2024-05-24T11:29:17.760 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 125/162 2024-05-24T11:29:17.961 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 110/162 2024-05-24T11:29:17.969 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 109/162 2024-05-24T11:29:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 110/162 2024-05-24T11:29:18.172 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 111/162 2024-05-24T11:29:18.191 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 111/162 2024-05-24T11:29:18.690 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 110/162 2024-05-24T11:29:18.755 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:18.2.2-1349.gb9b067bc.el8.x86_64 126/162 2024-05-24T11:29:18.838 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 111/162 2024-05-24T11:29:18.860 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 111/162 2024-05-24T11:29:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-24T11:29:19.312 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-24T11:29:19.690 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 129/162 2024-05-24T11:29:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-24T11:29:20.205 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-24T11:29:20.218 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-24T11:29:20.306 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 112/162 2024-05-24T11:29:20.441 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-24T11:29:20.488 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 112/162 2024-05-24T11:29:20.609 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-24T11:29:20.620 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1349.gb9b067bc.e 113/162 2024-05-24T11:29:20.756 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:18.2.2-1349.gb9b067bc.el8.x86_ 134/162 2024-05-24T11:29:20.866 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1349.gb9b067bc. 114/162 2024-05-24T11:29:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 112/162 2024-05-24T11:29:21.071 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 112/162 2024-05-24T11:29:21.143 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noa 115/162 2024-05-24T11:29:21.212 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1349.gb9b067bc.e 113/162 2024-05-24T11:29:21.417 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1349.gb9b067bc. 114/162 2024-05-24T11:29:21.667 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noa 115/162 2024-05-24T11:29:22.235 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noa 115/162 2024-05-24T11:29:22.501 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.n 116/162 2024-05-24T11:29:22.812 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noa 115/162 2024-05-24T11:29:23.033 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.n 116/162 2024-05-24T11:29:23.082 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-24T11:29:23.381 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-24T11:29:23.418 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-24T11:29:23.527 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.n 116/162 2024-05-24T11:29:23.688 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b 117/162 2024-05-24T11:29:23.787 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b 117/162 2024-05-24T11:29:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-24T11:29:24.108 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.n 116/162 2024-05-24T11:29:24.156 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1349.gb9b067bc.el 118/162 2024-05-24T11:29:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-24T11:29:24.252 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b 117/162 2024-05-24T11:29:24.335 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 119/162 2024-05-24T11:29:24.363 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b 117/162 2024-05-24T11:29:24.375 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 119/162 2024-05-24T11:29:24.450 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-24T11:29:24.563 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 120/162 2024-05-24T11:29:24.576 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 120/162 2024-05-24T11:29:24.663 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:18.2.2-1349.gb9b067bc.el8.noarch 139/162 2024-05-24T11:29:24.719 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1349.gb9b067bc.el 118/162 2024-05-24T11:29:24.871 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 140/162 2024-05-24T11:29:24.917 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 119/162 2024-05-24T11:29:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 119/162 2024-05-24T11:29:25.120 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 120/162 2024-05-24T11:29:25.135 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 120/162 2024-05-24T11:29:25.198 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 141/162 2024-05-24T11:29:25.508 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-24T11:29:25.704 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-24T11:29:25.785 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-24T11:29:25.948 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:18.2.2-1349.gb9b067bc.el8.x86_64 121/162 2024-05-24T11:29:26.012 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-24T11:29:26.026 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-24T11:29:26.142 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 122/162 2024-05-24T11:29:26.179 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 122/162 2024-05-24T11:29:26.221 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-24T11:29:26.230 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-24T11:29:26.363 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1349.gb9b06 123/162 2024-05-24T11:29:26.390 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-24T11:29:26.409 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1349.gb9b06 123/162 2024-05-24T11:29:26.560 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:18.2.2-1349.gb9b067bc.el8.x86_64 121/162 2024-05-24T11:29:26.568 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-24T11:29:26.596 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 124/162 2024-05-24T11:29:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 122/162 2024-05-24T11:29:26.764 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-24T11:29:26.765 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 122/162 2024-05-24T11:29:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-24T11:29:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-24T11:29:26.961 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1349.gb9b06 123/162 2024-05-24T11:29:26.964 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 124/162 2024-05-24T11:29:27.012 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1349.gb9b06 123/162 2024-05-24T11:29:27.178 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 124/162 2024-05-24T11:29:27.188 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 125/162 2024-05-24T11:29:27.207 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-24T11:29:27.207 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-24T11:29:27.399 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-24T11:29:27.399 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-24T11:29:27.564 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 124/162 2024-05-24T11:29:27.651 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-24T11:29:27.761 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 125/162 2024-05-24T11:29:27.897 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-24T11:29:28.072 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-24T11:29:28.339 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-24T11:29:28.339 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-24T11:29:28.573 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-24T11:29:28.573 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-24T11:29:28.856 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-24T11:29:28.856 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-24T11:29:29.081 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-24T11:29:29.081 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-24T11:29:29.306 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-24T11:29:29.306 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-24T11:29:29.497 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-24T11:29:29.664 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-24T11:29:29.665 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-24T11:29:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:18.2.2-1349.gb9b067bc.el8.x86_64 1/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 2/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 3/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 4/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1349.gb9b06 5/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 6/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 7/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 8/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 9/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 10/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 11/162 2024-05-24T11:29:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:18.2.2-1349.gb9b067bc.el8.x86_64 12/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 13/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:18.2.2-1349.gb9b067bc.el8.x86_ 14/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 15/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 16/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:18.2.2-1349.gb9b067bc.el8.x86_6 17/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86 18/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 19/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1349.gb9b067bc.el 20/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:18.2.2-1349.gb9b067bc.el8. 21/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:18.2.2-1349.gb9b067bc.el8.x86_6 22/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:18.2.2-1349.gb9b067bc.el8.x86_64 23/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 24/162 2024-05-24T11:29:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 25/162 2024-05-24T11:29:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 26/162 2024-05-24T11:29:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 27/162 2024-05-24T11:29:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 28/162 2024-05-24T11:29:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1349.gb9b067bc. 29/162 2024-05-24T11:29:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noa 30/162 2024-05-24T11:29:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.n 31/162 2024-05-24T11:29:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b 32/162 2024-05-24T11:29:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1349.gb9b067bc.el 33/162 2024-05-24T11:29:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 34/162 2024-05-24T11:29:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1349.gb9b067bc.e 35/162 2024-05-24T11:29:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 36/162 2024-05-24T11:29:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 37/162 2024-05-24T11:29:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:18.2.2-1349.gb9b067bc.el8.noarch 38/162 2024-05-24T11:29:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-24T11:29:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-24T11:29:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-24T11:29:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-24T11:29:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-24T11:29:30.574 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-24T11:29:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/162 2024-05-24T11:29:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 110/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 111/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 113/162 2024-05-24T11:29:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 115/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 117/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 118/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 119/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 120/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.1-8.el8.noarch 121/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpq-13.11-1.el8.x86_64 122/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : jq-1.6-7.el8.x86_64 123/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 124/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 125/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 126/162 2024-05-24T11:29:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 127/162 2024-05-24T11:29:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 128/162 2024-05-24T11:29:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 129/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 130/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 131/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 133/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-24T11:29:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-24T11:29:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-24T11:29:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-24T11:29:31.933 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 125/162 2024-05-24T11:29:32.695 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 125/162 2024-05-24T11:29:32.989 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:18.2.2-1349.gb9b067bc.el8.x86_64 126/162 2024-05-24T11:29:33.204 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:18.2.2-1349.gb9b067bc.el8.x86_64 126/162 2024-05-24T11:29:33.309 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-24T11:29:33.563 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-24T11:29:33.608 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-24T11:29:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-24T11:29:34.048 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 129/162 2024-05-24T11:29:34.393 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-24T11:29:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-24T11:29:34.645 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-24T11:29:34.659 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-24T11:29:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-24T11:29:34.916 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-24T11:29:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-24T11:29:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-24T11:29:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:18.2.2-1349.gb9b067bc.el8.x86_ 134/162 2024-05-24T11:29:35.443 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 131/162 2024-05-24T11:29:35.774 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-24T11:29:35.990 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-24T11:29:36.144 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:18.2.2-1349.gb9b067bc.el8.x86_ 134/162 2024-05-24T11:29:37.701 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-24T11:29:38.020 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-24T11:29:38.059 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-24T11:29:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-24T11:29:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-24T11:29:38.898 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-24T11:29:39.043 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-24T11:29:39.085 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-24T11:29:39.149 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-24T11:29:39.345 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-top-2:18.2.2-1349.gb9b067bc.el8.noarch 139/162 2024-05-24T11:29:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-24T11:29:39.579 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 140/162 2024-05-24T11:29:39.885 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 141/162 2024-05-24T11:29:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-24T11:29:40.226 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-24T11:29:40.256 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-24T11:29:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:18.2.2-1349.gb9b067bc.el8.noarch 139/162 2024-05-24T11:29:40.477 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-24T11:29:40.563 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-24T11:29:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 140/162 2024-05-24T11:29:40.802 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-24T11:29:40.818 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-24T11:29:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 141/162 2024-05-24T11:29:41.036 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-24T11:29:41.047 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-24T11:29:41.239 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-24T11:29:41.270 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-24T11:29:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-24T11:29:41.437 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-24T11:29:41.446 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-24T11:29:41.654 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-24T11:29:41.677 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-24T11:29:41.832 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-24T11:29:41.832 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-24T11:29:41.912 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-24T11:29:42.000 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-24T11:29:42.114 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-24T11:29:42.114 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-24T11:29:42.192 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-24T11:29:42.208 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-24T11:29:42.247 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-24T11:29:42.247 INFO:teuthology.orchestra.run.smithi137.stdout:Installed products updated. 2024-05-24T11:29:42.373 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-24T11:29:42.373 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-24T11:29:42.403 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-24T11:29:42.570 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-24T11:29:42.651 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-24T11:29:42.713 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-24T11:29:42.714 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-24T11:29:42.853 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:29:42.853 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-24T11:29:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.1-8.el8.noarch 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.0.4-10.el8.x86_64 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.1-9.el8.x86_64 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-24T11:29:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: jq-1.6-7.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: libpq-13.11-1.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-24T11:29:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-24T11:29:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-24T11:29:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-24T11:29:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-24T11:29:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-24T11:29:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-24T11:29:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-24T11:29:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-24T11:29:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-2.6-1.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-24T11:29:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-24T11:29:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-24T11:29:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:29:42.869 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-05-24T11:29:42.904 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-24T11:29:43.004 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-24T11:29:43.004 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-24T11:29:43.091 DEBUG:teuthology.parallel:result is None 2024-05-24T11:29:43.107 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-24T11:29:43.255 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-24T11:29:43.255 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-24T11:29:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-24T11:29:43.379 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-24T11:29:43.508 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-24T11:29:43.638 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-24T11:29:43.639 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-24T11:29:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-24T11:29:43.872 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-24T11:29:43.872 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-24T11:29:43.913 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-24T11:29:44.130 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-24T11:29:44.130 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-24T11:29:44.169 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-24T11:29:44.170 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-24T11:29:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-24T11:29:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-24T11:29:44.406 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-24T11:29:44.406 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-24T11:29:44.637 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-24T11:29:44.637 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-24T11:29:44.821 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-24T11:29:44.845 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-24T11:29:44.846 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-24T11:29:45.013 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-24T11:29:45.013 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-24T11:29:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-24T11:29:45.087 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-24T11:29:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-24T11:29:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-24T11:29:45.470 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:18.2.2-1349.gb9b067bc.el8.x86_64 1/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 2/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 3/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 4/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1349.gb9b06 5/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 6/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 7/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 8/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 9/162 2024-05-24T11:29:45.999 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 10/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 11/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:18.2.2-1349.gb9b067bc.el8.x86_64 12/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 13/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:18.2.2-1349.gb9b067bc.el8.x86_ 14/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 15/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 16/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:18.2.2-1349.gb9b067bc.el8.x86_6 17/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86 18/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 19/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1349.gb9b067bc.el 20/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:18.2.2-1349.gb9b067bc.el8. 21/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:18.2.2-1349.gb9b067bc.el8.x86_6 22/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:18.2.2-1349.gb9b067bc.el8.x86_64 23/162 2024-05-24T11:29:46.000 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 24/162 2024-05-24T11:29:46.001 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 25/162 2024-05-24T11:29:46.001 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 26/162 2024-05-24T11:29:46.001 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 27/162 2024-05-24T11:29:46.001 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 28/162 2024-05-24T11:29:46.001 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1349.gb9b067bc. 29/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noa 30/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.n 31/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b 32/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1349.gb9b067bc.el 33/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 34/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1349.gb9b067bc.e 35/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 36/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 37/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-top-2:18.2.2-1349.gb9b067bc.el8.noarch 38/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-24T11:29:46.013 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-24T11:29:46.014 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-24T11:29:46.015 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-24T11:29:46.016 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-24T11:29:46.017 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/162 2024-05-24T11:29:46.018 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 110/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 111/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 113/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 115/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 117/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 118/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 119/162 2024-05-24T11:29:46.019 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 120/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : automake-1.16.1-8.el8.noarch 121/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpq-13.11-1.el8.x86_64 122/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : jq-1.6-7.el8.x86_64 123/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 124/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 125/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 126/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 127/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 128/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 129/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 130/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 131/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-24T11:29:46.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 133/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-24T11:29:46.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-24T11:29:46.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-24T11:29:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:18.2.2-1349.gb9b067bc.el8.x86_64 1/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 2/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 3/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 4/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1349.gb9b06 5/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 6/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 7/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 8/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 9/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 10/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 11/162 2024-05-24T11:29:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:18.2.2-1349.gb9b067bc.el8.x86_64 12/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 13/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:18.2.2-1349.gb9b067bc.el8.x86_ 14/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 15/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 16/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:18.2.2-1349.gb9b067bc.el8.x86_6 17/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86 18/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 19/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1349.gb9b067bc.el 20/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:18.2.2-1349.gb9b067bc.el8. 21/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:18.2.2-1349.gb9b067bc.el8.x86_6 22/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:18.2.2-1349.gb9b067bc.el8.x86_64 23/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 24/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 25/162 2024-05-24T11:29:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 26/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 27/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 28/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1349.gb9b067bc. 29/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noa 30/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.n 31/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b 32/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1349.gb9b067bc.el 33/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 34/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1349.gb9b067bc.e 35/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 36/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 37/162 2024-05-24T11:29:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:18.2.2-1349.gb9b067bc.el8.noarch 38/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-24T11:29:46.459 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-24T11:29:46.460 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-24T11:29:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-24T11:29:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-24T11:29:46.463 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-24T11:29:46.464 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 131/162 2024-05-24T11:29:46.465 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 133/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-24T11:29:46.466 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-24T11:29:46.467 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-24T11:29:46.468 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-24T11:29:46.468 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-24T11:29:59.195 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-24T11:29:59.195 INFO:teuthology.orchestra.run.smithi039.stdout:Installed products updated. 2024-05-24T11:29:59.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:29:59.837 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-05-24T11:29:59.837 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-24T11:29:59.837 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-24T11:29:59.838 INFO:teuthology.orchestra.run.smithi039.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-24T11:29:59.838 INFO:teuthology.orchestra.run.smithi039.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:59.838 INFO:teuthology.orchestra.run.smithi039.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:59.838 INFO:teuthology.orchestra.run.smithi039.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:59.838 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.838 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.838 INFO:teuthology.orchestra.run.smithi039.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:59.838 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:59.838 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: automake-1.16.1-8.el8.noarch 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: bison-3.0.4-10.el8.x86_64 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.839 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.840 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-24T11:29:59.841 INFO:teuthology.orchestra.run.smithi039.stdout: flex-2.6.1-9.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: jq-1.6-7.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-24T11:29:59.842 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: libpq-13.11-1.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-24T11:29:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-24T11:29:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-24T11:29:59.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-24T11:29:59.846 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-2.6-1.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-24T11:29:59.847 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-24T11:29:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-24T11:29:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-24T11:29:59.850 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-24T11:29:59.850 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-24T11:29:59.850 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-24T11:29:59.850 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:29:59.850 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-05-24T11:30:00.046 DEBUG:teuthology.parallel:result is None 2024-05-24T11:30:00.407 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-24T11:30:00.407 INFO:teuthology.orchestra.run.smithi053.stdout:Installed products updated. 2024-05-24T11:30:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:30:01.093 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-05-24T11:30:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-24T11:30:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-24T11:30:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-24T11:30:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:30:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-24T11:30:01.094 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.1-8.el8.noarch 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.0.4-10.el8.x86_64 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:18.2.2-1349.gb9b067bc.el8.noarch 2024-05-24T11:30:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.1-9.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-7.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-24T11:30:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: libpq-13.11-1.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-24T11:30:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-24T11:30:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-24T11:30:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-24T11:30:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-2.6-1.el8.noarch 2024-05-24T11:30:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-24T11:30:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-24T11:30:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-24T11:30:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-24T11:30:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-24T11:30:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-24T11:30:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-24T11:30:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:18.2.2-1349.gb9b067bc.el8.x86_64 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-24T11:30:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-24T11:30:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-24T11:30:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-05-24T11:30:01.105 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-05-24T11:30:01.313 DEBUG:teuthology.parallel:result is None 2024-05-24T11:30:01.315 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:30:01.455 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-24T11:30:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:18.2.2-1349.gb9b067bc.el8 2024-05-24T11:30:01.485 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1349.gb9b067bc.el8 2024-05-24T11:30:01.486 INFO:teuthology.task.install:The correct ceph version 18.2.2-1349.gb9b067bc is installed. 2024-05-24T11:30:01.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:30:01.628 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-24T11:30:01.659 INFO:teuthology.orchestra.run.smithi053.stdout:18.2.2-1349.gb9b067bc.el8 2024-05-24T11:30:01.659 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1349.gb9b067bc.el8 2024-05-24T11:30:01.659 INFO:teuthology.task.install:The correct ceph version 18.2.2-1349.gb9b067bc is installed. 2024-05-24T11:30:01.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:30:01.939 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-24T11:30:01.969 INFO:teuthology.orchestra.run.smithi137.stdout:18.2.2-1349.gb9b067bc.el8 2024-05-24T11:30:01.969 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1349.gb9b067bc.el8 2024-05-24T11:30:01.969 INFO:teuthology.task.install:The correct ceph version 18.2.2-1349.gb9b067bc is installed. 2024-05-24T11:30:01.973 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-24T11:30:01.973 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:30:01.973 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-24T11:30:02.026 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:30:02.026 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-24T11:30:02.081 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:30:02.081 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-24T11:30:02.131 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-24T11:30:02.132 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:30:02.132 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-05-24T11:30:02.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-24T11:30:02.236 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:30:02.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-05-24T11:30:02.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-24T11:30:02.347 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:30:02.347 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-05-24T11:30:02.383 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-24T11:30:02.454 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-24T11:30:02.455 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:30:02.455 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-24T11:30:02.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-24T11:30:02.560 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:30:02.560 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-24T11:30:02.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-24T11:30:02.669 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:30:02.669 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-24T11:30:02.701 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-24T11:30:02.772 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-24T11:30:02.772 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:30:02.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-05-24T11:30:02.805 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-24T11:30:02.878 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:30:02.879 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-05-24T11:30:02.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-24T11:30:02.989 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:30:02.990 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-05-24T11:30:03.022 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-24T11:30:03.093 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-24T11:30:03.176 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-24T11:30:03.177 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:30:03.177 INFO:tasks.cephadm:Cluster fsid is f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:30:03.177 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-24T11:30:03.178 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.39', 'mon.b': '172.21.15.53', 'mon.c': '172.21.15.137'} 2024-05-24T11:30:03.178 INFO:tasks.cephadm:First mon is mon.a on smithi039 2024-05-24T11:30:03.178 INFO:tasks.cephadm:First mgr is x 2024-05-24T11:30:03.178 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-24T11:30:03.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname $(hostname -s) 2024-05-24T11:30:03.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-05-24T11:30:03.248 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-05-24T11:30:03.279 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-24T11:30:03.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:30:03.554 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/55183/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1349-gb9b067bc', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1349.gb9b067bc'}, 'url': 'https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-19 16:19:00.550989', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-24T11:30:03.698 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 b9b067bc6d30ec00e391f0fa43f50affe981ba2f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-24T11:30:03.700 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/x86_64/flavors/default/cephadm 2024-05-24T11:30:03.700 DEBUG:teuthology.orchestra.run.smithi039:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-24T11:30:04.127 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 24 11:30 /home/ubuntu/cephtest/cephadm 2024-05-24T11:30:04.127 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-24T11:30:04.824 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 24 11:30 /home/ubuntu/cephtest/cephadm 2024-05-24T11:30:04.824 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-24T11:30:05.442 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 24 11:30 /home/ubuntu/cephtest/cephadm 2024-05-24T11:30:05.443 DEBUG:teuthology.orchestra.run.smithi039:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-24T11:30:05.465 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-24T11:30:05.488 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-24T11:30:05.525 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f on all hosts... 2024-05-24T11:30:05.525 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f pull 2024-05-24T11:30:05.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f pull 2024-05-24T11:30:05.530 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f pull 2024-05-24T11:30:06.487 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f... 2024-05-24T11:30:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f... 2024-05-24T11:30:07.107 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f... 2024-05-24T11:30:37.948 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-05-24T11:30:37.949 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 18.2.2-1349-gb9b067bc (b9b067bc6d30ec00e391f0fa43f50affe981ba2f) reef (stable)", 2024-05-24T11:30:37.949 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "5a1940d32fa011048f48611e92384965e43c0dce8f9087f5b96de9e456fd9588", 2024-05-24T11:30:37.949 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-05-24T11:30:37.949 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:28a65aa35b85be8a7ae9c38b142fa163279ca51d4fa673be28d851744eb6b442" 2024-05-24T11:30:37.949 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-05-24T11:30:37.949 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-05-24T11:30:38.358 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:30:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 18.2.2-1349-gb9b067bc (b9b067bc6d30ec00e391f0fa43f50affe981ba2f) reef (stable)", 2024-05-24T11:30:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "image_id": "5a1940d32fa011048f48611e92384965e43c0dce8f9087f5b96de9e456fd9588", 2024-05-24T11:30:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "repo_digests": [ 2024-05-24T11:30:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:28a65aa35b85be8a7ae9c38b142fa163279ca51d4fa673be28d851744eb6b442" 2024-05-24T11:30:38.359 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:30:38.359 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:30:46.367 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-24T11:30:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 18.2.2-1349-gb9b067bc (b9b067bc6d30ec00e391f0fa43f50affe981ba2f) reef (stable)", 2024-05-24T11:30:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "5a1940d32fa011048f48611e92384965e43c0dce8f9087f5b96de9e456fd9588", 2024-05-24T11:30:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-05-24T11:30:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:28a65aa35b85be8a7ae9c38b142fa163279ca51d4fa673be28d851744eb6b442" 2024-05-24T11:30:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-05-24T11:30:46.368 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-24T11:30:46.396 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph 2024-05-24T11:30:46.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-05-24T11:30:46.475 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-05-24T11:30:46.513 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /etc/ceph 2024-05-24T11:30:46.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-05-24T11:30:46.580 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-05-24T11:30:46.611 INFO:tasks.cephadm:Writing seed config... 2024-05-24T11:30:46.612 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-24T11:30:46.612 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-24T11:30:46.612 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-24T11:30:46.612 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-24T11:30:46.612 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-24T11:30:46.612 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-24T11:30:46.613 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-24T11:30:46.614 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-24T11:30:46.615 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-24T11:30:46.616 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-24T11:30:46.616 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:30:46.616 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-24T11:30:46.636 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f68008c4-19c0-11ef-bc9a-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-24T11:30:46.636 DEBUG:teuthology.orchestra.run.smithi039:mon.a> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.a.service 2024-05-24T11:30:46.679 DEBUG:teuthology.orchestra.run.smithi039:mgr.x> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-24T11:30:46.710 INFO:journalctl@ceph.mon.a.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:30:46.711 INFO:tasks.cephadm:Bootstrapping... 2024-05-24T11:30:46.711 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f -v bootstrap --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.39 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-24T11:30:46.740 INFO:journalctl@ceph.mgr.x.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:30:46.883 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-05-24T11:30:46.883 INFO:teuthology.orchestra.run.smithi039.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f', '-v', 'bootstrap', '--fsid', 'f68008c4-19c0-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.39', '--skip-admin-label'] 2024-05-24T11:30:46.929 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.6.1 2024-05-24T11:30:46.930 INFO:teuthology.orchestra.run.smithi039.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-24T11:30:46.930 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying podman|docker is present... 2024-05-24T11:30:46.969 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.6.1 2024-05-24T11:30:46.969 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying lvm2 is present... 2024-05-24T11:30:46.969 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying time synchronization is in place... 2024-05-24T11:30:46.976 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-24T11:30:46.976 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-24T11:30:46.983 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-24T11:30:46.984 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-24T11:30:46.988 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-05-24T11:30:46.994 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-05-24T11:30:46.994 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-05-24T11:30:46.994 INFO:teuthology.orchestra.run.smithi039.stdout:Repeating the final host check... 2024-05-24T11:30:47.025 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.6.1 2024-05-24T11:30:47.026 INFO:teuthology.orchestra.run.smithi039.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-24T11:30:47.026 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl is present 2024-05-24T11:30:47.026 INFO:teuthology.orchestra.run.smithi039.stdout:lvcreate is present 2024-05-24T11:30:47.032 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-24T11:30:47.032 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-24T11:30:47.037 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-24T11:30:47.038 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-24T11:30:47.044 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-05-24T11:30:47.050 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-05-24T11:30:47.050 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-05-24T11:30:47.050 INFO:teuthology.orchestra.run.smithi039.stdout:Host looks OK 2024-05-24T11:30:47.050 INFO:teuthology.orchestra.run.smithi039.stdout:Cluster fsid: f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:30:47.050 INFO:teuthology.orchestra.run.smithi039.stdout:Acquiring lock 140288662977000 on /run/cephadm/f68008c4-19c0-11ef-bc9a-c7b262605968.lock 2024-05-24T11:30:47.050 INFO:teuthology.orchestra.run.smithi039.stdout:Lock 140288662977000 acquired on /run/cephadm/f68008c4-19c0-11ef-bc9a-c7b262605968.lock 2024-05-24T11:30:47.050 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 3300 ... 2024-05-24T11:30:47.051 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 6789 ... 2024-05-24T11:30:47.051 INFO:teuthology.orchestra.run.smithi039.stdout:Base mon IP(s) is [172.21.15.39:3300, 172.21.15.39:6789], mon addrv is [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-05-24T11:30:47.055 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2024-05-24T11:30:47.055 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.39 metric 100 2024-05-24T11:30:47.057 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-24T11:30:47.058 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-05-24T11:30:47.058 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2024-05-24T11:30:47.061 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-24T11:30:47.061 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-24T11:30:47.061 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-24T11:30:47.061 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-24T11:30:47.061 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-05-24T11:30:47.061 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-24T11:30:47.062 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-05-24T11:30:47.062 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-05-24T11:30:47.062 INFO:teuthology.orchestra.run.smithi039.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-24T11:30:47.062 INFO:teuthology.orchestra.run.smithi039.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-24T11:30:47.063 INFO:teuthology.orchestra.run.smithi039.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f... 2024-05-24T11:30:47.598 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 5a1940d32fa011048f48611e92384965e43c0dce8f9087f5b96de9e456fd9588 2024-05-24T11:30:47.599 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f... 2024-05-24T11:30:47.599 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Getting image source signatures 2024-05-24T11:30:47.599 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:93912a0c397b14449e141c31407f5c1bd7432401ecc8325f482acd853bd2f1ec 2024-05-24T11:30:47.599 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-24T11:30:47.599 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying config sha256:5a1940d32fa011048f48611e92384965e43c0dce8f9087f5b96de9e456fd9588 2024-05-24T11:30:47.599 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-24T11:30:49.407 INFO:teuthology.orchestra.run.smithi039.stdout:ceph: stdout ceph version 18.2.2-1349-gb9b067bc (b9b067bc6d30ec00e391f0fa43f50affe981ba2f) reef (stable) 2024-05-24T11:30:49.407 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph version: ceph version 18.2.2-1349-gb9b067bc (b9b067bc6d30ec00e391f0fa43f50affe981ba2f) reef (stable) 2024-05-24T11:30:49.407 INFO:teuthology.orchestra.run.smithi039.stdout:Extracting ceph user uid/gid from container image... 2024-05-24T11:30:50.835 INFO:teuthology.orchestra.run.smithi039.stdout:stat: stdout 167 167 2024-05-24T11:30:50.835 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial keys... 2024-05-24T11:30:51.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQBrelBm/OHcDxAA4k5y5hY6K6+KRZDKlduf9w== 2024-05-24T11:30:52.246 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQBrelBmG1wCNBAAS8eCSTlu6ow2IZZsSWs9xA== 2024-05-24T11:30:52.867 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQBselBmkWaiHRAABgP3A8CW906UJX94XTIhrQ== 2024-05-24T11:30:52.868 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial monmap... 2024-05-24T11:30:53.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-24T11:30:53.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-24T11:30:53.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:30:53.577 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-24T11:30:53.577 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool for a [v2:172.21.15.39:3300,v1:172.21.15.39:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-24T11:30:53.578 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = pacific 2024-05-24T11:30:53.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: set fsid to f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:30:53.578 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-24T11:30:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:30:53.578 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mon... 2024-05-24T11:30:54.193 INFO:teuthology.orchestra.run.smithi039.stdout:create mon.a on 2024-05-24T11:30:54.328 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-24T11:30:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-24T11:30:54.552 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968.target. 2024-05-24T11:30:54.552 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968.target. 2024-05-24T11:30:54.734 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.a 2024-05-24T11:30:54.735 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-24T11:30:54.831 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968.target.wants/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@.service. 2024-05-24T11:30:55.194 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-24T11:30:55.194 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-05-24T11:30:55.202 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-24T11:30:55.202 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-24T11:30:55.202 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-05-24T11:30:55.202 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-05-24T11:30:55.202 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon to start... 2024-05-24T11:30:55.203 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon... 2024-05-24T11:30:55.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:55 smithi039 ceph-mon[42435]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-24T11:30:56.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout cluster: 2024-05-24T11:30:56.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout id: f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:30:56.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-24T11:30:56.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:30:56.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout services: 2024-05-24T11:30:56.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.468464s) 2024-05-24T11:30:56.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout data: 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pgs: 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.591+0000 7fc36b21c700 1 Processor -- start 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.591+0000 7fc36b21c700 1 -- start start 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.591+0000 7fc36b21c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364109580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:30:56.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.591+0000 7fc36b21c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc364079de0 con 0x7fc364107190 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.592+0000 7fc368fb8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.592+0000 7fc368fb8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364109580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60020/0 (socket says 172.21.15.39:60020) 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.592+0000 7fc368fb8700 1 -- 172.21.15.39:0/812682798 learned_addr learned my addr 172.21.15.39:0/812682798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.592+0000 7fc368fb8700 1 -- 172.21.15.39:0/812682798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364079f20 con 0x7fc364107190 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.592+0000 7fc368fb8700 1 --2- 172.21.15.39:0/812682798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364109580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3580098a0 tx=0x7fc358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d406d55cc93f2b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.592+0000 7fc3637fe700 1 -- 172.21.15.39:0/812682798 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc358004160 con 0x7fc364107190 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.592+0000 7fc3637fe700 1 -- 172.21.15.39:0/812682798 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3580042c0 con 0x7fc364107190 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.592+0000 7fc3637fe700 1 -- 172.21.15.39:0/812682798 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3580045a0 con 0x7fc364107190 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.593+0000 7fc36b21c700 1 -- 172.21.15.39:0/812682798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 msgr2=0x7fc364109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.593+0000 7fc36b21c700 1 --2- 172.21.15.39:0/812682798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364109580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3580098a0 tx=0x7fc358009bb0 comp rx=0 tx=0).stop 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.593+0000 7fc36b21c700 1 -- 172.21.15.39:0/812682798 shutdown_connections 2024-05-24T11:30:56.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.593+0000 7fc36b21c700 1 --2- 172.21.15.39:0/812682798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364109580 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.593+0000 7fc36b21c700 1 -- 172.21.15.39:0/812682798 >> 172.21.15.39:0/812682798 conn(0x7fc364100b00 msgr2=0x7fc364102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.593+0000 7fc36b21c700 1 -- 172.21.15.39:0/812682798 shutdown_connections 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.593+0000 7fc36b21c700 1 -- 172.21.15.39:0/812682798 wait complete. 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc36b21c700 1 Processor -- start 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc36b21c700 1 -- start start 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc36b21c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc36b21c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36419a0b0 con 0x7fc364107190 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc368fb8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc368fb8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60032/0 (socket says 172.21.15.39:60032) 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc368fb8700 1 -- 172.21.15.39:0/1341539927 learned_addr learned my addr 172.21.15.39:0/1341539927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc368fb8700 1 -- 172.21.15.39:0/1341539927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc358009550 con 0x7fc364107190 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc368fb8700 1 --2- 172.21.15.39:0/1341539927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364199b70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc364079220 tx=0x7fc358004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:30:56.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.594+0000 7fc361ffb700 1 -- 172.21.15.39:0/1341539927 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc358011630 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.595+0000 7fc361ffb700 1 -- 172.21.15.39:0/1341539927 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc358011790 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.595+0000 7fc36b21c700 1 -- 172.21.15.39:0/1341539927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36419a2b0 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.595+0000 7fc361ffb700 1 -- 172.21.15.39:0/1341539927 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc358011a30 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.595+0000 7fc36b21c700 1 -- 172.21.15.39:0/1341539927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36419a750 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.595+0000 7fc361ffb700 1 -- 172.21.15.39:0/1341539927 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fc358022070 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.595+0000 7fc361ffb700 1 -- 172.21.15.39:0/1341539927 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc35801c450 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.596+0000 7fc36b21c700 1 -- 172.21.15.39:0/1341539927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc364193940 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.598+0000 7fc361ffb700 1 -- 172.21.15.39:0/1341539927 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fc35801b450 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.671+0000 7fc36b21c700 1 -- 172.21.15.39:0/1341539927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fc364109ed0 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.671+0000 7fc361ffb700 1 -- 172.21.15.39:0/1341539927 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fc35801b5f0 con 0x7fc364107190 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.672+0000 7fc36b21c700 1 -- 172.21.15.39:0/1341539927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 msgr2=0x7fc364199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.672+0000 7fc36b21c700 1 --2- 172.21.15.39:0/1341539927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364199b70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc364079220 tx=0x7fc358004750 comp rx=0 tx=0).stop 2024-05-24T11:30:56.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.672+0000 7fc36b21c700 1 -- 172.21.15.39:0/1341539927 shutdown_connections 2024-05-24T11:30:56.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.672+0000 7fc36b21c700 1 --2- 172.21.15.39:0/1341539927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364107190 0x7fc364199b70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:30:56.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.672+0000 7fc36b21c700 1 -- 172.21.15.39:0/1341539927 >> 172.21.15.39:0/1341539927 conn(0x7fc364100b00 msgr2=0x7fc364101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:30:56.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.672+0000 7fc36b21c700 1 -- 172.21.15.39:0/1341539927 shutdown_connections 2024-05-24T11:30:56.333 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:55.672+0000 7fc36b21c700 1 -- 172.21.15.39:0/1341539927 wait complete. 2024-05-24T11:30:56.333 INFO:teuthology.orchestra.run.smithi039.stdout:mon is available 2024-05-24T11:30:56.333 INFO:teuthology.orchestra.run.smithi039.stdout:Assimilating anything we can from ceph.conf... 2024-05-24T11:30:56.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:56 smithi039 ceph-mon[42435]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-24T11:30:56.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:56 smithi039 ceph-mon[42435]: monmap e1: 1 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:30:56.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:56 smithi039 ceph-mon[42435]: fsmap 2024-05-24T11:30:56.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:56 smithi039 ceph-mon[42435]: osdmap e1: 0 total, 0 up, 0 in 2024-05-24T11:30:56.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:56 smithi039 ceph-mon[42435]: mgrmap e1: no daemons active 2024-05-24T11:30:56.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:56 smithi039 ceph-mon[42435]: from='client.? 172.21.15.39:0/1341539927' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-24T11:30:57.114 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-24T11:30:57.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.688+0000 7fef40ae7700 1 Processor -- start 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.688+0000 7fef40ae7700 1 -- start start 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.688+0000 7fef40ae7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.688+0000 7fef40ae7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c107030 con 0x7fef3c106640 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.688+0000 7fef3a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.688+0000 7fef3a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60034/0 (socket says 172.21.15.39:60034) 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.688+0000 7fef3a59c700 1 -- 172.21.15.39:0/3754182226 learned_addr learned my addr 172.21.15.39:0/3754182226 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:30:57.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.689+0000 7fef3a59c700 1 -- 172.21.15.39:0/3754182226 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c107840 con 0x7fef3c106640 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.689+0000 7fef3a59c700 1 --2- 172.21.15.39:0/3754182226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c106a60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0098a0 tx=0x7fef2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818be2a8876e0ea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.689+0000 7fef3959a700 1 -- 172.21.15.39:0/3754182226 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef2c004160 con 0x7fef3c106640 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.689+0000 7fef3959a700 1 -- 172.21.15.39:0/3754182226 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c0042c0 con 0x7fef3c106640 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.689+0000 7fef3959a700 1 -- 172.21.15.39:0/3754182226 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef2c0045a0 con 0x7fef3c106640 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.689+0000 7fef40ae7700 1 -- 172.21.15.39:0/3754182226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 msgr2=0x7fef3c106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.689+0000 7fef40ae7700 1 --2- 172.21.15.39:0/3754182226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c106a60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0098a0 tx=0x7fef2c009bb0 comp rx=0 tx=0).stop 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.690+0000 7fef40ae7700 1 -- 172.21.15.39:0/3754182226 shutdown_connections 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.690+0000 7fef40ae7700 1 --2- 172.21.15.39:0/3754182226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c106a60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.690+0000 7fef40ae7700 1 -- 172.21.15.39:0/3754182226 >> 172.21.15.39:0/3754182226 conn(0x7fef3c101c00 msgr2=0x7fef3c104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.690+0000 7fef40ae7700 1 -- 172.21.15.39:0/3754182226 shutdown_connections 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.690+0000 7fef40ae7700 1 -- 172.21.15.39:0/3754182226 wait complete. 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.690+0000 7fef40ae7700 1 Processor -- start 2024-05-24T11:30:57.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.690+0000 7fef40ae7700 1 -- start start 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef40ae7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef40ae7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c19a090 con 0x7fef3c106640 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef3a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef3a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60038/0 (socket says 172.21.15.39:60038) 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef3a59c700 1 -- 172.21.15.39:0/785789810 learned_addr learned my addr 172.21.15.39:0/785789810 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef3a59c700 1 -- 172.21.15.39:0/785789810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c009550 con 0x7fef3c106640 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef3a59c700 1 --2- 172.21.15.39:0/785789810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c199b50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fef2c004000 tx=0x7fef2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef2b7fe700 1 -- 172.21.15.39:0/785789810 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef2c011630 con 0x7fef3c106640 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef2b7fe700 1 -- 172.21.15.39:0/785789810 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c011790 con 0x7fef3c106640 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef40ae7700 1 -- 172.21.15.39:0/785789810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c19a290 con 0x7fef3c106640 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef2b7fe700 1 -- 172.21.15.39:0/785789810 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef2c011a30 con 0x7fef3c106640 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef40ae7700 1 -- 172.21.15.39:0/785789810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c19a730 con 0x7fef3c106640 2024-05-24T11:30:57.123 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.691+0000 7fef2b7fe700 1 -- 172.21.15.39:0/785789810 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fef2c022070 con 0x7fef3c106640 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.692+0000 7fef2b7fe700 1 -- 172.21.15.39:0/785789810 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fef2c01c450 con 0x7fef3c106640 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.692+0000 7fef40ae7700 1 -- 172.21.15.39:0/785789810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3c193900 con 0x7fef3c106640 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.694+0000 7fef2b7fe700 1 -- 172.21.15.39:0/785789810 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fef2c01b450 con 0x7fef3c106640 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.765+0000 7fef40ae7700 1 -- 172.21.15.39:0/785789810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fef3c107de0 con 0x7fef3c106640 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.771+0000 7fef2b7fe700 1 -- 172.21.15.39:0/785789810 <== mon.0 v2:172.21.15.39:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fef2c01b5f0 con 0x7fef3c106640 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.771+0000 7fef2b7fe700 1 -- 172.21.15.39:0/785789810 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fef2c037540 con 0x7fef3c106640 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.772+0000 7fef40ae7700 1 -- 172.21.15.39:0/785789810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 msgr2=0x7fef3c199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.772+0000 7fef40ae7700 1 --2- 172.21.15.39:0/785789810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c199b50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fef2c004000 tx=0x7fef2c004750 comp rx=0 tx=0).stop 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.772+0000 7fef40ae7700 1 -- 172.21.15.39:0/785789810 shutdown_connections 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.772+0000 7fef40ae7700 1 --2- 172.21.15.39:0/785789810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef3c106640 0x7fef3c199b50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.772+0000 7fef40ae7700 1 -- 172.21.15.39:0/785789810 >> 172.21.15.39:0/785789810 conn(0x7fef3c101c00 msgr2=0x7fef3c190a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.772+0000 7fef40ae7700 1 -- 172.21.15.39:0/785789810 shutdown_connections 2024-05-24T11:30:57.124 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:56.772+0000 7fef40ae7700 1 -- 172.21.15.39:0/785789810 wait complete. 2024-05-24T11:30:57.125 INFO:teuthology.orchestra.run.smithi039.stdout:Generating new minimal ceph.conf... 2024-05-24T11:30:57.370 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:57 smithi039 ceph-mon[42435]: from='client.? 172.21.15.39:0/785789810' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-24T11:30:57.370 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:57 smithi039 ceph-mon[42435]: from='client.? 172.21.15.39:0/785789810' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-24T11:30:58.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.632+0000 7f5c1defc700 1 Processor -- start 2024-05-24T11:30:58.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.632+0000 7f5c1defc700 1 -- start start 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.632+0000 7f5c1defc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c181094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.632+0000 7f5c1defc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c180622c0 con 0x7f5c181070c0 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.632+0000 7f5c177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c181094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.632+0000 7f5c177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c181094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49286/0 (socket says 172.21.15.39:49286) 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.632+0000 7f5c177fe700 1 -- 172.21.15.39:0/2192094963 learned_addr learned my addr 172.21.15.39:0/2192094963 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.632+0000 7f5c177fe700 1 -- 172.21.15.39:0/2192094963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c181099f0 con 0x7f5c181070c0 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.633+0000 7f5c177fe700 1 --2- 172.21.15.39:0/2192094963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c181094b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c080098a0 tx=0x7f5c08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88099d463e449f51 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.633+0000 7f5c167fc700 1 -- 172.21.15.39:0/2192094963 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c08004160 con 0x7f5c181070c0 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.633+0000 7f5c167fc700 1 -- 172.21.15.39:0/2192094963 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5c080042c0 con 0x7f5c181070c0 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.633+0000 7f5c167fc700 1 -- 172.21.15.39:0/2192094963 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c08011940 con 0x7f5c181070c0 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.633+0000 7f5c1defc700 1 -- 172.21.15.39:0/2192094963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 msgr2=0x7f5c181094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.633+0000 7f5c1defc700 1 --2- 172.21.15.39:0/2192094963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c181094b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c080098a0 tx=0x7f5c08009bb0 comp rx=0 tx=0).stop 2024-05-24T11:30:58.101 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.633+0000 7f5c1defc700 1 -- 172.21.15.39:0/2192094963 shutdown_connections 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.633+0000 7f5c1defc700 1 --2- 172.21.15.39:0/2192094963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c181094b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.633+0000 7f5c1defc700 1 -- 172.21.15.39:0/2192094963 >> 172.21.15.39:0/2192094963 conn(0x7f5c18100b10 msgr2=0x7f5c18102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.634+0000 7f5c1defc700 1 -- 172.21.15.39:0/2192094963 shutdown_connections 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.634+0000 7f5c1defc700 1 -- 172.21.15.39:0/2192094963 wait complete. 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.634+0000 7f5c1defc700 1 Processor -- start 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.634+0000 7f5c1defc700 1 -- start start 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.634+0000 7f5c1defc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c1819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.634+0000 7f5c1defc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1819a810 con 0x7f5c181070c0 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c1819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c1819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49294/0 (socket says 172.21.15.39:49294) 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c177fe700 1 -- 172.21.15.39:0/475130620 learned_addr learned my addr 172.21.15.39:0/475130620 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c177fe700 1 -- 172.21.15.39:0/475130620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c08009550 con 0x7f5c181070c0 2024-05-24T11:30:58.102 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c177fe700 1 --2- 172.21.15.39:0/475130620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c1819a2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c08006af0 tx=0x7f5c08011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c14ff9700 1 -- 172.21.15.39:0/475130620 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c08003bf0 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c1defc700 1 -- 172.21.15.39:0/475130620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1819aa10 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c14ff9700 1 -- 172.21.15.39:0/475130620 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5c0801b8b0 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c14ff9700 1 -- 172.21.15.39:0/475130620 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c08023830 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c1defc700 1 -- 172.21.15.39:0/475130620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1819aeb0 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.635+0000 7f5c14ff9700 1 -- 172.21.15.39:0/475130620 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5c0802c3f0 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.636+0000 7f5c14ff9700 1 -- 172.21.15.39:0/475130620 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5c08019710 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.636+0000 7f5c1defc700 1 -- 172.21.15.39:0/475130620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1810a790 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.639+0000 7f5c14ff9700 1 -- 172.21.15.39:0/475130620 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f5c0801ba20 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.708+0000 7f5c1defc700 1 -- 172.21.15.39:0/475130620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5c1810a950 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.708+0000 7f5c14ff9700 1 -- 172.21.15.39:0/475130620 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f5c0801bd70 con 0x7f5c181070c0 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.709+0000 7f5c1defc700 1 -- 172.21.15.39:0/475130620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 msgr2=0x7f5c1819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:30:58.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.709+0000 7f5c1defc700 1 --2- 172.21.15.39:0/475130620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c1819a2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c08006af0 tx=0x7f5c08011a50 comp rx=0 tx=0).stop 2024-05-24T11:30:58.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.709+0000 7f5c1defc700 1 -- 172.21.15.39:0/475130620 shutdown_connections 2024-05-24T11:30:58.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.709+0000 7f5c1defc700 1 --2- 172.21.15.39:0/475130620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c181070c0 0x7f5c1819a2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:30:58.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.709+0000 7f5c1defc700 1 -- 172.21.15.39:0/475130620 >> 172.21.15.39:0/475130620 conn(0x7f5c18100b10 msgr2=0x7f5c181016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:30:58.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.709+0000 7f5c1defc700 1 -- 172.21.15.39:0/475130620 shutdown_connections 2024-05-24T11:30:58.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:57.709+0000 7f5c1defc700 1 -- 172.21.15.39:0/475130620 wait complete. 2024-05-24T11:30:58.104 INFO:teuthology.orchestra.run.smithi039.stdout:Restarting the monitor... 2024-05-24T11:30:58.301 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:58 smithi039 systemd[1]: Stopping Ceph mon.a for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:30:58.302 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:58 smithi039 ceph-mon[42435]: from='client.? 172.21.15.39:0/475130620' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:30:58.302 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:58 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-a[42411]: 2024-05-24T11:30:58.298+0000 7fbf07af2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:30:58.561 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:58 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-a[42411]: 2024-05-24T11:30:58.298+0000 7fbf07af2700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-24T11:30:58.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:58 smithi039 bash[42761]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-a 2024-05-24T11:30:59.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:58 smithi039 systemd[1]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.a.service: Succeeded. 2024-05-24T11:30:59.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:58 smithi039 systemd[1]: Stopped Ceph mon.a for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:30:59.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:58 smithi039 systemd[1]: Starting Ceph mon.a for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:30:59.207 INFO:teuthology.orchestra.run.smithi039.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-24T11:30:59.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: set uid:gid to 167:167 (ceph:ceph) 2024-05-24T11:30:59.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: ceph version 18.2.2-1349-gb9b067bc (b9b067bc6d30ec00e391f0fa43f50affe981ba2f) reef (stable), process ceph-mon, pid 7 2024-05-24T11:30:59.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: pidfile_write: ignore empty --pid-file 2024-05-24T11:30:59.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 bash[42882]: 861f9e130e3c1997369e7ea61f03fd710fee9348ccb6fe0e8c8cb59e0c28eae3 2024-05-24T11:30:59.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: load: jerasure load: lrc 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: RocksDB version: 7.9.2 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Git sha 0 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Compile date 2024-05-19 15:03:31 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: DB SUMMARY 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: DB Session ID: VIKLMFOGCVT3ZQA9KO5F 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: CURRENT file: CURRENT 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: IDENTITY file: IDENTITY 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99085 ; 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.error_if_exists: 0 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.create_if_missing: 0 2024-05-24T11:30:59.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.paranoid_checks: 1 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.env: 0x558a47406820 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.fs: PosixFileSystem 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.info_log: 0x558a4a95f540 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_file_opening_threads: 16 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.statistics: (nil) 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.use_fsync: 0 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_log_file_size: 0 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-24T11:30:59.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.keep_log_file_num: 1000 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.recycle_log_file_num: 0 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.allow_fallocate: 1 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.allow_mmap_reads: 0 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.allow_mmap_writes: 0 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.use_direct_reads: 0 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.create_missing_column_families: 0 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.db_log_dir: 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.wal_dir: 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-24T11:30:59.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.advise_random_on_open: 1 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.db_write_buffer_size: 0 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.write_buffer_manager: 0x558a49bf4320 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.rate_limiter: (nil) 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.wal_recovery_mode: 2 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.enable_thread_tracking: 0 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.enable_pipelined_write: 0 2024-05-24T11:30:59.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.unordered_write: 0 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.row_cache: None 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.wal_filter: None 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.allow_ingest_behind: 0 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.two_write_queues: 0 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.manual_wal_flush: 0 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.wal_compression: 0 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.atomic_flush: 0 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-24T11:30:59.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-24T11:30:59.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.log_readahead_size: 0 2024-05-24T11:30:59.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-24T11:30:59.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.best_efforts_recovery: 0 2024-05-24T11:30:59.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-24T11:30:59.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-24T11:30:59.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.allow_data_in_errors: 0 2024-05-24T11:30:59.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.db_host_id: __hostname__ 2024-05-24T11:30:59.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-24T11:30:59.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_background_jobs: 2 2024-05-24T11:30:59.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_background_compactions: -1 2024-05-24T11:30:59.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_subcompactions: 1 2024-05-24T11:30:59.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_total_wal_size: 0 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_open_files: -1 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bytes_per_sync: 0 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_readahead_size: 0 2024-05-24T11:30:59.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_background_flushes: -1 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Compression algorithms supported: 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: kZSTD supported: 0 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: kXpressCompression supported: 0 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: kLZ4HCCompression supported: 1 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: kZlibCompression supported: 1 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: kSnappyCompression supported: 1 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: kLZ4Compression supported: 1 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: kBZip2Compression supported: 0 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-24T11:30:59.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.merge_operator: 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_filter: None 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_filter_factory: None 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.sst_partitioner_factory: None 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558a4a95f640) 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks: 1 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_top_level_index_and_filter: 1 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_type: 0 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_index_type: 0 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_shortening: 1 2024-05-24T11:30:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: checksum: 4 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: no_block_cache: 0 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache: 0x558a49c771f0 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_name: BinnedLRUCache 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_options: 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: capacity : 536870912 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_shard_bits : 4 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: strict_capacity_limit : 0 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: high_pri_pool_ratio: 0.000 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_compressed: (nil) 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: persistent_cache: (nil) 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size: 4096 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size_deviation: 10 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_restart_interval: 16 2024-05-24T11:30:59.587 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_block_restart_interval: 1 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: metadata_block_size: 4096 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: partition_filters: 0 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: use_delta_encoding: 1 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: filter_policy: bloomfilter 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: whole_key_filtering: 1 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: verify_compression: 0 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: read_amp_bytes_per_bit: 0 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: format_version: 5 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_index_compression: 1 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_align: 0 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: max_auto_readahead_size: 262144 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: prepopulate_block_cache: 0 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: initial_auto_readahead_size: 8192 2024-05-24T11:30:59.588 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_file_reads_for_auto_readahead: 2 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.write_buffer_size: 33554432 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_write_buffer_number: 2 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression: NoCompression 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression: Disabled 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.prefix_extractor: nullptr 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.num_levels: 7 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-24T11:30:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression_opts.level: 32767 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression_opts.strategy: 0 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression_opts.enabled: false 2024-05-24T11:30:59.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.target_file_size_base: 67108864 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-24T11:30:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-24T11:30:59.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-24T11:30:59.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-24T11:30:59.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-24T11:30:59.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-24T11:30:59.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-24T11:30:59.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-24T11:30:59.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-24T11:30:59.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.arena_block_size: 1048576 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.disable_auto_compactions: 0 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-24T11:30:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.table_properties_collectors: 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.inplace_update_support: 0 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.bloom_locality: 0 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.max_successive_merges: 0 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.paranoid_file_checks: 0 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.force_consistency_checks: 1 2024-05-24T11:30:59.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.report_bg_io_stats: 0 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.ttl: 2592000 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.enable_blob_files: false 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.min_blob_size: 0 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.blob_file_size: 268435456 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-24T11:30:59.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.blob_file_starting_level: 0 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3100dae6-b60f-44ea-bfa4-6d4ffb8cda99 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716550259203835, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 systemd[1]: Started Ceph mon.a for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716550259205084, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94305, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92303, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13723, "raw_average_key_size": 49, "raw_value_size": 84823, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716550259, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3100dae6-b60f-44ea-bfa4-6d4ffb8cda99", "db_session_id": "VIKLMFOGCVT3ZQA9KO5F", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716550259205160, "job": 1, "event": "recovery_finished"} 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-24T11:30:59.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558a49d14000 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: DB pointer 0x558a49d02000 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** DB Stats ** 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-24T11:30:59.597 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: L0 2/0 93.93 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 111.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: Sum 2/0 93.93 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 111.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 111.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 111.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-24T11:30:59.598 INFO:journalctl@ceph.mon.a.smithi039.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative compaction: 0.00 GB write, 21.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval compaction: 0.00 GB write, 21.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache BinnedLRUCache@0x558a49c771f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-24T11:30:59.599 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: starting mon.a rank 0 at public addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] at bind addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mon.a@-1(???) e1 preinit fsid f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mon.a@-1(???).mds e1 new map 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mon.a@-1(???).mds e1 print_map 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout: e1 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout: legacy client fscid: -1 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout: No filesystems configured 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-24T11:30:59.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-24T11:30:59.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-24T11:30:59.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-24T11:30:59.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: monmap e1: 1 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:30:59.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: fsmap 2024-05-24T11:30:59.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: osdmap e1: 0 total, 0 up, 0 in 2024-05-24T11:30:59.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:30:59 smithi039 ceph-mon[42919]: mgrmap e1: no daemons active 2024-05-24T11:30:59.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.544+0000 7fc2818f0700 1 Processor -- start 2024-05-24T11:30:59.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.544+0000 7fc2818f0700 1 -- start start 2024-05-24T11:30:59.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.544+0000 7fc2818f0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c105980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:30:59.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.544+0000 7fc2818f0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c105f50 con 0x7fc27c105560 2024-05-24T11:30:59.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.545+0000 7fc27affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c105980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:30:59.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.545+0000 7fc27affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c105980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49302/0 (socket says 172.21.15.39:49302) 2024-05-24T11:30:59.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.545+0000 7fc27affd700 1 -- 172.21.15.39:0/2659359664 learned_addr learned my addr 172.21.15.39:0/2659359664 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:30:59.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.545+0000 7fc27affd700 1 -- 172.21.15.39:0/2659359664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c106760 con 0x7fc27c105560 2024-05-24T11:30:59.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.545+0000 7fc27affd700 1 --2- 172.21.15.39:0/2659359664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c105980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0098a0 tx=0x7fc26c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00fa85927af9aa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.545+0000 7fc279ffb700 1 -- 172.21.15.39:0/2659359664 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc26c004160 con 0x7fc27c105560 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.545+0000 7fc279ffb700 1 -- 172.21.15.39:0/2659359664 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc26c0042c0 con 0x7fc27c105560 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.545+0000 7fc279ffb700 1 -- 172.21.15.39:0/2659359664 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc26c011940 con 0x7fc27c105560 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.546+0000 7fc2818f0700 1 -- 172.21.15.39:0/2659359664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 msgr2=0x7fc27c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.546+0000 7fc2818f0700 1 --2- 172.21.15.39:0/2659359664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c105980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0098a0 tx=0x7fc26c009bb0 comp rx=0 tx=0).stop 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.546+0000 7fc2818f0700 1 -- 172.21.15.39:0/2659359664 shutdown_connections 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.546+0000 7fc2818f0700 1 --2- 172.21.15.39:0/2659359664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c105980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.546+0000 7fc2818f0700 1 -- 172.21.15.39:0/2659359664 >> 172.21.15.39:0/2659359664 conn(0x7fc27c100b20 msgr2=0x7fc27c102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.546+0000 7fc2818f0700 1 -- 172.21.15.39:0/2659359664 shutdown_connections 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.546+0000 7fc2818f0700 1 -- 172.21.15.39:0/2659359664 wait complete. 2024-05-24T11:30:59.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.547+0000 7fc2818f0700 1 Processor -- start 2024-05-24T11:30:59.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.547+0000 7fc2818f0700 1 -- start start 2024-05-24T11:30:59.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.547+0000 7fc2818f0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:30:59.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.547+0000 7fc2818f0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c19a890 con 0x7fc27c105560 2024-05-24T11:30:59.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.547+0000 7fc27affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:30:59.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.547+0000 7fc27affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49310/0 (socket says 172.21.15.39:49310) 2024-05-24T11:30:59.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.547+0000 7fc27affd700 1 -- 172.21.15.39:0/462918831 learned_addr learned my addr 172.21.15.39:0/462918831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.548+0000 7fc27affd700 1 -- 172.21.15.39:0/462918831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c009550 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.548+0000 7fc27affd700 1 --2- 172.21.15.39:0/462918831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c19a350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc26c003f20 tx=0x7fc26c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.548+0000 7fc2808ee700 1 -- 172.21.15.39:0/462918831 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc26c0036a0 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.548+0000 7fc2808ee700 1 -- 172.21.15.39:0/462918831 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc26c01b440 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.548+0000 7fc2818f0700 1 -- 172.21.15.39:0/462918831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27c19aa90 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.548+0000 7fc2808ee700 1 -- 172.21.15.39:0/462918831 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc26c023400 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.548+0000 7fc2818f0700 1 -- 172.21.15.39:0/462918831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c19af30 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.548+0000 7fc2808ee700 1 -- 172.21.15.39:0/462918831 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fc26c023970 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.548+0000 7fc2808ee700 1 -- 172.21.15.39:0/462918831 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc26c02cdf0 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.549+0000 7fc2818f0700 1 -- 172.21.15.39:0/462918831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27c194510 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.551+0000 7fc2808ee700 1 -- 172.21.15.39:0/462918831 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fc26c023d60 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.623+0000 7fc2818f0700 1 -- 172.21.15.39:0/462918831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fc27c0625d0 con 0x7fc27c105560 2024-05-24T11:30:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.625+0000 7fc2808ee700 1 -- 172.21.15.39:0/462918831 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc26c006d60 con 0x7fc27c105560 2024-05-24T11:30:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.626+0000 7fc2818f0700 1 -- 172.21.15.39:0/462918831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 msgr2=0x7fc27c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:30:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.626+0000 7fc2818f0700 1 --2- 172.21.15.39:0/462918831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c19a350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc26c003f20 tx=0x7fc26c011f00 comp rx=0 tx=0).stop 2024-05-24T11:30:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.626+0000 7fc2818f0700 1 -- 172.21.15.39:0/462918831 shutdown_connections 2024-05-24T11:30:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.626+0000 7fc2818f0700 1 --2- 172.21.15.39:0/462918831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc27c105560 0x7fc27c19a350 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:30:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.626+0000 7fc2818f0700 1 -- 172.21.15.39:0/462918831 >> 172.21.15.39:0/462918831 conn(0x7fc27c100b20 msgr2=0x7fc27c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:30:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.626+0000 7fc2818f0700 1 -- 172.21.15.39:0/462918831 shutdown_connections 2024-05-24T11:30:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:30:59.626+0000 7fc2818f0700 1 -- 172.21.15.39:0/462918831 wait complete. 2024-05-24T11:30:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-24T11:31:00.020 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-24T11:31:00.020 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mgr... 2024-05-24T11:31:00.020 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-24T11:31:00.020 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-24T11:31:00.237 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.x 2024-05-24T11:31:00.237 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-24T11:31:00.337 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968.target.wants/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@.service. 2024-05-24T11:31:00.575 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:00 smithi039 systemd[1]: Starting Ceph mgr.x for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:31:00.767 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-24T11:31:00.767 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-05-24T11:31:00.775 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-24T11:31:00.775 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-24T11:31:00.775 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-05-24T11:31:00.775 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-05-24T11:31:00.784 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-24T11:31:00.784 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-05-24T11:31:00.792 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-24T11:31:00.792 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-24T11:31:00.792 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-05-24T11:31:00.792 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-24T11:31:00.792 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr to start... 2024-05-24T11:31:00.792 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr... 2024-05-24T11:31:00.922 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:00 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/462918831' entity='client.admin' 2024-05-24T11:31:00.922 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:00 smithi039 bash[43154]: e7663b17d7988afa76b15168475365d2ebce628fdb31ce5bab99eac6fd053b4b 2024-05-24T11:31:00.922 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:00 smithi039 systemd[1]: Started Ceph mgr.x for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:31:01.428 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:01.172+0000 7fb8374bc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-24T11:31:01.429 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:01.272+0000 7fb8374bc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-24T11:31:01.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:01 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3064689616' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-24T11:31:01.825 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:01.365+0000 7fb8374bc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-24T11:31:01.825 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:01.453+0000 7fb8374bc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-24T11:31:02.398 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:02.133+0000 7fb8374bc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:31:02.768 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:02.395+0000 7fb8374bc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-24T11:31:02.769 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:02.485+0000 7fb8374bc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-24T11:31:02.769 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:02.601+0000 7fb8374bc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-24T11:31:03.075 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:02.766+0000 7fb8374bc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-24T11:31:03.075 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:02.899+0000 7fb8374bc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-24T11:31:03.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:02.988+0000 7fb8374bc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-24T11:31:03.305 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:31:03.305 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-24T11:31:03.305 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-24T11:31:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-24T11:31:03.308 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-05-24T11:30:55.204819+0000", 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-24T11:31:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.193+0000 7feb2e0b2700 1 Processor -- start 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.193+0000 7feb2e0b2700 1 -- start start 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.193+0000 7feb2e0b2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb28106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.193+0000 7feb2e0b2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb28107030 con 0x7feb28106640 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.193+0000 7feb277fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb28106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.194+0000 7feb277fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb28106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49342/0 (socket says 172.21.15.39:49342) 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.194+0000 7feb277fe700 1 -- 172.21.15.39:0/3029893068 learned_addr learned my addr 172.21.15.39:0/3029893068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.194+0000 7feb277fe700 1 -- 172.21.15.39:0/3029893068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb28107840 con 0x7feb28106640 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.194+0000 7feb277fe700 1 --2- 172.21.15.39:0/3029893068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb28106a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb180098a0 tx=0x7feb18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5980317c788f466 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.194+0000 7feb267fc700 1 -- 172.21.15.39:0/3029893068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb18004160 con 0x7feb28106640 2024-05-24T11:31:03.310 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.194+0000 7feb267fc700 1 -- 172.21.15.39:0/3029893068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7feb180042c0 con 0x7feb28106640 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.194+0000 7feb267fc700 1 -- 172.21.15.39:0/3029893068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb18011940 con 0x7feb28106640 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.195+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3029893068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 msgr2=0x7feb28106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.195+0000 7feb2e0b2700 1 --2- 172.21.15.39:0/3029893068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb28106a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb180098a0 tx=0x7feb18009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.195+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3029893068 shutdown_connections 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.195+0000 7feb2e0b2700 1 --2- 172.21.15.39:0/3029893068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb28106a60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.195+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3029893068 >> 172.21.15.39:0/3029893068 conn(0x7feb28101c00 msgr2=0x7feb28104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.195+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3029893068 shutdown_connections 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.195+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3029893068 wait complete. 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.196+0000 7feb2e0b2700 1 Processor -- start 2024-05-24T11:31:03.311 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.196+0000 7feb2e0b2700 1 -- start start 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.196+0000 7feb2e0b2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb2819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.196+0000 7feb2e0b2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2819a7e0 con 0x7feb28106640 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.196+0000 7feb277fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb2819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.196+0000 7feb277fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb2819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49348/0 (socket says 172.21.15.39:49348) 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.196+0000 7feb277fe700 1 -- 172.21.15.39:0/3064689616 learned_addr learned my addr 172.21.15.39:0/3064689616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.197+0000 7feb277fe700 1 -- 172.21.15.39:0/3064689616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb18009550 con 0x7feb28106640 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.197+0000 7feb277fe700 1 --2- 172.21.15.39:0/3064689616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb2819a2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb18003f20 tx=0x7feb18011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.197+0000 7feb24ff9700 1 -- 172.21.15.39:0/3064689616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb180036a0 con 0x7feb28106640 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.197+0000 7feb24ff9700 1 -- 172.21.15.39:0/3064689616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7feb1801b440 con 0x7feb28106640 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.197+0000 7feb24ff9700 1 -- 172.21.15.39:0/3064689616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb18023400 con 0x7feb28106640 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.197+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3064689616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2819a9e0 con 0x7feb28106640 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.197+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3064689616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2819ae80 con 0x7feb28106640 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.197+0000 7feb24ff9700 1 -- 172.21.15.39:0/3064689616 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7feb18023970 con 0x7feb28106640 2024-05-24T11:31:03.312 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.197+0000 7feb24ff9700 1 -- 172.21.15.39:0/3064689616 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feb1802cdf0 con 0x7feb28106640 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.198+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3064689616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb280625d0 con 0x7feb28106640 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.200+0000 7feb24ff9700 1 -- 172.21.15.39:0/3064689616 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7feb18023d60 con 0x7feb28106640 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.274+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3064689616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7feb2810aa70 con 0x7feb28106640 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.275+0000 7feb24ff9700 1 -- 172.21.15.39:0/3064689616 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7feb1801b5b0 con 0x7feb28106640 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.276+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3064689616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 msgr2=0x7feb2819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.276+0000 7feb2e0b2700 1 --2- 172.21.15.39:0/3064689616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb2819a2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb18003f20 tx=0x7feb18011f00 comp rx=0 tx=0).stop 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.276+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3064689616 shutdown_connections 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.276+0000 7feb2e0b2700 1 --2- 172.21.15.39:0/3064689616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb28106640 0x7feb2819a2a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.276+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3064689616 >> 172.21.15.39:0/3064689616 conn(0x7feb28101c00 msgr2=0x7feb28102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.276+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3064689616 shutdown_connections 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:01.276+0000 7feb2e0b2700 1 -- 172.21.15.39:0/3064689616 wait complete. 2024-05-24T11:31:03.313 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (1/15)... 2024-05-24T11:31:03.575 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:03 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:03.379+0000 7fb8374bc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-24T11:31:04.075 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:03 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:03.784+0000 7fb8374bc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-24T11:31:04.075 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:03 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:03.986+0000 7fb8374bc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-24T11:31:04.575 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:04 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:04.084+0000 7fb8374bc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-24T11:31:05.147 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:04 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:04.888+0000 7fb8374bc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-24T11:31:05.147 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:04 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:04.989+0000 7fb8374bc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-24T11:31:05.575 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:05.145+0000 7fb8374bc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-24T11:31:06.075 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:05.807+0000 7fb8374bc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-24T11:31:06.075 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:05.997+0000 7fb8374bc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-24T11:31:06.516 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:06 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:06.202+0000 7fb8374bc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-24T11:31:07.488 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:07 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1756974672' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-24T11:31:07.489 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:07.185+0000 7fb8374bc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-24T11:31:07.489 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:07.486+0000 7fb8374bc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:31:07.780 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:07.580+0000 7fb8374bc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-24T11:31:08.075 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:07.778+0000 7fb8374bc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-24T11:31:08.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: Activating manager daemon x 2024-05-24T11:31:08.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: mgrmap e2: x(active, starting, since 0.00519047s) 2024-05-24T11:31:08.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-24T11:31:08.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-24T11:31:08.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-24T11:31:08.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:31:08.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-24T11:31:08.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: Manager daemon x is now available 2024-05-24T11:31:08.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-24T11:31:08.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' 2024-05-24T11:31:08.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-24T11:31:08.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' 2024-05-24T11:31:08.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:08 smithi039 ceph-mon[42919]: from='mgr.14100 172.21.15.39:0/1889402501' entity='mgr.x' 2024-05-24T11:31:08.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:31:08.998 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-24T11:31:08.998 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:31:08.998 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-05-24T11:31:08.998 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-24T11:31:08.998 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-24T11:31:08.998 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-24T11:31:08.999 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-24T11:31:09.000 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-24T11:31:09.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-24T11:31:09.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-05-24T11:30:55.204819+0000", 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.206+0000 7fa5a7c42700 1 Processor -- start 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.207+0000 7fa5a7c42700 1 -- start start 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.207+0000 7fa5a7c42700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.207+0000 7fa5a7c42700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0105f40 con 0x7fa5a0105550 2024-05-24T11:31:09.003 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.207+0000 7fa5a59de700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.207+0000 7fa5a59de700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49362/0 (socket says 172.21.15.39:49362) 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.207+0000 7fa5a59de700 1 -- 172.21.15.39:0/4245657526 learned_addr learned my addr 172.21.15.39:0/4245657526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.207+0000 7fa5a59de700 1 -- 172.21.15.39:0/4245657526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a0106750 con 0x7fa5a0105550 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.207+0000 7fa5a59de700 1 --2- 172.21.15.39:0/4245657526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0105970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0098a0 tx=0x7fa59c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c013828183bad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a49dc700 1 -- 172.21.15.39:0/4245657526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa59c004160 con 0x7fa5a0105550 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a49dc700 1 -- 172.21.15.39:0/4245657526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa59c0042c0 con 0x7fa5a0105550 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a49dc700 1 -- 172.21.15.39:0/4245657526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa59c011940 con 0x7fa5a0105550 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a7c42700 1 -- 172.21.15.39:0/4245657526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 msgr2=0x7fa5a0105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a7c42700 1 --2- 172.21.15.39:0/4245657526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0105970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0098a0 tx=0x7fa59c009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a7c42700 1 -- 172.21.15.39:0/4245657526 shutdown_connections 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a7c42700 1 --2- 172.21.15.39:0/4245657526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0105970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:09.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a7c42700 1 -- 172.21.15.39:0/4245657526 >> 172.21.15.39:0/4245657526 conn(0x7fa5a0100b10 msgr2=0x7fa5a0102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a7c42700 1 -- 172.21.15.39:0/4245657526 shutdown_connections 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.208+0000 7fa5a7c42700 1 -- 172.21.15.39:0/4245657526 wait complete. 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.209+0000 7fa5a7c42700 1 Processor -- start 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.209+0000 7fa5a7c42700 1 -- start start 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.209+0000 7fa5a7c42700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0061e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.209+0000 7fa5a7c42700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0062350 con 0x7fa5a0105550 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.209+0000 7fa5a59de700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0061e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.209+0000 7fa5a59de700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0061e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49366/0 (socket says 172.21.15.39:49366) 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.209+0000 7fa5a59de700 1 -- 172.21.15.39:0/1756974672 learned_addr learned my addr 172.21.15.39:0/1756974672 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.210+0000 7fa5a59de700 1 -- 172.21.15.39:0/1756974672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c009550 con 0x7fa5a0105550 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.210+0000 7fa5a59de700 1 --2- 172.21.15.39:0/1756974672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0061e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa59c003f20 tx=0x7fa59c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.210+0000 7fa596ffd700 1 -- 172.21.15.39:0/1756974672 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa59c0036a0 con 0x7fa5a0105550 2024-05-24T11:31:09.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.210+0000 7fa596ffd700 1 -- 172.21.15.39:0/1756974672 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa59c01b440 con 0x7fa5a0105550 2024-05-24T11:31:09.006 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.210+0000 7fa5a7c42700 1 -- 172.21.15.39:0/1756974672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a019d060 con 0x7fa5a0105550 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.210+0000 7fa596ffd700 1 -- 172.21.15.39:0/1756974672 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa59c023400 con 0x7fa5a0105550 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.210+0000 7fa5a7c42700 1 -- 172.21.15.39:0/1756974672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a019d4a0 con 0x7fa5a0105550 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.210+0000 7fa596ffd700 1 -- 172.21.15.39:0/1756974672 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa59c0239c0 con 0x7fa5a0105550 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.210+0000 7fa596ffd700 1 -- 172.21.15.39:0/1756974672 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa59c02ce40 con 0x7fa5a0105550 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.211+0000 7fa5a7c42700 1 -- 172.21.15.39:0/1756974672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a0109980 con 0x7fa5a0105550 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.213+0000 7fa596ffd700 1 -- 172.21.15.39:0/1756974672 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fa59c023d60 con 0x7fa5a0105550 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.288+0000 7fa5a7c42700 1 -- 172.21.15.39:0/1756974672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa5a019d6e0 con 0x7fa5a0105550 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.288+0000 7fa596ffd700 1 -- 172.21.15.39:0/1756974672 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fa59c01b5b0 con 0x7fa5a0105550 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.289+0000 7fa5a7c42700 1 -- 172.21.15.39:0/1756974672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 msgr2=0x7fa5a0061e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.289+0000 7fa5a7c42700 1 --2- 172.21.15.39:0/1756974672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0061e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa59c003f20 tx=0x7fa59c011f00 comp rx=0 tx=0).stop 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.289+0000 7fa5a7c42700 1 -- 172.21.15.39:0/1756974672 shutdown_connections 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.289+0000 7fa5a7c42700 1 --2- 172.21.15.39:0/1756974672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a0105550 0x7fa5a0061e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:09.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.289+0000 7fa5a7c42700 1 -- 172.21.15.39:0/1756974672 >> 172.21.15.39:0/1756974672 conn(0x7fa5a0100b10 msgr2=0x7fa5a01016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:09.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.289+0000 7fa5a7c42700 1 -- 172.21.15.39:0/1756974672 shutdown_connections 2024-05-24T11:31:09.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:07.289+0000 7fa5a7c42700 1 -- 172.21.15.39:0/1756974672 wait complete. 2024-05-24T11:31:09.008 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (2/15)... 2024-05-24T11:31:10.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:09 smithi039 ceph-mon[42919]: mgrmap e3: x(active, since 1.01378s) 2024-05-24T11:31:11.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:11 smithi039 ceph-mon[42919]: mgrmap e4: x(active, since 3s) 2024-05-24T11:31:11.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:11 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3241154297' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-24T11:31:12.030 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-24T11:31:12.031 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-24T11:31:12.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-24T11:31:12.040 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-24T11:31:12.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-24T11:31:12.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-05-24T11:30:55.204819+0000", 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.363+0000 7f904bb60700 1 Processor -- start 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.363+0000 7f904bb60700 1 -- start start 2024-05-24T11:31:12.043 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.363+0000 7f904bb60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.363+0000 7f904bb60700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440622c0 con 0x7f90441070c0 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.364+0000 7f90498fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.364+0000 7f90498fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45922/0 (socket says 172.21.15.39:45922) 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.364+0000 7f90498fc700 1 -- 172.21.15.39:0/477890205 learned_addr learned my addr 172.21.15.39:0/477890205 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.364+0000 7f90498fc700 1 -- 172.21.15.39:0/477890205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90441099f0 con 0x7f90441070c0 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.364+0000 7f90498fc700 1 --2- 172.21.15.39:0/477890205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441094b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f90340098a0 tx=0x7f9034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67989b55899b59b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.364+0000 7f90488fa700 1 -- 172.21.15.39:0/477890205 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9034004160 con 0x7f90441070c0 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.365+0000 7f90488fa700 1 -- 172.21.15.39:0/477890205 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f90340042c0 con 0x7f90441070c0 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.365+0000 7f90488fa700 1 -- 172.21.15.39:0/477890205 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9034011940 con 0x7f90441070c0 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.365+0000 7f904bb60700 1 -- 172.21.15.39:0/477890205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 msgr2=0x7f90441094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.365+0000 7f904bb60700 1 --2- 172.21.15.39:0/477890205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441094b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f90340098a0 tx=0x7f9034009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.365+0000 7f904bb60700 1 -- 172.21.15.39:0/477890205 shutdown_connections 2024-05-24T11:31:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.365+0000 7f904bb60700 1 --2- 172.21.15.39:0/477890205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441094b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.365+0000 7f904bb60700 1 -- 172.21.15.39:0/477890205 >> 172.21.15.39:0/477890205 conn(0x7f9044100b10 msgr2=0x7f9044102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.365+0000 7f904bb60700 1 -- 172.21.15.39:0/477890205 shutdown_connections 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.365+0000 7f904bb60700 1 -- 172.21.15.39:0/477890205 wait complete. 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.366+0000 7f904bb60700 1 Processor -- start 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.366+0000 7f904bb60700 1 -- start start 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.366+0000 7f904bb60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441068f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.366+0000 7f904bb60700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9044106e30 con 0x7f90441070c0 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.366+0000 7f90498fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.366+0000 7f90498fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45924/0 (socket says 172.21.15.39:45924) 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.366+0000 7f90498fc700 1 -- 172.21.15.39:0/3241154297 learned_addr learned my addr 172.21.15.39:0/3241154297 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.366+0000 7f90498fc700 1 -- 172.21.15.39:0/3241154297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9034009550 con 0x7f90441070c0 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.367+0000 7f90498fc700 1 --2- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441068f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9034006af0 tx=0x7f9034011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:12.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.367+0000 7f903affd700 1 -- 172.21.15.39:0/3241154297 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9034003bf0 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.367+0000 7f903affd700 1 -- 172.21.15.39:0/3241154297 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f903401b8b0 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.367+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9044105070 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.367+0000 7f903affd700 1 -- 172.21.15.39:0/3241154297 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9034023830 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.367+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9044105510 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.368+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904410a790 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.368+0000 7f903affd700 1 -- 172.21.15.39:0/3241154297 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f903401b440 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.368+0000 7f903affd700 1 --2- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f90300386b0 0x7f903003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.368+0000 7f903affd700 1 -- 172.21.15.39:0/3241154297 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f903404cb60 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.368+0000 7f90490fb700 1 --2- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f90300386b0 0x7f903003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.369+0000 7f90490fb700 1 --2- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f90300386b0 0x7f903003ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9040006fd0 tx=0x7f9040006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.373+0000 7f903affd700 1 -- 172.21.15.39:0/3241154297 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f903401b6f0 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.657+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f904410a9d0 con 0x7f90441070c0 2024-05-24T11:31:12.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.657+0000 7f903affd700 1 -- 172.21.15.39:0/3241154297 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f903402b350 con 0x7f90441070c0 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.659+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f90300386b0 msgr2=0x7f903003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.659+0000 7f904bb60700 1 --2- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f90300386b0 0x7f903003ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9040006fd0 tx=0x7f9040006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.659+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 msgr2=0x7f90441068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.659+0000 7f904bb60700 1 --2- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441068f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9034006af0 tx=0x7f9034011a50 comp rx=0 tx=0).stop 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.659+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 shutdown_connections 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.659+0000 7f904bb60700 1 --2- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f90300386b0 0x7f903003ab70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.660+0000 7f904bb60700 1 --2- 172.21.15.39:0/3241154297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90441070c0 0x7f90441068f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.660+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 >> 172.21.15.39:0/3241154297 conn(0x7f9044100b10 msgr2=0x7f90441016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.660+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 shutdown_connections 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:11.660+0000 7f904bb60700 1 -- 172.21.15.39:0/3241154297 wait complete. 2024-05-24T11:31:12.047 INFO:teuthology.orchestra.run.smithi039.stdout:mgr is available 2024-05-24T11:31:12.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:12 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2157602641' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-24T11:31:12.986 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:31:12.986 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-05-24T11:31:12.986 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:31:12.986 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-24T11:31:12.986 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-24T11:31:12.986 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-24T11:31:12.986 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-24T11:31:12.986 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:31:12.986 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.393+0000 7f2d8cf3a700 1 Processor -- start 2024-05-24T11:31:12.987 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.394+0000 7f2d8cf3a700 1 -- start start 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.394+0000 7f2d8cf3a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d88109570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.394+0000 7f2d8cf3a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d88062200 con 0x7f2d88107180 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.394+0000 7f2d8659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d88109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.394+0000 7f2d8659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d88109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45930/0 (socket says 172.21.15.39:45930) 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.394+0000 7f2d8659c700 1 -- 172.21.15.39:0/4236320456 learned_addr learned my addr 172.21.15.39:0/4236320456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.394+0000 7f2d8659c700 1 -- 172.21.15.39:0/4236320456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d88062340 con 0x7f2d88107180 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8659c700 1 --2- 172.21.15.39:0/4236320456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d88109570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2d780098a0 tx=0x7f2d78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aca3df83c2e7da7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8559a700 1 -- 172.21.15.39:0/4236320456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d78004160 con 0x7f2d88107180 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8559a700 1 -- 172.21.15.39:0/4236320456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2d780042c0 con 0x7f2d88107180 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8559a700 1 -- 172.21.15.39:0/4236320456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d78011940 con 0x7f2d88107180 2024-05-24T11:31:12.988 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/4236320456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 msgr2=0x7f2d88109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8cf3a700 1 --2- 172.21.15.39:0/4236320456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d88109570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2d780098a0 tx=0x7f2d78009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/4236320456 shutdown_connections 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8cf3a700 1 --2- 172.21.15.39:0/4236320456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d88109570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/4236320456 >> 172.21.15.39:0/4236320456 conn(0x7f2d88100b10 msgr2=0x7f2d88102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/4236320456 shutdown_connections 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.395+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/4236320456 wait complete. 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.396+0000 7f2d8cf3a700 1 Processor -- start 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.396+0000 7f2d8cf3a700 1 -- start start 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.396+0000 7f2d8cf3a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d8819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:12.989 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.396+0000 7f2d8cf3a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8819a7e0 con 0x7f2d88107180 2024-05-24T11:31:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d8659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d8819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d8659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d8819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45936/0 (socket says 172.21.15.39:45936) 2024-05-24T11:31:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d8659c700 1 -- 172.21.15.39:0/2157602641 learned_addr learned my addr 172.21.15.39:0/2157602641 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d8659c700 1 -- 172.21.15.39:0/2157602641 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009550 con 0x7f2d88107180 2024-05-24T11:31:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d8659c700 1 --2- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d8819a2a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2d78003f20 tx=0x7f2d78011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d7f7fe700 1 -- 172.21.15.39:0/2157602641 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d780037d0 con 0x7f2d88107180 2024-05-24T11:31:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d7f7fe700 1 -- 172.21.15.39:0/2157602641 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2d7801b440 con 0x7f2d88107180 2024-05-24T11:31:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d7f7fe700 1 -- 172.21.15.39:0/2157602641 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d78023590 con 0x7f2d88107180 2024-05-24T11:31:12.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8819a9e0 con 0x7f2d88107180 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.397+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8819ae80 con 0x7f2d88107180 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.398+0000 7f2d7f7fe700 1 -- 172.21.15.39:0/2157602641 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f2d78003930 con 0x7f2d88107180 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.398+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8810a780 con 0x7f2d88107180 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.398+0000 7f2d7f7fe700 1 --2- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f2d74038700 0x7f2d7403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.398+0000 7f2d7f7fe700 1 -- 172.21.15.39:0/2157602641 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2d7804c5e0 con 0x7f2d88107180 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.398+0000 7f2d85d9b700 1 --2- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f2d74038700 0x7f2d7403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.399+0000 7f2d85d9b700 1 --2- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f2d74038700 0x7f2d7403abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006fd0 tx=0x7f2d70006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.403+0000 7f2d7f7fe700 1 -- 172.21.15.39:0/2157602641 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2d7800bbd0 con 0x7f2d88107180 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.604+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2d8810a9c0 con 0x7f2d88107180 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.605+0000 7f2d7f7fe700 1 -- 172.21.15.39:0/2157602641 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f2d7800bbd0 con 0x7f2d88107180 2024-05-24T11:31:12.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f2d74038700 msgr2=0x7f2d7403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 --2- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f2d74038700 0x7f2d7403abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006fd0 tx=0x7f2d70006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 msgr2=0x7f2d8819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 --2- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d8819a2a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2d78003f20 tx=0x7f2d78011f00 comp rx=0 tx=0).stop 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 shutdown_connections 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 --2- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f2d74038700 0x7f2d7403abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 --2- 172.21.15.39:0/2157602641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d88107180 0x7f2d8819a2a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 >> 172.21.15.39:0/2157602641 conn(0x7f2d88100b10 msgr2=0x7f2d88101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 shutdown_connections 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:12.607+0000 7f2d8cf3a700 1 -- 172.21.15.39:0/2157602641 wait complete. 2024-05-24T11:31:12.992 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling cephadm module... 2024-05-24T11:31:14.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:13 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4084497786' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-24T11:31:14.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:13 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: ignoring --setuser ceph since I am not root 2024-05-24T11:31:14.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:13 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: ignoring --setgroup ceph since I am not root 2024-05-24T11:31:14.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.346+0000 7f9cc23c3700 1 Processor -- start 2024-05-24T11:31:14.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.346+0000 7f9cc23c3700 1 -- start start 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.346+0000 7f9cc23c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc109530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.346+0000 7f9cc23c3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc079e60 con 0x7f9cbc107140 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.346+0000 7f9cbbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.346+0000 7f9cbbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc109530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45946/0 (socket says 172.21.15.39:45946) 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.346+0000 7f9cbbfff700 1 -- 172.21.15.39:0/2353694580 learned_addr learned my addr 172.21.15.39:0/2353694580 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.347+0000 7f9cbbfff700 1 -- 172.21.15.39:0/2353694580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc109a70 con 0x7f9cbc107140 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.347+0000 7f9cbbfff700 1 --2- 172.21.15.39:0/2353694580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc109530 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40098a0 tx=0x7f9ca4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220d118d87c321a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.347+0000 7f9cbaffd700 1 -- 172.21.15.39:0/2353694580 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4004160 con 0x7f9cbc107140 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.347+0000 7f9cbaffd700 1 -- 172.21.15.39:0/2353694580 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9ca40042c0 con 0x7f9cbc107140 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.347+0000 7f9cbaffd700 1 -- 172.21.15.39:0/2353694580 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4011940 con 0x7f9cbc107140 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.348+0000 7f9cc23c3700 1 -- 172.21.15.39:0/2353694580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 msgr2=0x7f9cbc109530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.348+0000 7f9cc23c3700 1 --2- 172.21.15.39:0/2353694580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc109530 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40098a0 tx=0x7f9ca4009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:14.152 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.348+0000 7f9cc23c3700 1 -- 172.21.15.39:0/2353694580 shutdown_connections 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.348+0000 7f9cc23c3700 1 --2- 172.21.15.39:0/2353694580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc109530 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.348+0000 7f9cc23c3700 1 -- 172.21.15.39:0/2353694580 >> 172.21.15.39:0/2353694580 conn(0x7f9cbc100b90 msgr2=0x7f9cbc102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.348+0000 7f9cc23c3700 1 -- 172.21.15.39:0/2353694580 shutdown_connections 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.348+0000 7f9cc23c3700 1 -- 172.21.15.39:0/2353694580 wait complete. 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.348+0000 7f9cc23c3700 1 Processor -- start 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.348+0000 7f9cc23c3700 1 -- start start 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cc23c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cc23c3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc19a7e0 con 0x7f9cbc107140 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cbbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cbbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc19a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45956/0 (socket says 172.21.15.39:45956) 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cbbfff700 1 -- 172.21.15.39:0/4084497786 learned_addr learned my addr 172.21.15.39:0/4084497786 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cbbfff700 1 -- 172.21.15.39:0/4084497786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4009550 con 0x7f9cbc107140 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cbbfff700 1 --2- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc19a2a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4003f20 tx=0x7f9ca4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:14.153 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cb97fa700 1 -- 172.21.15.39:0/4084497786 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca40037d0 con 0x7f9cbc107140 2024-05-24T11:31:14.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cb97fa700 1 -- 172.21.15.39:0/4084497786 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9ca401b440 con 0x7f9cbc107140 2024-05-24T11:31:14.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cb97fa700 1 -- 172.21.15.39:0/4084497786 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4023590 con 0x7f9cbc107140 2024-05-24T11:31:14.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc19a9e0 con 0x7f9cbc107140 2024-05-24T11:31:14.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.349+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc19ae80 con 0x7f9cbc107140 2024-05-24T11:31:14.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.350+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cbc10a7b0 con 0x7f9cbc107140 2024-05-24T11:31:14.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.350+0000 7f9cb97fa700 1 -- 172.21.15.39:0/4084497786 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f9ca401b5b0 con 0x7f9cbc107140 2024-05-24T11:31:14.154 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.350+0000 7f9cb97fa700 1 --2- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f9ca8038700 0x7f9ca803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.351+0000 7f9cb97fa700 1 -- 172.21.15.39:0/4084497786 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9ca404c590 con 0x7f9cbc107140 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.351+0000 7f9cbb7fe700 1 --2- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f9ca8038700 0x7f9ca803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.351+0000 7f9cbb7fe700 1 --2- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f9ca8038700 0x7f9ca803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9cac006fd0 tx=0x7f9cac006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.355+0000 7f9cb97fa700 1 -- 172.21.15.39:0/4084497786 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9ca401b920 con 0x7f9cbc107140 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.617+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f9cbc10a9f0 con 0x7f9cbc107140 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.831+0000 7f9cb97fa700 1 -- 172.21.15.39:0/4084497786 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f9ca4003930 con 0x7f9cbc107140 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.833+0000 7f9cb97fa700 1 -- 172.21.15.39:0/4084497786 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f9ca404bfb0 con 0x7f9cbc107140 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f9ca8038700 msgr2=0x7f9ca803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 --2- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f9ca8038700 0x7f9ca803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9cac006fd0 tx=0x7f9cac006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:14.155 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 msgr2=0x7f9cbc19a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:14.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 --2- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc19a2a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4003f20 tx=0x7f9ca4011f00 comp rx=0 tx=0).stop 2024-05-24T11:31:14.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 shutdown_connections 2024-05-24T11:31:14.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 --2- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f9ca8038700 0x7f9ca803abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:14.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 --2- 172.21.15.39:0/4084497786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cbc107140 0x7f9cbc19a2a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:14.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 >> 172.21.15.39:0/4084497786 conn(0x7f9cbc100b90 msgr2=0x7f9cbc101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:14.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 shutdown_connections 2024-05-24T11:31:14.156 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:13.835+0000 7f9cc23c3700 1 -- 172.21.15.39:0/4084497786 wait complete. 2024-05-24T11:31:14.484 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:14 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:14.290+0000 7f9e58533200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-24T11:31:14.485 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:14 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:14.389+0000 7f9e58533200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-24T11:31:14.825 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:14 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:14.482+0000 7f9e58533200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-24T11:31:14.825 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:14 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:14.589+0000 7f9e58533200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-24T11:31:15.163 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:14 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4084497786' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-24T11:31:15.164 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:14 smithi039 ceph-mon[42919]: mgrmap e5: x(active, since 6s) 2024-05-24T11:31:15.164 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:14 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2719212069' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-24T11:31:15.224 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.528+0000 7f73e63db700 1 Processor -- start 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.528+0000 7f73e63db700 1 -- start start 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.528+0000 7f73e63db700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e01094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.528+0000 7f73e63db700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e0079ec0 con 0x7f73e01070d0 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.528+0000 7f73dffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e01094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.528+0000 7f73dffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e01094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45982/0 (socket says 172.21.15.39:45982) 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.528+0000 7f73dffff700 1 -- 172.21.15.39:0/3967765511 learned_addr learned my addr 172.21.15.39:0/3967765511 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:15.225 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.528+0000 7f73dffff700 1 -- 172.21.15.39:0/3967765511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e0109a00 con 0x7f73e01070d0 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.529+0000 7f73dffff700 1 --2- 172.21.15.39:0/3967765511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e01094c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f73d00098a0 tx=0x7f73d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879c3232a0d8b0be server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.529+0000 7f73deffd700 1 -- 172.21.15.39:0/3967765511 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73d0004160 con 0x7f73e01070d0 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.529+0000 7f73deffd700 1 -- 172.21.15.39:0/3967765511 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f73d00042c0 con 0x7f73e01070d0 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.529+0000 7f73deffd700 1 -- 172.21.15.39:0/3967765511 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73d0011940 con 0x7f73e01070d0 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.529+0000 7f73e63db700 1 -- 172.21.15.39:0/3967765511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 msgr2=0x7f73e01094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.529+0000 7f73e63db700 1 --2- 172.21.15.39:0/3967765511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e01094c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f73d00098a0 tx=0x7f73d0009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.529+0000 7f73e63db700 1 -- 172.21.15.39:0/3967765511 shutdown_connections 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.529+0000 7f73e63db700 1 --2- 172.21.15.39:0/3967765511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e01094c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.529+0000 7f73e63db700 1 -- 172.21.15.39:0/3967765511 >> 172.21.15.39:0/3967765511 conn(0x7f73e0100b80 msgr2=0x7f73e0102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.530+0000 7f73e63db700 1 -- 172.21.15.39:0/3967765511 shutdown_connections 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.530+0000 7f73e63db700 1 -- 172.21.15.39:0/3967765511 wait complete. 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.530+0000 7f73e63db700 1 Processor -- start 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.530+0000 7f73e63db700 1 -- start start 2024-05-24T11:31:15.226 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.530+0000 7f73e63db700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e0106900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.530+0000 7f73e63db700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e0106e40 con 0x7f73e01070d0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73dffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e0106900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73dffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e0106900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45988/0 (socket says 172.21.15.39:45988) 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73dffff700 1 -- 172.21.15.39:0/2719212069 learned_addr learned my addr 172.21.15.39:0/2719212069 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73dffff700 1 -- 172.21.15.39:0/2719212069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0009550 con 0x7f73e01070d0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73dffff700 1 --2- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e0106900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006af0 tx=0x7f73d0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73dd7fa700 1 -- 172.21.15.39:0/2719212069 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73d0003bf0 con 0x7f73e01070d0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e0105080 con 0x7f73e01070d0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73dd7fa700 1 -- 172.21.15.39:0/2719212069 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f73d001b8b0 con 0x7f73e01070d0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73dd7fa700 1 -- 172.21.15.39:0/2719212069 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73d0023800 con 0x7f73e01070d0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.531+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e0105520 con 0x7f73e01070d0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.532+0000 7f73dd7fa700 1 -- 172.21.15.39:0/2719212069 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f73d0023a70 con 0x7f73e01070d0 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.532+0000 7f73dd7fa700 1 --2- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f73c80386b0 0x7f73c803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:15.227 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.532+0000 7f73dd7fa700 1 -- 172.21.15.39:0/2719212069 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f73d004d700 con 0x7f73e01070d0 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.532+0000 7f73df7fe700 1 -- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f73c80386b0 msgr2=0x7f73c803ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1790202809 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.532+0000 7f73df7fe700 1 --2- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f73c80386b0 0x7f73c803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.532+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e00625d0 con 0x7f73e01070d0 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.537+0000 7f73dd7fa700 1 -- 172.21.15.39:0/2719212069 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f73d0023d20 con 0x7f73e01070d0 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.733+0000 7f73df7fe700 1 -- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f73c80386b0 msgr2=0x7f73c803ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1790202809 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.733+0000 7f73df7fe700 1 --2- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f73c80386b0 0x7f73c803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.821+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f73e0105980 con 0x7f73e01070d0 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.821+0000 7f73dd7fa700 1 -- 172.21.15.39:0/2719212069 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f73d000b3e0 con 0x7f73e01070d0 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f73c80386b0 msgr2=0x7f73c803ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 --2- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f73c80386b0 0x7f73c803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 msgr2=0x7f73e0106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 --2- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e0106900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006af0 tx=0x7f73d0011a50 comp rx=0 tx=0).stop 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 shutdown_connections 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 --2- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f73c80386b0 0x7f73c803ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 --2- 172.21.15.39:0/2719212069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e01070d0 0x7f73e0106900 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:15.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 >> 172.21.15.39:0/2719212069 conn(0x7f73e0100b80 msgr2=0x7f73e01016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:15.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 shutdown_connections 2024-05-24T11:31:15.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:14.823+0000 7f73e63db700 1 -- 172.21.15.39:0/2719212069 wait complete. 2024-05-24T11:31:15.229 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-05-24T11:31:15.229 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 5... 2024-05-24T11:31:15.544 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:15.270+0000 7f9e58533200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:31:15.825 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:15.541+0000 7f9e58533200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-24T11:31:15.825 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:15.634+0000 7f9e58533200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-24T11:31:15.825 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:15.748+0000 7f9e58533200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-24T11:31:16.325 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:15.912+0000 7f9e58533200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-24T11:31:16.325 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:16.045+0000 7f9e58533200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-24T11:31:16.325 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:16.134+0000 7f9e58533200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-24T11:31:16.825 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:16.520+0000 7f9e58533200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-24T11:31:17.228 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:16.925+0000 7f9e58533200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-24T11:31:17.228 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:17 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:17.129+0000 7f9e58533200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-24T11:31:17.228 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:17 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:17.226+0000 7f9e58533200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-24T11:31:18.281 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:18.023+0000 7f9e58533200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-24T11:31:18.281 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:18.123+0000 7f9e58533200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-24T11:31:18.281 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:18.278+0000 7f9e58533200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-24T11:31:19.325 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:18.936+0000 7f9e58533200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-24T11:31:19.326 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:19 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:19.125+0000 7f9e58533200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-24T11:31:19.825 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:19 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:19.326+0000 7f9e58533200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-24T11:31:20.575 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:20.296+0000 7f9e58533200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-24T11:31:20.882 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:20.591+0000 7f9e58533200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:31:20.883 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:20.683+0000 7f9e58533200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-24T11:31:20.883 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:20.880+0000 7f9e58533200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-24T11:31:21.325 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: Active manager daemon x restarted 2024-05-24T11:31:21.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: Activating manager daemon x 2024-05-24T11:31:21.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: osdmap e2: 0 total, 0 up, 0 in 2024-05-24T11:31:21.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: mgrmap e6: x(active, starting, since 0.0122124s) 2024-05-24T11:31:21.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:31:21.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-24T11:31:21.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-24T11:31:21.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-24T11:31:21.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-24T11:31:21.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: Manager daemon x is now available 2024-05-24T11:31:21.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:20 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:22.221 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: Found migration_current of "None". Setting to last migration. 2024-05-24T11:31:22.221 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:22.221 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:31:22.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:31:22.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-24T11:31:22.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-24T11:31:22.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:22.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:22.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: [24/May/2024:11:31:21] ENGINE Bus STARTING 2024-05-24T11:31:22.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: [24/May/2024:11:31:21] ENGINE Serving on https://172.21.15.39:7150 2024-05-24T11:31:22.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: mgrmap e7: x(active, since 1.01193s) 2024-05-24T11:31:22.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:21 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:31:22.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-24T11:31:22.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-24T11:31:22.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-24T11:31:22.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-24T11:31:22.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.578+0000 7f673342f700 1 Processor -- start 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.579+0000 7f673342f700 1 -- start start 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.579+0000 7f673342f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c1094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.579+0000 7f673342f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c079ec0 con 0x7f672c1070f0 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.579+0000 7f67311cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c1094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.579+0000 7f67311cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c1094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45998/0 (socket says 172.21.15.39:45998) 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.579+0000 7f67311cb700 1 -- 172.21.15.39:0/4174383354 learned_addr learned my addr 172.21.15.39:0/4174383354 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.579+0000 7f67311cb700 1 -- 172.21.15.39:0/4174383354 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c109a20 con 0x7f672c1070f0 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.580+0000 7f67311cb700 1 --2- 172.21.15.39:0/4174383354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c1094e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67280098a0 tx=0x7f6728009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7c2bbfe64baa36 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.580+0000 7f6723fff700 1 -- 172.21.15.39:0/4174383354 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6728004160 con 0x7f672c1070f0 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.580+0000 7f6723fff700 1 -- 172.21.15.39:0/4174383354 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f67280042c0 con 0x7f672c1070f0 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.580+0000 7f6723fff700 1 -- 172.21.15.39:0/4174383354 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6728011940 con 0x7f672c1070f0 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.580+0000 7f673342f700 1 -- 172.21.15.39:0/4174383354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 msgr2=0x7f672c1094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:22.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.580+0000 7f673342f700 1 --2- 172.21.15.39:0/4174383354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c1094e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67280098a0 tx=0x7f6728009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.580+0000 7f673342f700 1 -- 172.21.15.39:0/4174383354 shutdown_connections 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.580+0000 7f673342f700 1 --2- 172.21.15.39:0/4174383354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c1094e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.580+0000 7f673342f700 1 -- 172.21.15.39:0/4174383354 >> 172.21.15.39:0/4174383354 conn(0x7f672c100b80 msgr2=0x7f672c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.581+0000 7f673342f700 1 -- 172.21.15.39:0/4174383354 shutdown_connections 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.581+0000 7f673342f700 1 -- 172.21.15.39:0/4174383354 wait complete. 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.581+0000 7f673342f700 1 Processor -- start 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.581+0000 7f673342f700 1 -- start start 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f673342f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f673342f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c19a820 con 0x7f672c1070f0 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f67311cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f67311cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46004/0 (socket says 172.21.15.39:46004) 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f67311cb700 1 -- 172.21.15.39:0/2453139636 learned_addr learned my addr 172.21.15.39:0/2453139636 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:22.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f67311cb700 1 -- 172.21.15.39:0/2453139636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6728009550 con 0x7f672c1070f0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f67311cb700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c19a2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6728006af0 tx=0x7f6728011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6728003bf0 con 0x7f672c1070f0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f673342f700 1 -- 172.21.15.39:0/2453139636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c19aa20 con 0x7f672c1070f0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f672801b8b0 con 0x7f672c1070f0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6728023800 con 0x7f672c1070f0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.582+0000 7f673342f700 1 -- 172.21.15.39:0/2453139636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c19aec0 con 0x7f672c1070f0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.584+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f672801ba20 con 0x7f672c1070f0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.584+0000 7f67227fc700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 0x7f671803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.584+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 --> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f671803b280 con 0x7f67180386b0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.584+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f672804cc50 con 0x7f672c1070f0 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.584+0000 7f67309ca700 1 -- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 msgr2=0x7f671803ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1790202809 2024-05-24T11:31:22.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.584+0000 7f67309ca700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 0x7f671803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.785+0000 7f67309ca700 1 -- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 msgr2=0x7f671803ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1790202809 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:15.785+0000 7f67309ca700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 0x7f671803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:16.186+0000 7f67309ca700 1 -- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 msgr2=0x7f671803ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1790202809 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:16.186+0000 7f67309ca700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 0x7f671803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:16.987+0000 7f67309ca700 1 -- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 msgr2=0x7f671803ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1790202809 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:16.987+0000 7f67309ca700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 0x7f671803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:18.588+0000 7f67309ca700 1 -- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 msgr2=0x7f671803ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1790202809 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:18.588+0000 7f67309ca700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 0x7f671803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:20.905+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7f672801b440 con 0x7f672c1070f0 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:20.905+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 msgr2=0x7f671803ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:31:22.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:20.905+0000 7f67227fc700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 0x7f671803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.903+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f672800b3e0 con 0x7f672c1070f0 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.903+0000 7f67227fc700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f671803bfe0 0x7f671803e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.903+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f671803b280 con 0x7f671803bfe0 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.904+0000 7f67309ca700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f671803bfe0 0x7f671803e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.904+0000 7f67309ca700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f671803bfe0 0x7f671803e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f671c003a10 tx=0x7f671c009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.905+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 <== mgr.14118 v2:172.21.15.39:6800/3960296835 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f671803b280 con 0x7f671803bfe0 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.911+0000 7f673342f700 1 -- 172.21.15.39:0/2453139636 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f672c19b050 con 0x7f671803bfe0 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.911+0000 7f67227fc700 1 -- 172.21.15.39:0/2453139636 <== mgr.14118 v2:172.21.15.39:6800/3960296835 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f671c008850 con 0x7f671803bfe0 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 -- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f671803bfe0 msgr2=0x7f671803e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f671803bfe0 0x7f671803e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f671c003a10 tx=0x7f671c009290 comp rx=0 tx=0).stop 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 -- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 msgr2=0x7f672c19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c19a2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6728006af0 tx=0x7f6728011a50 comp rx=0 tx=0).stop 2024-05-24T11:31:22.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 -- 172.21.15.39:0/2453139636 shutdown_connections 2024-05-24T11:31:22.273 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f672c1070f0 0x7f672c19a2e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:22.273 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f671803bfe0 0x7f671803e3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:22.273 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 --2- 172.21.15.39:0/2453139636 >> [v2:172.21.15.39:6800/1790202809,v1:172.21.15.39:6801/1790202809] conn(0x7f67180386b0 0x7f671803ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:22.273 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 -- 172.21.15.39:0/2453139636 >> 172.21.15.39:0/2453139636 conn(0x7f672c100b80 msgr2=0x7f672c1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:22.273 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 -- 172.21.15.39:0/2453139636 shutdown_connections 2024-05-24T11:31:22.273 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:21.912+0000 7f673342f700 1 -- 172.21.15.39:0/2453139636 wait complete. 2024-05-24T11:31:22.273 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 5 is available 2024-05-24T11:31:22.273 INFO:teuthology.orchestra.run.smithi039.stdout:Setting orchestrator backend to cephadm... 2024-05-24T11:31:23.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.601+0000 7fae89758700 1 Processor -- start 2024-05-24T11:31:23.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.601+0000 7fae89758700 1 -- start start 2024-05-24T11:31:23.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.601+0000 7fae89758700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae84109510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:23.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.601+0000 7fae89758700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae84079ec0 con 0x7fae84107120 2024-05-24T11:31:23.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.602+0000 7fae82ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae84109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:23.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.602+0000 7fae82ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae84109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51970/0 (socket says 172.21.15.39:51970) 2024-05-24T11:31:23.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.602+0000 7fae82ffd700 1 -- 172.21.15.39:0/2972073293 learned_addr learned my addr 172.21.15.39:0/2972073293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:23.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.602+0000 7fae82ffd700 1 -- 172.21.15.39:0/2972073293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84109a50 con 0x7fae84107120 2024-05-24T11:31:23.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.602+0000 7fae82ffd700 1 --2- 172.21.15.39:0/2972073293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae84109510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae780098a0 tx=0x7fae78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e78edab109b781 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.602+0000 7fae81ffb700 1 -- 172.21.15.39:0/2972073293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae78004160 con 0x7fae84107120 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.602+0000 7fae81ffb700 1 -- 172.21.15.39:0/2972073293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fae780042c0 con 0x7fae84107120 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.602+0000 7fae81ffb700 1 -- 172.21.15.39:0/2972073293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae78011940 con 0x7fae84107120 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.603+0000 7fae89758700 1 -- 172.21.15.39:0/2972073293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 msgr2=0x7fae84109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.603+0000 7fae89758700 1 --2- 172.21.15.39:0/2972073293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae84109510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae780098a0 tx=0x7fae78009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.603+0000 7fae89758700 1 -- 172.21.15.39:0/2972073293 shutdown_connections 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.603+0000 7fae89758700 1 --2- 172.21.15.39:0/2972073293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae84109510 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.603+0000 7fae89758700 1 -- 172.21.15.39:0/2972073293 >> 172.21.15.39:0/2972073293 conn(0x7fae84100bb0 msgr2=0x7fae84102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.603+0000 7fae89758700 1 -- 172.21.15.39:0/2972073293 shutdown_connections 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.603+0000 7fae89758700 1 -- 172.21.15.39:0/2972073293 wait complete. 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.604+0000 7fae89758700 1 Processor -- start 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.604+0000 7fae89758700 1 -- start start 2024-05-24T11:31:23.218 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.604+0000 7fae89758700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae8419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.604+0000 7fae89758700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8419a820 con 0x7fae84107120 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.604+0000 7fae82ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae8419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.604+0000 7fae82ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae8419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51986/0 (socket says 172.21.15.39:51986) 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.604+0000 7fae82ffd700 1 -- 172.21.15.39:0/1360018044 learned_addr learned my addr 172.21.15.39:0/1360018044 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.604+0000 7fae82ffd700 1 -- 172.21.15.39:0/1360018044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae78009550 con 0x7fae84107120 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.604+0000 7fae82ffd700 1 --2- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae8419a2e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fae78006af0 tx=0x7fae78011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.605+0000 7fae6bfff700 1 -- 172.21.15.39:0/1360018044 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae78003bf0 con 0x7fae84107120 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.605+0000 7fae6bfff700 1 -- 172.21.15.39:0/1360018044 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fae7801b8b0 con 0x7fae84107120 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.605+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8419aa20 con 0x7fae84107120 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.605+0000 7fae6bfff700 1 -- 172.21.15.39:0/1360018044 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae78023800 con 0x7fae84107120 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.605+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8419aec0 con 0x7fae84107120 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.605+0000 7fae6bfff700 1 -- 172.21.15.39:0/1360018044 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fae78023960 con 0x7fae84107120 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.605+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8410a7f0 con 0x7fae84107120 2024-05-24T11:31:23.219 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.606+0000 7fae6bfff700 1 --2- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7fae6c0385e0 0x7fae6c03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.606+0000 7fae6bfff700 1 -- 172.21.15.39:0/1360018044 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fae7804c9b0 con 0x7fae84107120 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.606+0000 7fae827fc700 1 --2- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7fae6c0385e0 0x7fae6c03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.606+0000 7fae827fc700 1 --2- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7fae6c0385e0 0x7fae6c03aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae74006fd0 tx=0x7fae74006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.611+0000 7fae6bfff700 1 -- 172.21.15.39:0/1360018044 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fae78023c10 con 0x7fae84107120 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.832+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fae84101df0 con 0x7fae6c0385e0 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.840+0000 7fae6bfff700 1 -- 172.21.15.39:0/1360018044 <== mgr.14118 v2:172.21.15.39:6800/3960296835 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fae84101df0 con 0x7fae6c0385e0 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7fae6c0385e0 msgr2=0x7fae6c03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 --2- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7fae6c0385e0 0x7fae6c03aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae74006fd0 tx=0x7fae74006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 msgr2=0x7fae8419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 --2- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae8419a2e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fae78006af0 tx=0x7fae78011a50 comp rx=0 tx=0).stop 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 shutdown_connections 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 --2- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7fae6c0385e0 0x7fae6c03aaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:23.220 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 --2- 172.21.15.39:0/1360018044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae84107120 0x7fae8419a2e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:23.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 >> 172.21.15.39:0/1360018044 conn(0x7fae84100bb0 msgr2=0x7fae841016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:23.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 shutdown_connections 2024-05-24T11:31:23.221 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:22.843+0000 7fae89758700 1 -- 172.21.15.39:0/1360018044 wait complete. 2024-05-24T11:31:24.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:23 smithi039 ceph-mon[42919]: [24/May/2024:11:31:21] ENGINE Serving on http://172.21.15.39:8765 2024-05-24T11:31:24.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:23 smithi039 ceph-mon[42919]: [24/May/2024:11:31:21] ENGINE Bus STARTED 2024-05-24T11:31:24.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:23 smithi039 ceph-mon[42919]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-24T11:31:24.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:23 smithi039 ceph-mon[42919]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-24T11:31:24.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:23 smithi039 ceph-mon[42919]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:24.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:23 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:24.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:23 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:31:24.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-24T11:31:24.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.575+0000 7f1ed0798700 1 Processor -- start 2024-05-24T11:31:24.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.575+0000 7f1ed0798700 1 -- start start 2024-05-24T11:31:24.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.575+0000 7f1ed0798700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec8105330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:24.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.575+0000 7f1ed0798700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec8079e60 con 0x7f1ec8104f10 2024-05-24T11:31:24.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.576+0000 7f1ece534700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec8105330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:24.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.576+0000 7f1ece534700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec8105330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52000/0 (socket says 172.21.15.39:52000) 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.576+0000 7f1ece534700 1 -- 172.21.15.39:0/592468901 learned_addr learned my addr 172.21.15.39:0/592468901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.576+0000 7f1ece534700 1 -- 172.21.15.39:0/592468901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8105870 con 0x7f1ec8104f10 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.576+0000 7f1ece534700 1 --2- 172.21.15.39:0/592468901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec8105330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40098a0 tx=0x7f1ec4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf276cf7ee2b38f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.576+0000 7f1ecd532700 1 -- 172.21.15.39:0/592468901 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ec4004160 con 0x7f1ec8104f10 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.576+0000 7f1ecd532700 1 -- 172.21.15.39:0/592468901 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f1ec40042c0 con 0x7f1ec8104f10 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.576+0000 7f1ecd532700 1 -- 172.21.15.39:0/592468901 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ec4011940 con 0x7f1ec8104f10 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.577+0000 7f1ed0798700 1 -- 172.21.15.39:0/592468901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 msgr2=0x7f1ec8105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.577+0000 7f1ed0798700 1 --2- 172.21.15.39:0/592468901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec8105330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40098a0 tx=0x7f1ec4009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.577+0000 7f1ed0798700 1 -- 172.21.15.39:0/592468901 shutdown_connections 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.577+0000 7f1ed0798700 1 --2- 172.21.15.39:0/592468901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec8105330 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.577+0000 7f1ed0798700 1 -- 172.21.15.39:0/592468901 >> 172.21.15.39:0/592468901 conn(0x7f1ec8100b30 msgr2=0x7f1ec8102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.577+0000 7f1ed0798700 1 -- 172.21.15.39:0/592468901 shutdown_connections 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.577+0000 7f1ed0798700 1 -- 172.21.15.39:0/592468901 wait complete. 2024-05-24T11:31:24.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ed0798700 1 Processor -- start 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ed0798700 1 -- start start 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ed0798700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ed0798700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec819ca10 con 0x7f1ec8104f10 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ece534700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ece534700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52016/0 (socket says 172.21.15.39:52016) 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ece534700 1 -- 172.21.15.39:0/685974146 learned_addr learned my addr 172.21.15.39:0/685974146 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ece534700 1 -- 172.21.15.39:0/685974146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec4009550 con 0x7f1ec8104f10 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ece534700 1 --2- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec819c4d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006af0 tx=0x7f1ec40119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ebb7fe700 1 -- 172.21.15.39:0/685974146 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ec4003af0 con 0x7f1ec8104f10 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ebb7fe700 1 -- 172.21.15.39:0/685974146 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f1ec401b8b0 con 0x7f1ec8104f10 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec819cc10 con 0x7f1ec8104f10 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.578+0000 7f1ebb7fe700 1 -- 172.21.15.39:0/685974146 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ec4023640 con 0x7f1ec8104f10 2024-05-24T11:31:24.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.579+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec819d0b0 con 0x7f1ec8104f10 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.579+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec8196290 con 0x7f1ec8104f10 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.580+0000 7f1ebb7fe700 1 -- 172.21.15.39:0/685974146 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f1ec401ba20 con 0x7f1ec8104f10 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.580+0000 7f1ebb7fe700 1 --2- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f1eb4038590 0x7f1eb403aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.580+0000 7f1ebb7fe700 1 -- 172.21.15.39:0/685974146 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1ec404ca90 con 0x7f1ec8104f10 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.580+0000 7f1ecdd33700 1 --2- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f1eb4038590 0x7f1eb403aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.580+0000 7f1ecdd33700 1 --2- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f1eb4038590 0x7f1eb403aa50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc006fd0 tx=0x7f1ebc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.585+0000 7f1ebb7fe700 1 -- 172.21.15.39:0/685974146 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ec401bd70 con 0x7f1ec8104f10 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.788+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ec802d010 con 0x7f1eb4038590 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.789+0000 7f1ebb7fe700 1 -- 172.21.15.39:0/685974146 <== mgr.14118 v2:172.21.15.39:6800/3960296835 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f1ec802d010 con 0x7f1eb4038590 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f1eb4038590 msgr2=0x7f1eb403aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 --2- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f1eb4038590 0x7f1eb403aa50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc006fd0 tx=0x7f1ebc006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 msgr2=0x7f1ec819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 --2- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec819c4d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006af0 tx=0x7f1ec40119c0 comp rx=0 tx=0).stop 2024-05-24T11:31:24.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 shutdown_connections 2024-05-24T11:31:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 --2- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f1eb4038590 0x7f1eb403aa50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 --2- 172.21.15.39:0/685974146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec8104f10 0x7f1ec819c4d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 >> 172.21.15.39:0/685974146 conn(0x7f1ec8100b30 msgr2=0x7f1ec81933d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 shutdown_connections 2024-05-24T11:31:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:23.791+0000 7f1ed0798700 1 -- 172.21.15.39:0/685974146 wait complete. 2024-05-24T11:31:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:Generating ssh key... 2024-05-24T11:31:25.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:24 smithi039 ceph-mon[42919]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:24 smithi039 ceph-mon[42919]: mgrmap e8: x(active, since 2s) 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:24 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:24 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: Generating public/private rsa key pair. 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: Your identification has been saved in /tmp/tmpxcfqftlx/key. 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: Your public key has been saved in /tmp/tmpxcfqftlx/key.pub. 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: The key fingerprint is: 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: SHA256:oCRYUpryPgIvSR44str0vpgcMYmX4Ql+B+vcODopXwI ceph-f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: The key's randomart image is: 2024-05-24T11:31:25.076 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: +---[RSA 3072]----+ 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: |..o | 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: | * | 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: |* o.. . | 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: |+= Bo. . | 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: |E+Xo.. S | 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: |=O=o+ | 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: |=.O=.. | 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: |oO.O. | 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: |ooB.+. | 2024-05-24T11:31:25.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:24 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: +----[SHA256]-----+ 2024-05-24T11:31:25.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.524+0000 7f676dc2c700 1 Processor -- start 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.524+0000 7f676dc2c700 1 -- start start 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f676dc2c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f6768106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f676dc2c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768107030 con 0x7f6768106640 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f67677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f6768106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f67677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f6768106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52032/0 (socket says 172.21.15.39:52032) 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f67677fe700 1 -- 172.21.15.39:0/972425481 learned_addr learned my addr 172.21.15.39:0/972425481 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f67677fe700 1 -- 172.21.15.39:0/972425481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6768107840 con 0x7f6768106640 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f67677fe700 1 --2- 172.21.15.39:0/972425481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f6768106a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f675c0098a0 tx=0x7f675c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81eb728d09d64d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f67667fc700 1 -- 172.21.15.39:0/972425481 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f675c004160 con 0x7f6768106640 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f67667fc700 1 -- 172.21.15.39:0/972425481 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f675c0042c0 con 0x7f6768106640 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.525+0000 7f67667fc700 1 -- 172.21.15.39:0/972425481 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f675c011940 con 0x7f6768106640 2024-05-24T11:31:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.526+0000 7f676dc2c700 1 -- 172.21.15.39:0/972425481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 msgr2=0x7f6768106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.526+0000 7f676dc2c700 1 --2- 172.21.15.39:0/972425481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f6768106a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f675c0098a0 tx=0x7f675c009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.526+0000 7f676dc2c700 1 -- 172.21.15.39:0/972425481 shutdown_connections 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.526+0000 7f676dc2c700 1 --2- 172.21.15.39:0/972425481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f6768106a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.526+0000 7f676dc2c700 1 -- 172.21.15.39:0/972425481 >> 172.21.15.39:0/972425481 conn(0x7f6768101c00 msgr2=0x7f6768104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.526+0000 7f676dc2c700 1 -- 172.21.15.39:0/972425481 shutdown_connections 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.526+0000 7f676dc2c700 1 -- 172.21.15.39:0/972425481 wait complete. 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.526+0000 7f676dc2c700 1 Processor -- start 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.527+0000 7f676dc2c700 1 -- start start 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.527+0000 7f676dc2c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f676819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.527+0000 7f676dc2c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676819a7e0 con 0x7f6768106640 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.527+0000 7f67677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f676819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.527+0000 7f67677fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f676819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52048/0 (socket says 172.21.15.39:52048) 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.527+0000 7f67677fe700 1 -- 172.21.15.39:0/126263405 learned_addr learned my addr 172.21.15.39:0/126263405 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.527+0000 7f67677fe700 1 -- 172.21.15.39:0/126263405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c009550 con 0x7f6768106640 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.527+0000 7f67677fe700 1 --2- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f676819a2a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f675c006af0 tx=0x7f675c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.528+0000 7f6764ff9700 1 -- 172.21.15.39:0/126263405 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f675c003bf0 con 0x7f6768106640 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.528+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676819a9e0 con 0x7f6768106640 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.528+0000 7f6764ff9700 1 -- 172.21.15.39:0/126263405 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f675c01b8b0 con 0x7f6768106640 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.528+0000 7f6764ff9700 1 -- 172.21.15.39:0/126263405 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f675c0237f0 con 0x7f6768106640 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.528+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676819ae80 con 0x7f6768106640 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.528+0000 7f6764ff9700 1 -- 172.21.15.39:0/126263405 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f675c023ab0 con 0x7f6768106640 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.529+0000 7f6764ff9700 1 --2- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f67500386b0 0x7f675003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.529+0000 7f6764ff9700 1 -- 172.21.15.39:0/126263405 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f675c04cc40 con 0x7f6768106640 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.529+0000 7f6766ffd700 1 --2- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f67500386b0 0x7f675003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.529+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67680625d0 con 0x7f6768106640 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.529+0000 7f6766ffd700 1 --2- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f67500386b0 0x7f675003ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6758006fd0 tx=0x7f6758006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.534+0000 7f6764ff9700 1 -- 172.21.15.39:0/126263405 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f675c018760 con 0x7f6768106640 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.736+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6768102e60 con 0x7f67500386b0 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.806+0000 7f6764ff9700 1 -- 172.21.15.39:0/126263405 <== mgr.14118 v2:172.21.15.39:6800/3960296835 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6768102e60 con 0x7f67500386b0 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f67500386b0 msgr2=0x7f675003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 --2- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f67500386b0 0x7f675003ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6758006fd0 tx=0x7f6758006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 msgr2=0x7f676819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 --2- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f676819a2a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f675c006af0 tx=0x7f675c011a50 comp rx=0 tx=0).stop 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 shutdown_connections 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 --2- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f67500386b0 0x7f675003ab70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 --2- 172.21.15.39:0/126263405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6768106640 0x7f676819a2a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 >> 172.21.15.39:0/126263405 conn(0x7f6768101c00 msgr2=0x7f6768102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:25.185 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 shutdown_connections 2024-05-24T11:31:25.186 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:24.809+0000 7f676dc2c700 1 -- 172.21.15.39:0/126263405 wait complete. 2024-05-24T11:31:26.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:25 smithi039 ceph-mon[42919]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:26.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:25 smithi039 ceph-mon[42919]: Generating ssh key... 2024-05-24T11:31:26.131 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1H4LQC8DpFBgI21N9m9sNd2eyc/cpdHTBN+gSyP3pno3I51itjjlPAAdRgaaJme6RiJQtiz7p36JzYw+Rg17MwzhlkDtARPQ8UoEPHmutZHYH2tHF5NzqOvJlI84M5nrdZTuSdzzYJsohhAPIozC0666fGcwGGMa4Z9M+++ctlhfdglfGWpM17iMgG7RB0JPWdoO/mHbQ7ruG7wsASs58a/1YyOu3QVWmfnU+wmvAXXR/7cus/MfouvrLUXTmUif2Pg4xYedVgfvTMfTsozFJqJjT2kHXDxLX5PUQ/poM0I/P0qjaCprlrGBEY9XefFTzJLslsDtILqzuIYVaY8agenFbKL5MTGeRzlK6mJx1iv9iCXChVI+MfaDvBAZwMf78AsmdlwDExF8L7x7EhnCSPfKdH5AcZphtTIfQX2hfP6ou1M/IWrmJ6by8akalPsXYvfHYlOnRcSHco8qyuzIZU6P64vW+gYlLSmjs++Vg3QoctJpPNsuKLzzQ+mxxCSc= ceph-f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.542+0000 7f8d01ec9700 1 Processor -- start 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.542+0000 7f8d01ec9700 1 -- start start 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.542+0000 7f8d01ec9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.542+0000 7f8d01ec9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc079790 con 0x7f8cfc07ad60 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.542+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.542+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc0791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52050/0 (socket says 172.21.15.39:52050) 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.542+0000 7f8cfb7fe700 1 -- 172.21.15.39:0/619549983 learned_addr learned my addr 172.21.15.39:0/619549983 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.542+0000 7f8cfb7fe700 1 -- 172.21.15.39:0/619549983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0798d0 con 0x7f8cfc07ad60 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.542+0000 7f8cfb7fe700 1 --2- 172.21.15.39:0/619549983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc0791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098a0 tx=0x7f8cec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c393e407b67319c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8cfa7fc700 1 -- 172.21.15.39:0/619549983 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cec004160 con 0x7f8cfc07ad60 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8cfa7fc700 1 -- 172.21.15.39:0/619549983 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8cec0042c0 con 0x7f8cfc07ad60 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8cfa7fc700 1 -- 172.21.15.39:0/619549983 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cec011940 con 0x7f8cfc07ad60 2024-05-24T11:31:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8d01ec9700 1 -- 172.21.15.39:0/619549983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 msgr2=0x7f8cfc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8d01ec9700 1 --2- 172.21.15.39:0/619549983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc0791c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098a0 tx=0x7f8cec009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8d01ec9700 1 -- 172.21.15.39:0/619549983 shutdown_connections 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8d01ec9700 1 --2- 172.21.15.39:0/619549983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc0791c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8d01ec9700 1 -- 172.21.15.39:0/619549983 >> 172.21.15.39:0/619549983 conn(0x7f8cfc101c20 msgr2=0x7f8cfc104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8d01ec9700 1 -- 172.21.15.39:0/619549983 shutdown_connections 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.543+0000 7f8d01ec9700 1 -- 172.21.15.39:0/619549983 wait complete. 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.544+0000 7f8d01ec9700 1 Processor -- start 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.544+0000 7f8d01ec9700 1 -- start start 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.544+0000 7f8d01ec9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.544+0000 7f8d01ec9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc19c9e0 con 0x7f8cfc07ad60 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.544+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.544+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52056/0 (socket says 172.21.15.39:52056) 2024-05-24T11:31:26.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.544+0000 7f8cfb7fe700 1 -- 172.21.15.39:0/2119588763 learned_addr learned my addr 172.21.15.39:0/2119588763 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:26.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.545+0000 7f8cfb7fe700 1 -- 172.21.15.39:0/2119588763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec009550 con 0x7f8cfc07ad60 2024-05-24T11:31:26.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.545+0000 7f8cfb7fe700 1 --2- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc19c4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8cec003f20 tx=0x7f8cec011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:26.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.545+0000 7f8cf8ff9700 1 -- 172.21.15.39:0/2119588763 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cec0037d0 con 0x7f8cfc07ad60 2024-05-24T11:31:26.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.545+0000 7f8cf8ff9700 1 -- 172.21.15.39:0/2119588763 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8cec01b440 con 0x7f8cfc07ad60 2024-05-24T11:31:26.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.545+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc19cbe0 con 0x7f8cfc07ad60 2024-05-24T11:31:26.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.545+0000 7f8cf8ff9700 1 -- 172.21.15.39:0/2119588763 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cec023590 con 0x7f8cfc07ad60 2024-05-24T11:31:26.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.545+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc19d080 con 0x7f8cfc07ad60 2024-05-24T11:31:26.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.546+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc10c9b0 con 0x7f8cfc07ad60 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.546+0000 7f8cf8ff9700 1 -- 172.21.15.39:0/2119588763 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f8cec01b5b0 con 0x7f8cfc07ad60 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.546+0000 7f8cf8ff9700 1 --2- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f8ce4038750 0x7f8ce403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.546+0000 7f8cf8ff9700 1 -- 172.21.15.39:0/2119588763 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8cec04c690 con 0x7f8cfc07ad60 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.546+0000 7f8cfaffd700 1 --2- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f8ce4038750 0x7f8ce403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.547+0000 7f8cfaffd700 1 --2- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f8ce4038750 0x7f8ce403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0006fd0 tx=0x7f8cf0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.551+0000 7f8cf8ff9700 1 -- 172.21.15.39:0/2119588763 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8cec01b920 con 0x7f8cfc07ad60 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.757+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cfc102e40 con 0x7f8ce4038750 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.758+0000 7f8cf8ff9700 1 -- 172.21.15.39:0/2119588763 <== mgr.14118 v2:172.21.15.39:6800/3960296835 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f8cfc102e40 con 0x7f8ce4038750 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f8ce4038750 msgr2=0x7f8ce403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 --2- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f8ce4038750 0x7f8ce403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0006fd0 tx=0x7f8cf0006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 msgr2=0x7f8cfc19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 --2- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc19c4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8cec003f20 tx=0x7f8cec011f00 comp rx=0 tx=0).stop 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 shutdown_connections 2024-05-24T11:31:26.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 --2- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f8ce4038750 0x7f8ce403ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 --2- 172.21.15.39:0/2119588763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cfc07ad60 0x7f8cfc19c4a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 >> 172.21.15.39:0/2119588763 conn(0x7f8cfc101c20 msgr2=0x7f8cfc102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 shutdown_connections 2024-05-24T11:31:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:25.760+0000 7f8d01ec9700 1 -- 172.21.15.39:0/2119588763 wait complete. 2024-05-24T11:31:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-24T11:31:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:Adding key to root@localhost authorized_keys... 2024-05-24T11:31:26.136 INFO:teuthology.orchestra.run.smithi039.stdout:Adding host smithi039... 2024-05-24T11:31:26.999 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:26 smithi039 ceph-mon[42919]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:28.325 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:27 smithi039 ceph-mon[42919]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:29.117 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:28 smithi039 ceph-mon[42919]: Deploying cephadm binary to smithi039 2024-05-24T11:31:29.549 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Added host 'smithi039' with addr '172.21.15.39' 2024-05-24T11:31:29.549 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.477+0000 7f57be6af700 1 Processor -- start 2024-05-24T11:31:29.549 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.477+0000 7f57be6af700 1 -- start start 2024-05-24T11:31:29.549 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.477+0000 7f57be6af700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8105980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:29.549 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.477+0000 7f57be6af700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8105f50 con 0x7f57b8105560 2024-05-24T11:31:29.549 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.477+0000 7f57b7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8105980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:29.549 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.477+0000 7f57b7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8105980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52072/0 (socket says 172.21.15.39:52072) 2024-05-24T11:31:29.549 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.477+0000 7f57b7fff700 1 -- 172.21.15.39:0/2123503114 learned_addr learned my addr 172.21.15.39:0/2123503114 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.478+0000 7f57b7fff700 1 -- 172.21.15.39:0/2123503114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8106760 con 0x7f57b8105560 2024-05-24T11:31:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.478+0000 7f57b7fff700 1 --2- 172.21.15.39:0/2123503114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8105980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57a80098a0 tx=0x7f57a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16fe73c8db096c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.478+0000 7f57b6ffd700 1 -- 172.21.15.39:0/2123503114 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a8004160 con 0x7f57b8105560 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.478+0000 7f57b6ffd700 1 -- 172.21.15.39:0/2123503114 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f57a80042c0 con 0x7f57b8105560 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.478+0000 7f57b6ffd700 1 -- 172.21.15.39:0/2123503114 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a8011940 con 0x7f57b8105560 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.478+0000 7f57be6af700 1 -- 172.21.15.39:0/2123503114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 msgr2=0x7f57b8105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.478+0000 7f57be6af700 1 --2- 172.21.15.39:0/2123503114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8105980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57a80098a0 tx=0x7f57a8009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.478+0000 7f57be6af700 1 -- 172.21.15.39:0/2123503114 shutdown_connections 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.478+0000 7f57be6af700 1 --2- 172.21.15.39:0/2123503114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8105980 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.479+0000 7f57be6af700 1 -- 172.21.15.39:0/2123503114 >> 172.21.15.39:0/2123503114 conn(0x7f57b8100b20 msgr2=0x7f57b8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.479+0000 7f57be6af700 1 -- 172.21.15.39:0/2123503114 shutdown_connections 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.479+0000 7f57be6af700 1 -- 172.21.15.39:0/2123503114 wait complete. 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.479+0000 7f57be6af700 1 Processor -- start 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.479+0000 7f57be6af700 1 -- start start 2024-05-24T11:31:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57be6af700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8190590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57be6af700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8190ad0 con 0x7f57b8105560 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57b7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8190590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57b7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8190590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52078/0 (socket says 172.21.15.39:52078) 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57b7fff700 1 -- 172.21.15.39:0/3630128015 learned_addr learned my addr 172.21.15.39:0/3630128015 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57b7fff700 1 -- 172.21.15.39:0/3630128015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a8009550 con 0x7f57b8105560 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57b7fff700 1 --2- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8190590 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f57a8003f20 tx=0x7f57a8011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57b57fa700 1 -- 172.21.15.39:0/3630128015 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a80036a0 con 0x7f57b8105560 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57b57fa700 1 -- 172.21.15.39:0/3630128015 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f57a801b440 con 0x7f57b8105560 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b818ed10 con 0x7f57b8105560 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57b57fa700 1 -- 172.21.15.39:0/3630128015 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a8023400 con 0x7f57b8105560 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.480+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b818f1b0 con 0x7f57b8105560 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.481+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b818fbc0 con 0x7f57b8105560 2024-05-24T11:31:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.481+0000 7f57b57fa700 1 -- 172.21.15.39:0/3630128015 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f57a801b5b0 con 0x7f57b8105560 2024-05-24T11:31:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.482+0000 7f57b57fa700 1 --2- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f57a4038750 0x7f57a403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.482+0000 7f57b57fa700 1 -- 172.21.15.39:0/3630128015 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f57a804c640 con 0x7f57b8105560 2024-05-24T11:31:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.482+0000 7f57b77fe700 1 --2- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f57a4038750 0x7f57a403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.482+0000 7f57b77fe700 1 --2- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f57a4038750 0x7f57a403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57a0006fd0 tx=0x7f57a0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.486+0000 7f57b57fa700 1 -- 172.21.15.39:0/3630128015 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f57a801b920 con 0x7f57b8105560 2024-05-24T11:31:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:26.693+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}) v1 -- 0x7f57b8194170 con 0x7f57a4038750 2024-05-24T11:31:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.947+0000 7f57b57fa700 1 -- 172.21.15.39:0/3630128015 <== mgr.14118 v2:172.21.15.39:6800/3960296835 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f57b8194170 con 0x7f57a4038750 2024-05-24T11:31:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.949+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f57a4038750 msgr2=0x7f57a403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.949+0000 7f57be6af700 1 --2- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f57a4038750 0x7f57a403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57a0006fd0 tx=0x7f57a0006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.950+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 msgr2=0x7f57b8190590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.950+0000 7f57be6af700 1 --2- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8190590 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f57a8003f20 tx=0x7f57a8011f00 comp rx=0 tx=0).stop 2024-05-24T11:31:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.950+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 shutdown_connections 2024-05-24T11:31:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.950+0000 7f57be6af700 1 --2- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f57a4038750 0x7f57a403ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.950+0000 7f57be6af700 1 --2- 172.21.15.39:0/3630128015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57b8105560 0x7f57b8190590 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.950+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 >> 172.21.15.39:0/3630128015 conn(0x7f57b8100b20 msgr2=0x7f57b81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.950+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 shutdown_connections 2024-05-24T11:31:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:28.950+0000 7f57be6af700 1 -- 172.21.15.39:0/3630128015 wait complete. 2024-05-24T11:31:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mon service... 2024-05-24T11:31:30.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:29 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:30.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:29 smithi039 ceph-mon[42919]: Added host smithi039 2024-05-24T11:31:30.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:29 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:31:30.632 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.028+0000 7f3d8165e700 1 Processor -- start 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.028+0000 7f3d8165e700 1 -- start start 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.028+0000 7f3d8165e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.028+0000 7f3d8165e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1070a0 con 0x7f3d7c1066b0 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.028+0000 7f3d7affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.028+0000 7f3d7affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38104/0 (socket says 172.21.15.39:38104) 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.028+0000 7f3d7affd700 1 -- 172.21.15.39:0/2698079383 learned_addr learned my addr 172.21.15.39:0/2698079383 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.028+0000 7f3d7affd700 1 -- 172.21.15.39:0/2698079383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c107860 con 0x7f3d7c1066b0 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d7affd700 1 --2- 172.21.15.39:0/2698079383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c106ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0098a0 tx=0x7f3d6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cedfeb26621b8e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d79ffb700 1 -- 172.21.15.39:0/2698079383 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6c004160 con 0x7f3d7c1066b0 2024-05-24T11:31:30.633 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d79ffb700 1 -- 172.21.15.39:0/2698079383 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3d6c0042c0 con 0x7f3d7c1066b0 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d79ffb700 1 -- 172.21.15.39:0/2698079383 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6c011940 con 0x7f3d7c1066b0 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d8165e700 1 -- 172.21.15.39:0/2698079383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 msgr2=0x7f3d7c106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d8165e700 1 --2- 172.21.15.39:0/2698079383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c106ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0098a0 tx=0x7f3d6c009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d8165e700 1 -- 172.21.15.39:0/2698079383 shutdown_connections 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d8165e700 1 --2- 172.21.15.39:0/2698079383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c106ad0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d8165e700 1 -- 172.21.15.39:0/2698079383 >> 172.21.15.39:0/2698079383 conn(0x7f3d7c101c50 msgr2=0x7f3d7c104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d8165e700 1 -- 172.21.15.39:0/2698079383 shutdown_connections 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.029+0000 7f3d8165e700 1 -- 172.21.15.39:0/2698079383 wait complete. 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.030+0000 7f3d8165e700 1 Processor -- start 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.030+0000 7f3d8165e700 1 -- start start 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.030+0000 7f3d8165e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c07bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.030+0000 7f3d8165e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c07c070 con 0x7f3d7c1066b0 2024-05-24T11:31:30.634 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d7affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c07bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d7affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c07bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38106/0 (socket says 172.21.15.39:38106) 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d7affd700 1 -- 172.21.15.39:0/4173501837 learned_addr learned my addr 172.21.15.39:0/4173501837 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d7affd700 1 -- 172.21.15.39:0/4173501837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c009550 con 0x7f3d7c1066b0 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d7affd700 1 --2- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c07bb30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c003f20 tx=0x7f3d6c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d73fff700 1 -- 172.21.15.39:0/4173501837 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6c0036a0 con 0x7f3d7c1066b0 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c07a2b0 con 0x7f3d7c1066b0 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d73fff700 1 -- 172.21.15.39:0/4173501837 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3d6c003c70 con 0x7f3d7c1066b0 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d73fff700 1 -- 172.21.15.39:0/4173501837 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d6c01be80 con 0x7f3d7c1066b0 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.031+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c07a750 con 0x7f3d7c1066b0 2024-05-24T11:31:30.635 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.032+0000 7f3d73fff700 1 -- 172.21.15.39:0/4173501837 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3d6c0234e0 con 0x7f3d7c1066b0 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.032+0000 7f3d73fff700 1 --2- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f3d68038700 0x7f3d6803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.032+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c0625d0 con 0x7f3d7c1066b0 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.032+0000 7f3d73fff700 1 -- 172.21.15.39:0/4173501837 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3d6c04c870 con 0x7f3d7c1066b0 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.032+0000 7f3d7a7fc700 1 --2- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f3d68038700 0x7f3d6803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.033+0000 7f3d7a7fc700 1 --2- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f3d68038700 0x7f3d6803abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d64006fd0 tx=0x7f3d64006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.037+0000 7f3d73fff700 1 -- 172.21.15.39:0/4173501837 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3d6c00bc70 con 0x7f3d7c1066b0 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.164+0000 7f3d73fff700 1 -- 172.21.15.39:0/4173501837 <== mon.0 v2:172.21.15.39:3300/0 7 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d6c00bc70 con 0x7f3d7c1066b0 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.259+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3d60000d90 con 0x7f3d68038700 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.268+0000 7f3d73fff700 1 -- 172.21.15.39:0/4173501837 <== mgr.14118 v2:172.21.15.39:6800/3960296835 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3d60000d90 con 0x7f3d68038700 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f3d68038700 msgr2=0x7f3d6803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 --2- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f3d68038700 0x7f3d6803abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d64006fd0 tx=0x7f3d64006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 msgr2=0x7f3d7c07bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:30.636 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 --2- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c07bb30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c003f20 tx=0x7f3d6c011f00 comp rx=0 tx=0).stop 2024-05-24T11:31:30.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 shutdown_connections 2024-05-24T11:31:30.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 --2- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f3d68038700 0x7f3d6803abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:30.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 --2- 172.21.15.39:0/4173501837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d7c1066b0 0x7f3d7c07bb30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:30.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 >> 172.21.15.39:0/4173501837 conn(0x7f3d7c101c50 msgr2=0x7f3d7c1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:30.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 shutdown_connections 2024-05-24T11:31:30.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:30.270+0000 7f3d8165e700 1 -- 172.21.15.39:0/4173501837 wait complete. 2024-05-24T11:31:30.637 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mgr service... 2024-05-24T11:31:31.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:31 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:31.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:31 smithi039 ceph-mon[42919]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:31.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:31 smithi039 ceph-mon[42919]: Saving service mon spec with placement count:5 2024-05-24T11:31:31.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:31 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:31.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:31 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:31.964 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-24T11:31:31.964 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.367+0000 7f36ca84d700 1 Processor -- start 2024-05-24T11:31:31.964 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.367+0000 7f36ca84d700 1 -- start start 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.367+0000 7f36ca84d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.367+0000 7f36ca84d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4079de0 con 0x7f36c41071b0 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.368+0000 7f36c3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.368+0000 7f36c3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38110/0 (socket says 172.21.15.39:38110) 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.368+0000 7f36c3fff700 1 -- 172.21.15.39:0/1994436478 learned_addr learned my addr 172.21.15.39:0/1994436478 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.368+0000 7f36c3fff700 1 -- 172.21.15.39:0/1994436478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c4079f20 con 0x7f36c41071b0 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.368+0000 7f36c3fff700 1 --2- 172.21.15.39:0/1994436478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41095a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36b40098a0 tx=0x7f36b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33cdcc6b4950fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.368+0000 7f36c2ffd700 1 -- 172.21.15.39:0/1994436478 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36b4004160 con 0x7f36c41071b0 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.368+0000 7f36c2ffd700 1 -- 172.21.15.39:0/1994436478 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36b40042c0 con 0x7f36c41071b0 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.368+0000 7f36c2ffd700 1 -- 172.21.15.39:0/1994436478 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36b40118e0 con 0x7f36c41071b0 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.369+0000 7f36ca84d700 1 -- 172.21.15.39:0/1994436478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 msgr2=0x7f36c41095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.369+0000 7f36ca84d700 1 --2- 172.21.15.39:0/1994436478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41095a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36b40098a0 tx=0x7f36b4009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:31.965 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.369+0000 7f36ca84d700 1 -- 172.21.15.39:0/1994436478 shutdown_connections 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.369+0000 7f36ca84d700 1 --2- 172.21.15.39:0/1994436478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41095a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.369+0000 7f36ca84d700 1 -- 172.21.15.39:0/1994436478 >> 172.21.15.39:0/1994436478 conn(0x7f36c4100b80 msgr2=0x7f36c4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.369+0000 7f36ca84d700 1 -- 172.21.15.39:0/1994436478 shutdown_connections 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.369+0000 7f36ca84d700 1 -- 172.21.15.39:0/1994436478 wait complete. 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.370+0000 7f36ca84d700 1 Processor -- start 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.370+0000 7f36ca84d700 1 -- start start 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.370+0000 7f36ca84d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41069e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.370+0000 7f36ca84d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4106f20 con 0x7f36c41071b0 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.370+0000 7f36c3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.370+0000 7f36c3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38114/0 (socket says 172.21.15.39:38114) 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.370+0000 7f36c3fff700 1 -- 172.21.15.39:0/2673743330 learned_addr learned my addr 172.21.15.39:0/2673743330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:31.966 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.370+0000 7f36c3fff700 1 -- 172.21.15.39:0/2673743330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4009550 con 0x7f36c41071b0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.370+0000 7f36c3fff700 1 --2- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41069e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36b4003e30 tx=0x7f36b4011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.371+0000 7f36c17fa700 1 -- 172.21.15.39:0/2673743330 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36b40036a0 con 0x7f36c41071b0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.371+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c4105160 con 0x7f36c41071b0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.371+0000 7f36c17fa700 1 -- 172.21.15.39:0/2673743330 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36b401b440 con 0x7f36c41071b0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.371+0000 7f36c17fa700 1 -- 172.21.15.39:0/2673743330 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36b40193d0 con 0x7f36c41071b0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.371+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c4105600 con 0x7f36c41071b0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.371+0000 7f36c17fa700 1 -- 172.21.15.39:0/2673743330 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f36b4019690 con 0x7f36c41071b0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.372+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c410a7f0 con 0x7f36c41071b0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.372+0000 7f36c17fa700 1 --2- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f36ac0386b0 0x7f36ac03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.372+0000 7f36c17fa700 1 -- 172.21.15.39:0/2673743330 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f36b404c530 con 0x7f36c41071b0 2024-05-24T11:31:31.967 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.372+0000 7f36c37fe700 1 --2- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f36ac0386b0 0x7f36ac03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:31.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.372+0000 7f36c37fe700 1 --2- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f36ac0386b0 0x7f36ac03ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f36b8006fd0 tx=0x7f36b8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:31.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.377+0000 7f36c17fa700 1 -- 172.21.15.39:0/2673743330 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f36b4018db0 con 0x7f36c41071b0 2024-05-24T11:31:31.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.597+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f36c4105db0 con 0x7f36ac0386b0 2024-05-24T11:31:31.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.602+0000 7f36c17fa700 1 -- 172.21.15.39:0/2673743330 <== mgr.14118 v2:172.21.15.39:6800/3960296835 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f36c4105db0 con 0x7f36ac0386b0 2024-05-24T11:31:31.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f36ac0386b0 msgr2=0x7f36ac03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:31.968 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 --2- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f36ac0386b0 0x7f36ac03ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f36b8006fd0 tx=0x7f36b8006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:31.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 msgr2=0x7f36c41069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:31.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 --2- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41069e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36b4003e30 tx=0x7f36b4011e80 comp rx=0 tx=0).stop 2024-05-24T11:31:31.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 shutdown_connections 2024-05-24T11:31:31.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 --2- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f36ac0386b0 0x7f36ac03ab70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:31.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 --2- 172.21.15.39:0/2673743330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36c41071b0 0x7f36c41069e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:31.969 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 >> 172.21.15.39:0/2673743330 conn(0x7f36c4100b80 msgr2=0x7f36c41017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:31.970 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 shutdown_connections 2024-05-24T11:31:31.970 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:31.604+0000 7f36ca84d700 1 -- 172.21.15.39:0/2673743330 wait complete. 2024-05-24T11:31:32.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:32 smithi039 ceph-mon[42919]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:32.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:32 smithi039 ceph-mon[42919]: Saving service mgr spec with placement count:2 2024-05-24T11:31:32.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:32 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:32.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:32 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:32.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:32 smithi039 ceph-mon[42919]: from='mgr.14118 172.21.15.39:0/1377388255' entity='mgr.x' 2024-05-24T11:31:32.817 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:32 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1982754473' entity='client.admin' 2024-05-24T11:31:32.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.308+0000 7f65a40dd700 1 Processor -- start 2024-05-24T11:31:32.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.308+0000 7f65a40dd700 1 -- start start 2024-05-24T11:31:32.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.309+0000 7f65a40dd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c105980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:32.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.309+0000 7f65a40dd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c105f50 con 0x7f659c105560 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.309+0000 7f65a1e79700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c105980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.309+0000 7f65a1e79700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c105980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38116/0 (socket says 172.21.15.39:38116) 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.309+0000 7f65a1e79700 1 -- 172.21.15.39:0/530717737 learned_addr learned my addr 172.21.15.39:0/530717737 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.309+0000 7f65a1e79700 1 -- 172.21.15.39:0/530717737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c106760 con 0x7f659c105560 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.309+0000 7f65a1e79700 1 --2- 172.21.15.39:0/530717737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c105980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f658c0098a0 tx=0x7f658c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06441726a7b96c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.310+0000 7f65a0e77700 1 -- 172.21.15.39:0/530717737 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f658c004160 con 0x7f659c105560 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.310+0000 7f65a0e77700 1 -- 172.21.15.39:0/530717737 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f658c0042c0 con 0x7f659c105560 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.310+0000 7f65a0e77700 1 -- 172.21.15.39:0/530717737 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f658c0118e0 con 0x7f659c105560 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.310+0000 7f65a40dd700 1 -- 172.21.15.39:0/530717737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 msgr2=0x7f659c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.310+0000 7f65a40dd700 1 --2- 172.21.15.39:0/530717737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c105980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f658c0098a0 tx=0x7f658c009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.310+0000 7f65a40dd700 1 -- 172.21.15.39:0/530717737 shutdown_connections 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.310+0000 7f65a40dd700 1 --2- 172.21.15.39:0/530717737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c105980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.310+0000 7f65a40dd700 1 -- 172.21.15.39:0/530717737 >> 172.21.15.39:0/530717737 conn(0x7f659c100b80 msgr2=0x7f659c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:32.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.310+0000 7f65a40dd700 1 -- 172.21.15.39:0/530717737 shutdown_connections 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.311+0000 7f65a40dd700 1 -- 172.21.15.39:0/530717737 wait complete. 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.311+0000 7f65a40dd700 1 Processor -- start 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.311+0000 7f65a40dd700 1 -- start start 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.311+0000 7f65a40dd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.311+0000 7f65a40dd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c19a920 con 0x7f659c105560 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.311+0000 7f65a1e79700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.311+0000 7f65a1e79700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38122/0 (socket says 172.21.15.39:38122) 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.311+0000 7f65a1e79700 1 -- 172.21.15.39:0/1982754473 learned_addr learned my addr 172.21.15.39:0/1982754473 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.312+0000 7f65a1e79700 1 -- 172.21.15.39:0/1982754473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c009550 con 0x7f659c105560 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.312+0000 7f65a1e79700 1 --2- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c19a3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f658c003f70 tx=0x7f658c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.312+0000 7f6592ffd700 1 -- 172.21.15.39:0/1982754473 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f658c0037d0 con 0x7f659c105560 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.312+0000 7f6592ffd700 1 -- 172.21.15.39:0/1982754473 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f658c01b440 con 0x7f659c105560 2024-05-24T11:31:32.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.312+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659c19ab20 con 0x7f659c105560 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.312+0000 7f6592ffd700 1 -- 172.21.15.39:0/1982754473 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f658c0193d0 con 0x7f659c105560 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.312+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659c19afc0 con 0x7f659c105560 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.313+0000 7f6592ffd700 1 -- 172.21.15.39:0/1982754473 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f658c01b860 con 0x7f659c105560 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.313+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659c194540 con 0x7f659c105560 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.313+0000 7f6592ffd700 1 --2- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f6588038750 0x7f658803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.313+0000 7f6592ffd700 1 -- 172.21.15.39:0/1982754473 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f658c04c550 con 0x7f659c105560 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.313+0000 7f65a1678700 1 --2- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f6588038750 0x7f658803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.314+0000 7f65a1678700 1 --2- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f6588038750 0x7f658803ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6598006fd0 tx=0x7f6598006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.318+0000 7f6592ffd700 1 -- 172.21.15.39:0/1982754473 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f658c018920 con 0x7f659c105560 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.522+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f659c02cee0 con 0x7f659c105560 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.528+0000 7f6592ffd700 1 -- 172.21.15.39:0/1982754473 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f658c01f0d0 con 0x7f659c105560 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f6588038750 msgr2=0x7f658803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:32.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 --2- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f6588038750 0x7f658803ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6598006fd0 tx=0x7f6598006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:32.951 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 msgr2=0x7f659c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:32.951 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 --2- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c19a3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f658c003f70 tx=0x7f658c0119a0 comp rx=0 tx=0).stop 2024-05-24T11:31:32.951 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 shutdown_connections 2024-05-24T11:31:32.951 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 --2- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f6588038750 0x7f658803ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:32.951 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 --2- 172.21.15.39:0/1982754473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f659c105560 0x7f659c19a3e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:32.951 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 >> 172.21.15.39:0/1982754473 conn(0x7f659c100b80 msgr2=0x7f659c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:32.951 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 shutdown_connections 2024-05-24T11:31:32.951 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:32.530+0000 7f65a40dd700 1 -- 172.21.15.39:0/1982754473 wait complete. 2024-05-24T11:31:34.080 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.495+0000 7f68f8e0e700 1 Processor -- start 2024-05-24T11:31:34.080 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.495+0000 7f68f8e0e700 1 -- start start 2024-05-24T11:31:34.081 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.495+0000 7f68f8e0e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f4106a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:34.081 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.495+0000 7f68f8e0e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f4107040 con 0x7f68f4106650 2024-05-24T11:31:34.081 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.495+0000 7f68f259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f4106a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:34.081 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.495+0000 7f68f259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f4106a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38134/0 (socket says 172.21.15.39:38134) 2024-05-24T11:31:34.081 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.495+0000 7f68f259c700 1 -- 172.21.15.39:0/186538442 learned_addr learned my addr 172.21.15.39:0/186538442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:34.081 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.495+0000 7f68f259c700 1 -- 172.21.15.39:0/186538442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f4107850 con 0x7f68f4106650 2024-05-24T11:31:34.081 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f259c700 1 --2- 172.21.15.39:0/186538442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f4106a70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0098a0 tx=0x7f68ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5ac562b9401f00 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:34.081 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f159a700 1 -- 172.21.15.39:0/186538442 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68ec004160 con 0x7f68f4106650 2024-05-24T11:31:34.081 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f159a700 1 -- 172.21.15.39:0/186538442 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f68ec0042c0 con 0x7f68f4106650 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f159a700 1 -- 172.21.15.39:0/186538442 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68ec0118e0 con 0x7f68f4106650 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f8e0e700 1 -- 172.21.15.39:0/186538442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 msgr2=0x7f68f4106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f8e0e700 1 --2- 172.21.15.39:0/186538442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f4106a70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0098a0 tx=0x7f68ec009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f8e0e700 1 -- 172.21.15.39:0/186538442 shutdown_connections 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f8e0e700 1 --2- 172.21.15.39:0/186538442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f4106a70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f8e0e700 1 -- 172.21.15.39:0/186538442 >> 172.21.15.39:0/186538442 conn(0x7f68f4101c30 msgr2=0x7f68f4104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f8e0e700 1 -- 172.21.15.39:0/186538442 shutdown_connections 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.496+0000 7f68f8e0e700 1 -- 172.21.15.39:0/186538442 wait complete. 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.497+0000 7f68f8e0e700 1 Processor -- start 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.497+0000 7f68f8e0e700 1 -- start start 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.497+0000 7f68f8e0e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:34.082 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.497+0000 7f68f8e0e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f419a8e0 con 0x7f68f4106650 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68f259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68f259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38144/0 (socket says 172.21.15.39:38144) 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68f259c700 1 -- 172.21.15.39:0/2127501330 learned_addr learned my addr 172.21.15.39:0/2127501330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68f259c700 1 -- 172.21.15.39:0/2127501330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec009550 con 0x7f68f4106650 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68f259c700 1 --2- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f419a3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f68ec003f70 tx=0x7f68ec0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68eb7fe700 1 -- 172.21.15.39:0/2127501330 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68ec0036a0 con 0x7f68f4106650 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68eb7fe700 1 -- 172.21.15.39:0/2127501330 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f68ec01b440 con 0x7f68f4106650 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f419aae0 con 0x7f68f4106650 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68eb7fe700 1 -- 172.21.15.39:0/2127501330 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68ec023400 con 0x7f68f4106650 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.498+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f419af80 con 0x7f68f4106650 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.499+0000 7f68eb7fe700 1 -- 172.21.15.39:0/2127501330 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f68ec023680 con 0x7f68f4106650 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.499+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f410aa80 con 0x7f68f4106650 2024-05-24T11:31:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.499+0000 7f68eb7fe700 1 --2- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f68e0038700 0x7f68e003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.499+0000 7f68eb7fe700 1 -- 172.21.15.39:0/2127501330 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f68ec04c900 con 0x7f68f4106650 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.499+0000 7f68f1d9b700 1 --2- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f68e0038700 0x7f68e003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.500+0000 7f68f1d9b700 1 --2- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f68e0038700 0x7f68e003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f68dc006fd0 tx=0x7f68dc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.505+0000 7f68eb7fe700 1 -- 172.21.15.39:0/2127501330 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f68ec018350 con 0x7f68f4106650 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.708+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f68f419d7c0 con 0x7f68f4106650 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.713+0000 7f68eb7fe700 1 -- 172.21.15.39:0/2127501330 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f68ec02bb40 con 0x7f68f4106650 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f68e0038700 msgr2=0x7f68e003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 --2- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f68e0038700 0x7f68e003abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f68dc006fd0 tx=0x7f68dc006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 msgr2=0x7f68f419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:34.084 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 --2- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f419a3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f68ec003f70 tx=0x7f68ec0119a0 comp rx=0 tx=0).stop 2024-05-24T11:31:34.085 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 shutdown_connections 2024-05-24T11:31:34.085 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 --2- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f68e0038700 0x7f68e003abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:34.085 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 --2- 172.21.15.39:0/2127501330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68f4106650 0x7f68f419a3a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:34.085 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 >> 172.21.15.39:0/2127501330 conn(0x7f68f4101c30 msgr2=0x7f68f4102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:34.085 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 shutdown_connections 2024-05-24T11:31:34.085 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:33.715+0000 7f68f8e0e700 1 -- 172.21.15.39:0/2127501330 wait complete. 2024-05-24T11:31:34.085 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling the dashboard module... 2024-05-24T11:31:35.063 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:34 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2127501330' entity='client.admin' 2024-05-24T11:31:35.063 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:34 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4221043993' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-24T11:31:35.063 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:34 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: ignoring --setuser ceph since I am not root 2024-05-24T11:31:35.063 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:34 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: ignoring --setgroup ceph since I am not root 2024-05-24T11:31:35.147 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.416+0000 7f78a408d700 1 Processor -- start 2024-05-24T11:31:35.147 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.416+0000 7f78a408d700 1 -- start start 2024-05-24T11:31:35.147 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.416+0000 7f78a408d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c105370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:35.147 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.416+0000 7f78a408d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c079e40 con 0x7f789c104f50 2024-05-24T11:31:35.147 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.416+0000 7f78a1e29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c105370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:35.147 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.416+0000 7f78a1e29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c105370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38150/0 (socket says 172.21.15.39:38150) 2024-05-24T11:31:35.147 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.416+0000 7f78a1e29700 1 -- 172.21.15.39:0/2000126333 learned_addr learned my addr 172.21.15.39:0/2000126333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.416+0000 7f78a1e29700 1 -- 172.21.15.39:0/2000126333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c1058b0 con 0x7f789c104f50 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.417+0000 7f78a1e29700 1 --2- 172.21.15.39:0/2000126333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c105370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78980098a0 tx=0x7f7898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a3dac18650a36b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.417+0000 7f78a0e27700 1 -- 172.21.15.39:0/2000126333 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7898004160 con 0x7f789c104f50 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.417+0000 7f78a0e27700 1 -- 172.21.15.39:0/2000126333 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78980042c0 con 0x7f789c104f50 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.417+0000 7f78a0e27700 1 -- 172.21.15.39:0/2000126333 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78980118e0 con 0x7f789c104f50 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.417+0000 7f78a408d700 1 -- 172.21.15.39:0/2000126333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 msgr2=0x7f789c105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.417+0000 7f78a408d700 1 --2- 172.21.15.39:0/2000126333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c105370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78980098a0 tx=0x7f7898009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.417+0000 7f78a408d700 1 -- 172.21.15.39:0/2000126333 shutdown_connections 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.417+0000 7f78a408d700 1 --2- 172.21.15.39:0/2000126333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c105370 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.417+0000 7f78a408d700 1 -- 172.21.15.39:0/2000126333 >> 172.21.15.39:0/2000126333 conn(0x7f789c100b90 msgr2=0x7f789c102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.418+0000 7f78a408d700 1 -- 172.21.15.39:0/2000126333 shutdown_connections 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.418+0000 7f78a408d700 1 -- 172.21.15.39:0/2000126333 wait complete. 2024-05-24T11:31:35.148 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.418+0000 7f78a408d700 1 Processor -- start 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.418+0000 7f78a408d700 1 -- start start 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.418+0000 7f78a408d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.418+0000 7f78a408d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c19ca90 con 0x7f789c104f50 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.418+0000 7f78a1e29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.418+0000 7f78a1e29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38166/0 (socket says 172.21.15.39:38166) 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.418+0000 7f78a1e29700 1 -- 172.21.15.39:0/4221043993 learned_addr learned my addr 172.21.15.39:0/4221043993 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.419+0000 7f78a1e29700 1 -- 172.21.15.39:0/4221043993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898009550 con 0x7f789c104f50 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.419+0000 7f78a1e29700 1 --2- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c19c550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7898003e30 tx=0x7f7898011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.419+0000 7f7892ffd700 1 -- 172.21.15.39:0/4221043993 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78980036a0 con 0x7f789c104f50 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.419+0000 7f7892ffd700 1 -- 172.21.15.39:0/4221043993 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f789801b440 con 0x7f789c104f50 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.419+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c19cc90 con 0x7f789c104f50 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.419+0000 7f7892ffd700 1 -- 172.21.15.39:0/4221043993 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78980193d0 con 0x7f789c104f50 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.419+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c19d130 con 0x7f789c104f50 2024-05-24T11:31:35.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.420+0000 7f7892ffd700 1 -- 172.21.15.39:0/4221043993 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f7898003800 con 0x7f789c104f50 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.420+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789c196280 con 0x7f789c104f50 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.420+0000 7f7892ffd700 1 --2- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f7888038700 0x7f788803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.420+0000 7f7892ffd700 1 -- 172.21.15.39:0/4221043993 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f789804c4e0 con 0x7f789c104f50 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.420+0000 7f78a1628700 1 --2- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f7888038700 0x7f788803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.421+0000 7f78a1628700 1 --2- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f7888038700 0x7f788803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f788c006fd0 tx=0x7f788c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.425+0000 7f7892ffd700 1 -- 172.21.15.39:0/4221043993 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f789802b960 con 0x7f789c104f50 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.686+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f789c1068d0 con 0x7f789c104f50 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.715+0000 7f7892ffd700 1 -- 172.21.15.39:0/4221043993 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f7898021dc0 con 0x7f789c104f50 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.716+0000 7f7892ffd700 1 -- 172.21.15.39:0/4221043993 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f789800fd80 con 0x7f789c104f50 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f7888038700 msgr2=0x7f788803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 --2- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f7888038700 0x7f788803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f788c006fd0 tx=0x7f788c006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 msgr2=0x7f789c19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 --2- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c19c550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7898003e30 tx=0x7f7898011e80 comp rx=0 tx=0).stop 2024-05-24T11:31:35.150 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 shutdown_connections 2024-05-24T11:31:35.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 --2- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f7888038700 0x7f788803abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:35.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 --2- 172.21.15.39:0/4221043993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f789c104f50 0x7f789c19c550 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:35.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 >> 172.21.15.39:0/4221043993 conn(0x7f789c100b90 msgr2=0x7f789c1933c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:35.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 shutdown_connections 2024-05-24T11:31:35.151 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:34.719+0000 7f78a408d700 1 -- 172.21.15.39:0/4221043993 wait complete. 2024-05-24T11:31:35.325 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:35 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:35.177+0000 7fb8f8abe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-24T11:31:35.325 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:35 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:35.279+0000 7fb8f8abe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-24T11:31:35.720 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:35 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:35.372+0000 7fb8f8abe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-24T11:31:35.720 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:35 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:35.463+0000 7fb8f8abe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-24T11:31:36.035 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:35 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4221043993' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-24T11:31:36.035 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:35 smithi039 ceph-mon[42919]: mgrmap e9: x(active, since 13s) 2024-05-24T11:31:36.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.548+0000 7f3905d69700 1 Processor -- start 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.548+0000 7f3905d69700 1 -- start start 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.548+0000 7f3905d69700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f3900105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.548+0000 7f3905d69700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900105f40 con 0x7f3900105550 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.549+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f3900105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.549+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f3900105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38182/0 (socket says 172.21.15.39:38182) 2024-05-24T11:31:36.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.549+0000 7f38ff7fe700 1 -- 172.21.15.39:0/1979838168 learned_addr learned my addr 172.21.15.39:0/1979838168 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.549+0000 7f38ff7fe700 1 -- 172.21.15.39:0/1979838168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900106750 con 0x7f3900105550 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.549+0000 7f38ff7fe700 1 --2- 172.21.15.39:0/1979838168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f3900105970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38f00098a0 tx=0x7f38f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c843422a4e5bc7e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.549+0000 7f38fe7fc700 1 -- 172.21.15.39:0/1979838168 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38f0004160 con 0x7f3900105550 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.549+0000 7f38fe7fc700 1 -- 172.21.15.39:0/1979838168 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38f00042c0 con 0x7f3900105550 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.549+0000 7f38fe7fc700 1 -- 172.21.15.39:0/1979838168 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38f00118e0 con 0x7f3900105550 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.550+0000 7f3905d69700 1 -- 172.21.15.39:0/1979838168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 msgr2=0x7f3900105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.550+0000 7f3905d69700 1 --2- 172.21.15.39:0/1979838168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f3900105970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38f00098a0 tx=0x7f38f0009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.550+0000 7f3905d69700 1 -- 172.21.15.39:0/1979838168 shutdown_connections 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.550+0000 7f3905d69700 1 --2- 172.21.15.39:0/1979838168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f3900105970 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.550+0000 7f3905d69700 1 -- 172.21.15.39:0/1979838168 >> 172.21.15.39:0/1979838168 conn(0x7f3900100b10 msgr2=0x7f3900102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.550+0000 7f3905d69700 1 -- 172.21.15.39:0/1979838168 shutdown_connections 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.550+0000 7f3905d69700 1 -- 172.21.15.39:0/1979838168 wait complete. 2024-05-24T11:31:36.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.551+0000 7f3905d69700 1 Processor -- start 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.551+0000 7f3905d69700 1 -- start start 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.551+0000 7f3905d69700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f390019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.551+0000 7f3905d69700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390019cb50 con 0x7f3900105550 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.551+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f390019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.551+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f390019c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38196/0 (socket says 172.21.15.39:38196) 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.551+0000 7f38ff7fe700 1 -- 172.21.15.39:0/578985437 learned_addr learned my addr 172.21.15.39:0/578985437 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.552+0000 7f38ff7fe700 1 -- 172.21.15.39:0/578985437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f0009550 con 0x7f3900105550 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.552+0000 7f38ff7fe700 1 --2- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f390019c610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38f0003f70 tx=0x7f38f00119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.552+0000 7f38fcff9700 1 -- 172.21.15.39:0/578985437 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38f00036a0 con 0x7f3900105550 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.552+0000 7f38fcff9700 1 -- 172.21.15.39:0/578985437 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38f001b440 con 0x7f3900105550 2024-05-24T11:31:36.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.552+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390019cd50 con 0x7f3900105550 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.552+0000 7f38fcff9700 1 -- 172.21.15.39:0/578985437 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38f0023400 con 0x7f3900105550 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.552+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390019d1f0 con 0x7f3900105550 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.553+0000 7f38fcff9700 1 -- 172.21.15.39:0/578985437 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f38f0003800 con 0x7f3900105550 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.553+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3900196930 con 0x7f3900105550 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.553+0000 7f38fcff9700 1 --2- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f38e8038750 0x7f38e803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.553+0000 7f38fcff9700 1 -- 172.21.15.39:0/578985437 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f38f004c6b0 con 0x7f3900105550 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.553+0000 7f38feffd700 1 -- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f38e8038750 msgr2=0x7f38e803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3960296835 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.553+0000 7f38feffd700 1 --2- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f38e8038750 0x7f38e803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.558+0000 7f38fcff9700 1 -- 172.21.15.39:0/578985437 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f38f002b350 con 0x7f3900105550 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.754+0000 7f38feffd700 1 -- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f38e8038750 msgr2=0x7f38e803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3960296835 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.754+0000 7f38feffd700 1 --2- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f38e8038750 0x7f38e803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.819+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3900109950 con 0x7f3900105550 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.820+0000 7f38fcff9700 1 -- 172.21.15.39:0/578985437 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f38f002b350 con 0x7f3900105550 2024-05-24T11:31:36.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f38e8038750 msgr2=0x7f38e803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 --2- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f38e8038750 0x7f38e803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 msgr2=0x7f390019c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 --2- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f390019c610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38f0003f70 tx=0x7f38f00119a0 comp rx=0 tx=0).stop 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 shutdown_connections 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 --2- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f38e8038750 0x7f38e803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 --2- 172.21.15.39:0/578985437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3900105550 0x7f390019c610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 >> 172.21.15.39:0/578985437 conn(0x7f3900100b10 msgr2=0x7f3900101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 shutdown_connections 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:35.822+0000 7f3905d69700 1 -- 172.21.15.39:0/578985437 wait complete. 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-05-24T11:31:36.210 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 9... 2024-05-24T11:31:36.325 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:36 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:36.140+0000 7fb8f8abe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:31:36.628 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:36 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:36.413+0000 7fb8f8abe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-24T11:31:36.628 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:36 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:36.506+0000 7fb8f8abe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-24T11:31:36.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:36 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/578985437' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-24T11:31:36.931 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:36 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:36.624+0000 7fb8f8abe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-24T11:31:36.931 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:36 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:36.790+0000 7fb8f8abe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-24T11:31:36.931 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:36 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:36.924+0000 7fb8f8abe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-24T11:31:37.325 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:37 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:37.014+0000 7fb8f8abe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-24T11:31:37.813 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:37 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:37.402+0000 7fb8f8abe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-24T11:31:37.814 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:37 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:37.810+0000 7fb8f8abe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-24T11:31:38.075 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:38 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:38.014+0000 7fb8f8abe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-24T11:31:38.575 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:38 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:38.112+0000 7fb8f8abe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-24T11:31:39.176 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:38 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:38.913+0000 7fb8f8abe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-24T11:31:39.176 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:39 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:39.015+0000 7fb8f8abe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-24T11:31:39.176 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:39 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:39.172+0000 7fb8f8abe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-24T11:31:40.230 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:39 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:39.832+0000 7fb8f8abe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-24T11:31:40.231 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:40 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:40.023+0000 7fb8f8abe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-24T11:31:40.231 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:40 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:40.227+0000 7fb8f8abe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-24T11:31:41.509 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:41 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:41.203+0000 7fb8f8abe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-24T11:31:41.803 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:41 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:41.506+0000 7fb8f8abe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:31:41.803 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:41 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:41.599+0000 7fb8f8abe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-24T11:31:41.803 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:31:41 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:31:41.800+0000 7fb8f8abe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-24T11:31:42.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: Active manager daemon x restarted 2024-05-24T11:31:42.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: Activating manager daemon x 2024-05-24T11:31:42.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: osdmap e3: 0 total, 0 up, 0 in 2024-05-24T11:31:42.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: mgrmap e10: x(active, starting, since 0.00508762s) 2024-05-24T11:31:42.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:31:42.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-24T11:31:42.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-24T11:31:42.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-24T11:31:42.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-24T11:31:42.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: Manager daemon x is now available 2024-05-24T11:31:42.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:31:43.118 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-24T11:31:43.118 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-24T11:31:43.118 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:42 smithi039 ceph-mon[42919]: mgrmap e11: x(active, since 1.00577s) 2024-05-24T11:31:43.119 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:43.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-24T11:31:43.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-24T11:31:43.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.584+0000 7f75c7fdc700 1 Processor -- start 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.584+0000 7f75c7fdc700 1 -- start start 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.584+0000 7f75c7fdc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c0105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.584+0000 7f75c7fdc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c0105f40 con 0x7f75c0105550 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.584+0000 7f75c5d78700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c0105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.584+0000 7f75c5d78700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c0105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38202/0 (socket says 172.21.15.39:38202) 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.584+0000 7f75c5d78700 1 -- 172.21.15.39:0/1250851070 learned_addr learned my addr 172.21.15.39:0/1250851070 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.585+0000 7f75c5d78700 1 -- 172.21.15.39:0/1250851070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c0106750 con 0x7f75c0105550 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.585+0000 7f75c5d78700 1 --2- 172.21.15.39:0/1250851070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c0105970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00bb40 tx=0x7f75bc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f86727749ac906 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.585+0000 7f75c4d76700 1 -- 172.21.15.39:0/1250851070 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75bc00dbe0 con 0x7f75c0105550 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.585+0000 7f75c4d76700 1 -- 172.21.15.39:0/1250851070 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75bc0044c0 con 0x7f75c0105550 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.585+0000 7f75c4d76700 1 -- 172.21.15.39:0/1250851070 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75bc003d50 con 0x7f75c0105550 2024-05-24T11:31:43.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.586+0000 7f75c7fdc700 1 -- 172.21.15.39:0/1250851070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 msgr2=0x7f75c0105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.586+0000 7f75c7fdc700 1 --2- 172.21.15.39:0/1250851070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c0105970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00bb40 tx=0x7f75bc00be50 comp rx=0 tx=0).stop 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.586+0000 7f75c7fdc700 1 -- 172.21.15.39:0/1250851070 shutdown_connections 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.586+0000 7f75c7fdc700 1 --2- 172.21.15.39:0/1250851070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c0105970 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.586+0000 7f75c7fdc700 1 -- 172.21.15.39:0/1250851070 >> 172.21.15.39:0/1250851070 conn(0x7f75c0100b10 msgr2=0x7f75c0102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.586+0000 7f75c7fdc700 1 -- 172.21.15.39:0/1250851070 shutdown_connections 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.586+0000 7f75c7fdc700 1 -- 172.21.15.39:0/1250851070 wait complete. 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.586+0000 7f75c7fdc700 1 Processor -- start 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.586+0000 7f75c7fdc700 1 -- start start 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75c7fdc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75c7fdc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c019cb00 con 0x7f75c0105550 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75c5d78700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:43.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75c5d78700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38210/0 (socket says 172.21.15.39:38210) 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75c5d78700 1 -- 172.21.15.39:0/974299423 learned_addr learned my addr 172.21.15.39:0/974299423 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75c5d78700 1 -- 172.21.15.39:0/974299423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc00b7f0 con 0x7f75c0105550 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75c5d78700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c019c5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00b340 tx=0x7f75bc004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75bc0106d0 con 0x7f75c0105550 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75bc010cc0 con 0x7f75c0105550 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75bc021ba0 con 0x7f75c0105550 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75c7fdc700 1 -- 172.21.15.39:0/974299423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c019cd00 con 0x7f75c0105550 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.587+0000 7f75c7fdc700 1 -- 172.21.15.39:0/974299423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c019d140 con 0x7f75c0105550 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.588+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f75bc010830 con 0x7f75c0105550 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.588+0000 7f75b6ffd700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 0x7f75ac03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:43.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.588+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 --> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75ac03b2d0 con 0x7f75ac038700 2024-05-24T11:31:43.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.589+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f75bc04cce0 con 0x7f75c0105550 2024-05-24T11:31:43.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.589+0000 7f75c5577700 1 -- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 msgr2=0x7f75ac03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3960296835 2024-05-24T11:31:43.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.589+0000 7f75c5577700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 0x7f75ac03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:31:43.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.790+0000 7f75c5577700 1 -- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 msgr2=0x7f75ac03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3960296835 2024-05-24T11:31:43.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:36.790+0000 7f75c5577700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 0x7f75ac03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-24T11:31:43.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:37.191+0000 7f75c5577700 1 -- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 msgr2=0x7f75ac03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3960296835 2024-05-24T11:31:43.233 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:37.191+0000 7f75c5577700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 0x7f75ac03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-24T11:31:43.233 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:37.992+0000 7f75c5577700 1 -- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 msgr2=0x7f75ac03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3960296835 2024-05-24T11:31:43.233 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:37.992+0000 7f75c5577700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 0x7f75ac03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-24T11:31:43.233 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:39.593+0000 7f75c5577700 1 -- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 msgr2=0x7f75ac03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3960296835 2024-05-24T11:31:43.233 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:39.593+0000 7f75c5577700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 0x7f75ac03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-24T11:31:43.233 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:41.817+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7f75bc01ed60 con 0x7f75c0105550 2024-05-24T11:31:43.234 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:41.817+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 msgr2=0x7f75ac03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:31:43.234 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:41.817+0000 7f75b6ffd700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 0x7f75ac03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:43.234 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.817+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f75bc04e850 con 0x7f75c0105550 2024-05-24T11:31:43.234 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.817+0000 7f75b6ffd700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f75ac03bff0 0x7f75ac03e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:43.234 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.817+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75ac03b2d0 con 0x7f75ac03bff0 2024-05-24T11:31:43.235 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.817+0000 7f75c5577700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f75ac03bff0 0x7f75ac03e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:43.235 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.818+0000 7f75c5577700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f75ac03bff0 0x7f75ac03e3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f75b0003a10 tx=0x7f75b00092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:43.235 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.819+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f75ac03b2d0 con 0x7f75ac03bff0 2024-05-24T11:31:43.235 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.825+0000 7f75c7fdc700 1 -- 172.21.15.39:0/974299423 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f75c00625d0 con 0x7f75ac03bff0 2024-05-24T11:31:43.235 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.825+0000 7f75b6ffd700 1 -- 172.21.15.39:0/974299423 <== mgr.14150 v2:172.21.15.39:6800/2090515453 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f75c00625d0 con 0x7f75ac03bff0 2024-05-24T11:31:43.236 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 -- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f75ac03bff0 msgr2=0x7f75ac03e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:43.236 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f75ac03bff0 0x7f75ac03e3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f75b0003a10 tx=0x7f75b00092b0 comp rx=0 tx=0).stop 2024-05-24T11:31:43.236 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 -- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 msgr2=0x7f75c019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:43.236 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c019c5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00b340 tx=0x7f75bc004000 comp rx=0 tx=0).stop 2024-05-24T11:31:43.236 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 -- 172.21.15.39:0/974299423 shutdown_connections 2024-05-24T11:31:43.236 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f75ac03bff0 0x7f75ac03e3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75c0105550 0x7f75c019c5c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 --2- 172.21.15.39:0/974299423 >> [v2:172.21.15.39:6800/3960296835,v1:172.21.15.39:6801/3960296835] conn(0x7f75ac038700 0x7f75ac03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 -- 172.21.15.39:0/974299423 >> 172.21.15.39:0/974299423 conn(0x7f75c0100b10 msgr2=0x7f75c0192de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 -- 172.21.15.39:0/974299423 shutdown_connections 2024-05-24T11:31:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:42.826+0000 7f75c7fdc700 1 -- 172.21.15.39:0/974299423 wait complete. 2024-05-24T11:31:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 9 is available 2024-05-24T11:31:43.238 INFO:teuthology.orchestra.run.smithi039.stdout:Generating a dashboard self-signed certificate... 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.617+0000 7f780fe9f700 1 Processor -- start 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.617+0000 7f780fe9f700 1 -- start start 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.617+0000 7f780fe9f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808106f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.617+0000 7f780fe9f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78081074f0 con 0x7f7808106b00 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.617+0000 7f780dc3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808106f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.618+0000 7f780dc3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808106f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41600/0 (socket says 172.21.15.39:41600) 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.618+0000 7f780dc3b700 1 -- 172.21.15.39:0/702471847 learned_addr learned my addr 172.21.15.39:0/702471847 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.618+0000 7f780dc3b700 1 -- 172.21.15.39:0/702471847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7808105760 con 0x7f7808106b00 2024-05-24T11:31:44.463 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.618+0000 7f780dc3b700 1 --2- 172.21.15.39:0/702471847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808106f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78040098a0 tx=0x7f7804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1371ac19c9ed631d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:44.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.618+0000 7f780cc39700 1 -- 172.21.15.39:0/702471847 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7804004160 con 0x7f7808106b00 2024-05-24T11:31:44.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.618+0000 7f780cc39700 1 -- 172.21.15.39:0/702471847 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78040042c0 con 0x7f7808106b00 2024-05-24T11:31:44.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.618+0000 7f780cc39700 1 -- 172.21.15.39:0/702471847 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78040118e0 con 0x7f7808106b00 2024-05-24T11:31:44.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.619+0000 7f780fe9f700 1 -- 172.21.15.39:0/702471847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 msgr2=0x7f7808106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:44.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.619+0000 7f780fe9f700 1 --2- 172.21.15.39:0/702471847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808106f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78040098a0 tx=0x7f7804009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:44.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.619+0000 7f780fe9f700 1 -- 172.21.15.39:0/702471847 shutdown_connections 2024-05-24T11:31:44.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.619+0000 7f780fe9f700 1 --2- 172.21.15.39:0/702471847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808106f20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.619+0000 7f780fe9f700 1 -- 172.21.15.39:0/702471847 >> 172.21.15.39:0/702471847 conn(0x7f7808100290 msgr2=0x7f78081026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.619+0000 7f780fe9f700 1 -- 172.21.15.39:0/702471847 shutdown_connections 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.619+0000 7f780fe9f700 1 -- 172.21.15.39:0/702471847 wait complete. 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.620+0000 7f780fe9f700 1 Processor -- start 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.620+0000 7f780fe9f700 1 -- start start 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.620+0000 7f780fe9f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808190510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.620+0000 7f780fe9f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7808190a50 con 0x7f7808106b00 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.620+0000 7f780dc3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808190510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.620+0000 7f780dc3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808190510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41606/0 (socket says 172.21.15.39:41606) 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.620+0000 7f780dc3b700 1 -- 172.21.15.39:0/3713690429 learned_addr learned my addr 172.21.15.39:0/3713690429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.621+0000 7f780dc3b700 1 -- 172.21.15.39:0/3713690429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7804009550 con 0x7f7808106b00 2024-05-24T11:31:44.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.621+0000 7f780dc3b700 1 --2- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808190510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7804003f70 tx=0x7f78040119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.621+0000 7f77feffd700 1 -- 172.21.15.39:0/3713690429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78040036a0 con 0x7f7808106b00 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.621+0000 7f77feffd700 1 -- 172.21.15.39:0/3713690429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f780401b440 con 0x7f7808106b00 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.621+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780818ece0 con 0x7f7808106b00 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.621+0000 7f77feffd700 1 -- 172.21.15.39:0/3713690429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7804023400 con 0x7f7808106b00 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.622+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780818f180 con 0x7f7808106b00 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.622+0000 7f77feffd700 1 -- 172.21.15.39:0/3713690429 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f7804023680 con 0x7f7808106b00 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.622+0000 7f77feffd700 1 --2- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f77f4038620 0x7f77f403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.622+0000 7f77feffd700 1 -- 172.21.15.39:0/3713690429 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7804051660 con 0x7f7808106b00 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.622+0000 7f780d43a700 1 --2- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f77f4038620 0x7f77f403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:44.466 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.623+0000 7f780d43a700 1 --2- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f77f4038620 0x7f77f403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006fd0 tx=0x7f77f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.623+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78080625d0 con 0x7f7808106b00 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.628+0000 7f77feffd700 1 -- 172.21.15.39:0/3713690429 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7804052060 con 0x7f7808106b00 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.845+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f7808193f30 con 0x7f77f4038620 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.957+0000 7f77feffd700 1 -- 172.21.15.39:0/3713690429 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f7808193f30 con 0x7f77f4038620 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f77f4038620 msgr2=0x7f77f403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 --2- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f77f4038620 0x7f77f403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006fd0 tx=0x7f77f8006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 msgr2=0x7f7808190510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 --2- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808190510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7804003f70 tx=0x7f78040119a0 comp rx=0 tx=0).stop 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 shutdown_connections 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 --2- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f77f4038620 0x7f77f403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 --2- 172.21.15.39:0/3713690429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808106b00 0x7f7808190510 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:44.467 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 >> 172.21.15.39:0/3713690429 conn(0x7f7808100290 msgr2=0x7f7808101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:44.468 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 shutdown_connections 2024-05-24T11:31:44.468 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:43.959+0000 7f780fe9f700 1 -- 172.21.15.39:0/3713690429 wait complete. 2024-05-24T11:31:44.468 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial admin user... 2024-05-24T11:31:44.774 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:44 smithi039 ceph-mon[42919]: [24/May/2024:11:31:43] ENGINE Bus STARTING 2024-05-24T11:31:44.775 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:44 smithi039 ceph-mon[42919]: [24/May/2024:11:31:43] ENGINE Serving on http://172.21.15.39:8765 2024-05-24T11:31:44.775 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:44 smithi039 ceph-mon[42919]: [24/May/2024:11:31:43] ENGINE Serving on https://172.21.15.39:7150 2024-05-24T11:31:44.775 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:44 smithi039 ceph-mon[42919]: [24/May/2024:11:31:43] ENGINE Bus STARTED 2024-05-24T11:31:44.775 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:44.775 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:44.775 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:45.742 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:45 smithi039 ceph-mon[42919]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:45.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:45 smithi039 ceph-mon[42919]: mgrmap e12: x(active, since 2s) 2024-05-24T11:31:45.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:45.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$njh1EZMiXddkQICkCybmquHV9tOAJKMiVFAxxY0rpxwgQjdrqeNmW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716550305, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-24T11:31:45.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.959+0000 7fb5a267d700 1 Processor -- start 2024-05-24T11:31:45.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.959+0000 7fb5a267d700 1 -- start start 2024-05-24T11:31:45.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.959+0000 7fb5a267d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c1094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:45.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.959+0000 7fb5a267d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0622c0 con 0x7fb59c1070c0 2024-05-24T11:31:45.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.959+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c1094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:45.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.959+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c1094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41614/0 (socket says 172.21.15.39:41614) 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.959+0000 7fb59bfff700 1 -- 172.21.15.39:0/3440336466 learned_addr learned my addr 172.21.15.39:0/3440336466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.960+0000 7fb59bfff700 1 -- 172.21.15.39:0/3440336466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c1099f0 con 0x7fb59c1070c0 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.960+0000 7fb59bfff700 1 --2- 172.21.15.39:0/3440336466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c1094b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0098a0 tx=0x7fb58c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d840aec99c2cf70 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.960+0000 7fb59affd700 1 -- 172.21.15.39:0/3440336466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb58c004160 con 0x7fb59c1070c0 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.960+0000 7fb59affd700 1 -- 172.21.15.39:0/3440336466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb58c0042c0 con 0x7fb59c1070c0 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.960+0000 7fb59affd700 1 -- 172.21.15.39:0/3440336466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb58c0118e0 con 0x7fb59c1070c0 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.961+0000 7fb5a267d700 1 -- 172.21.15.39:0/3440336466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 msgr2=0x7fb59c1094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.961+0000 7fb5a267d700 1 --2- 172.21.15.39:0/3440336466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c1094b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0098a0 tx=0x7fb58c009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.961+0000 7fb5a267d700 1 -- 172.21.15.39:0/3440336466 shutdown_connections 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.961+0000 7fb5a267d700 1 --2- 172.21.15.39:0/3440336466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c1094b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.961+0000 7fb5a267d700 1 -- 172.21.15.39:0/3440336466 >> 172.21.15.39:0/3440336466 conn(0x7fb59c100b10 msgr2=0x7fb59c102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:45.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.961+0000 7fb5a267d700 1 -- 172.21.15.39:0/3440336466 shutdown_connections 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.961+0000 7fb5a267d700 1 -- 172.21.15.39:0/3440336466 wait complete. 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.961+0000 7fb5a267d700 1 Processor -- start 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.961+0000 7fb5a267d700 1 -- start start 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb5a267d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb5a267d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19a860 con 0x7fb59c1070c0 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41618/0 (socket says 172.21.15.39:41618) 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb59bfff700 1 -- 172.21.15.39:0/3879234653 learned_addr learned my addr 172.21.15.39:0/3879234653 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb59bfff700 1 -- 172.21.15.39:0/3879234653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c009550 con 0x7fb59c1070c0 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb59bfff700 1 --2- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c19a320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb58c003f70 tx=0x7fb58c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb5997fa700 1 -- 172.21.15.39:0/3879234653 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb58c0036a0 con 0x7fb59c1070c0 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb5997fa700 1 -- 172.21.15.39:0/3879234653 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb58c01b440 con 0x7fb59c1070c0 2024-05-24T11:31:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c19aa60 con 0x7fb59c1070c0 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb5997fa700 1 -- 172.21.15.39:0/3879234653 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb58c023400 con 0x7fb59c1070c0 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.962+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c19af00 con 0x7fb59c1070c0 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.963+0000 7fb5997fa700 1 -- 172.21.15.39:0/3879234653 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fb58c003800 con 0x7fb59c1070c0 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.963+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c10a7d0 con 0x7fb59c1070c0 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.963+0000 7fb5997fa700 1 --2- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb584038750 0x7fb58403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.964+0000 7fb5997fa700 1 -- 172.21.15.39:0/3879234653 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb58c04c890 con 0x7fb59c1070c0 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.964+0000 7fb59b7fe700 1 --2- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb584038750 0x7fb58403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.964+0000 7fb59b7fe700 1 --2- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb584038750 0x7fb58403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb590006fd0 tx=0x7fb590006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:44.969+0000 7fb5997fa700 1 -- 172.21.15.39:0/3879234653 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb58c02b760 con 0x7fb59c1070c0 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.187+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb59c074760 con 0x7fb584038750 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.434+0000 7fb5997fa700 1 -- 172.21.15.39:0/3879234653 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fb59c074760 con 0x7fb584038750 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb584038750 msgr2=0x7fb58403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:45.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 --2- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb584038750 0x7fb58403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb590006fd0 tx=0x7fb590006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:45.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 msgr2=0x7fb59c19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:45.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 --2- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c19a320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb58c003f70 tx=0x7fb58c0119a0 comp rx=0 tx=0).stop 2024-05-24T11:31:45.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 shutdown_connections 2024-05-24T11:31:45.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 --2- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb584038750 0x7fb58403ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:45.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 --2- 172.21.15.39:0/3879234653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb59c1070c0 0x7fb59c19a320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:45.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 >> 172.21.15.39:0/3879234653 conn(0x7fb59c100b10 msgr2=0x7fb59c1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:45.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 shutdown_connections 2024-05-24T11:31:45.817 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:45.437+0000 7fb5a267d700 1 -- 172.21.15.39:0/3879234653 wait complete. 2024-05-24T11:31:45.817 INFO:teuthology.orchestra.run.smithi039.stdout:Fetching dashboard port number... 2024-05-24T11:31:46.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:46 smithi039 ceph-mon[42919]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:46.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:46 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4062629278' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 8443 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.198+0000 7fae0980a700 1 Processor -- start 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.198+0000 7fae0980a700 1 -- start start 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.198+0000 7fae0980a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04109540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.198+0000 7fae0980a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae04062240 con 0x7fae04107150 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.198+0000 7fae02ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04109540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.198+0000 7fae02ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04109540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41632/0 (socket says 172.21.15.39:41632) 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.198+0000 7fae02ffd700 1 -- 172.21.15.39:0/788914542 learned_addr learned my addr 172.21.15.39:0/788914542 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.199+0000 7fae02ffd700 1 -- 172.21.15.39:0/788914542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae04109a80 con 0x7fae04107150 2024-05-24T11:31:46.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.199+0000 7fae02ffd700 1 --2- 172.21.15.39:0/788914542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04109540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fadf80098a0 tx=0x7fadf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5455f84a5ee94224 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.199+0000 7fae01ffb700 1 -- 172.21.15.39:0/788914542 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadf8004160 con 0x7fae04107150 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.199+0000 7fae01ffb700 1 -- 172.21.15.39:0/788914542 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fadf80042c0 con 0x7fae04107150 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.199+0000 7fae01ffb700 1 -- 172.21.15.39:0/788914542 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadf80118e0 con 0x7fae04107150 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.199+0000 7fae0980a700 1 -- 172.21.15.39:0/788914542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 msgr2=0x7fae04109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.199+0000 7fae0980a700 1 --2- 172.21.15.39:0/788914542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04109540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fadf80098a0 tx=0x7fadf8009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.199+0000 7fae0980a700 1 -- 172.21.15.39:0/788914542 shutdown_connections 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.199+0000 7fae0980a700 1 --2- 172.21.15.39:0/788914542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04109540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.200+0000 7fae0980a700 1 -- 172.21.15.39:0/788914542 >> 172.21.15.39:0/788914542 conn(0x7fae04100b20 msgr2=0x7fae04102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.200+0000 7fae0980a700 1 -- 172.21.15.39:0/788914542 shutdown_connections 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.200+0000 7fae0980a700 1 -- 172.21.15.39:0/788914542 wait complete. 2024-05-24T11:31:46.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.200+0000 7fae0980a700 1 Processor -- start 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.200+0000 7fae0980a700 1 -- start start 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae0980a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04106980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae0980a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae04106ec0 con 0x7fae04107150 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae02ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae02ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41646/0 (socket says 172.21.15.39:41646) 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae02ffd700 1 -- 172.21.15.39:0/4062629278 learned_addr learned my addr 172.21.15.39:0/4062629278 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae02ffd700 1 -- 172.21.15.39:0/4062629278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf8009550 con 0x7fae04107150 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae02ffd700 1 --2- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04106980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fadf8006af0 tx=0x7fadf8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae08808700 1 -- 172.21.15.39:0/4062629278 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadf80037d0 con 0x7fae04107150 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae08808700 1 -- 172.21.15.39:0/4062629278 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fadf801b440 con 0x7fae04107150 2024-05-24T11:31:46.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae04105100 con 0x7fae04107150 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae08808700 1 -- 172.21.15.39:0/4062629278 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadf8023400 con 0x7fae04107150 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.201+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae041055a0 con 0x7fae04107150 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.202+0000 7fae08808700 1 -- 172.21.15.39:0/4062629278 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fadf8023670 con 0x7fae04107150 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.202+0000 7fae08808700 1 --2- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fadec038750 0x7fadec03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.202+0000 7fae08808700 1 -- 172.21.15.39:0/4062629278 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fadf804c860 con 0x7fae04107150 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.202+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0410a7b0 con 0x7fae04107150 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.203+0000 7fae027fc700 1 --2- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fadec038750 0x7fadec03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.203+0000 7fae027fc700 1 --2- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fadec038750 0x7fadec03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fadf4006fd0 tx=0x7fadf4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.207+0000 7fae08808700 1 -- 172.21.15.39:0/4062629278 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fadf8018350 con 0x7fae04107150 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.409+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fae0410a940 con 0x7fae04107150 2024-05-24T11:31:46.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.409+0000 7fae08808700 1 -- 172.21.15.39:0/4062629278 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fadf802bb40 con 0x7fae04107150 2024-05-24T11:31:46.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.411+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fadec038750 msgr2=0x7fadec03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:46.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.411+0000 7fae0980a700 1 --2- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fadec038750 0x7fadec03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fadf4006fd0 tx=0x7fadf4006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:46.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.412+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 msgr2=0x7fae04106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:46.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.412+0000 7fae0980a700 1 --2- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04106980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fadf8006af0 tx=0x7fadf8011ed0 comp rx=0 tx=0).stop 2024-05-24T11:31:46.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.412+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 shutdown_connections 2024-05-24T11:31:46.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.412+0000 7fae0980a700 1 --2- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fadec038750 0x7fadec03ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:46.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.412+0000 7fae0980a700 1 --2- 172.21.15.39:0/4062629278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04107150 0x7fae04106980 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:46.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.412+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 >> 172.21.15.39:0/4062629278 conn(0x7fae04100b20 msgr2=0x7fae04101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:46.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.412+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 shutdown_connections 2024-05-24T11:31:46.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:46.412+0000 7fae0980a700 1 -- 172.21.15.39:0/4062629278 wait complete. 2024-05-24T11:31:46.811 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-24T11:31:46.811 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-05-24T11:31:46.818 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-24T11:31:46.818 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-24T11:31:46.818 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-05-24T11:31:46.818 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-24T11:31:46.820 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph Dashboard is now available at: 2024-05-24T11:31:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: URL: https://smithi039:8443/ 2024-05-24T11:31:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: User: admin 2024-05-24T11:31:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: Password: q7vx0es8bm 2024-05-24T11:31:46.820 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:46.820 INFO:teuthology.orchestra.run.smithi039.stdout:Saving cluster configuration to /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config directory 2024-05-24T11:31:46.821 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling autotune for osd_memory_target 2024-05-24T11:31:47.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.159+0000 7f8c02cf2700 1 Processor -- start 2024-05-24T11:31:47.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.159+0000 7f8c02cf2700 1 -- start start 2024-05-24T11:31:47.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.159+0000 7f8c02cf2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc1094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:47.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.159+0000 7f8c02cf2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0622c0 con 0x7f8bfc1070d0 2024-05-24T11:31:47.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.159+0000 7f8c00a8e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc1094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:47.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.159+0000 7f8c00a8e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc1094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41662/0 (socket says 172.21.15.39:41662) 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.159+0000 7f8c00a8e700 1 -- 172.21.15.39:0/2330891959 learned_addr learned my addr 172.21.15.39:0/2330891959 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.159+0000 7f8c00a8e700 1 -- 172.21.15.39:0/2330891959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc109a00 con 0x7f8bfc1070d0 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.160+0000 7f8c00a8e700 1 --2- 172.21.15.39:0/2330891959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc1094c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00098a0 tx=0x7f8bf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dfa0076f8982a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.160+0000 7f8bfb7fe700 1 -- 172.21.15.39:0/2330891959 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bf0004160 con 0x7f8bfc1070d0 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.160+0000 7f8bfb7fe700 1 -- 172.21.15.39:0/2330891959 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bf00042c0 con 0x7f8bfc1070d0 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.160+0000 7f8bfb7fe700 1 -- 172.21.15.39:0/2330891959 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bf00118e0 con 0x7f8bfc1070d0 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.160+0000 7f8c02cf2700 1 -- 172.21.15.39:0/2330891959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 msgr2=0x7f8bfc1094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.160+0000 7f8c02cf2700 1 --2- 172.21.15.39:0/2330891959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc1094c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00098a0 tx=0x7f8bf0009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.160+0000 7f8c02cf2700 1 -- 172.21.15.39:0/2330891959 shutdown_connections 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.160+0000 7f8c02cf2700 1 --2- 172.21.15.39:0/2330891959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc1094c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.160+0000 7f8c02cf2700 1 -- 172.21.15.39:0/2330891959 >> 172.21.15.39:0/2330891959 conn(0x7f8bfc100b20 msgr2=0x7f8bfc102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.161+0000 7f8c02cf2700 1 -- 172.21.15.39:0/2330891959 shutdown_connections 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.161+0000 7f8c02cf2700 1 -- 172.21.15.39:0/2330891959 wait complete. 2024-05-24T11:31:47.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.161+0000 7f8c02cf2700 1 Processor -- start 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.161+0000 7f8c02cf2700 1 -- start start 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.161+0000 7f8c02cf2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc106900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.161+0000 7f8c02cf2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc106e40 con 0x7f8bfc1070d0 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.161+0000 7f8c00a8e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc106900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.161+0000 7f8c00a8e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc106900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41678/0 (socket says 172.21.15.39:41678) 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.161+0000 7f8c00a8e700 1 -- 172.21.15.39:0/222334725 learned_addr learned my addr 172.21.15.39:0/222334725 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.162+0000 7f8c00a8e700 1 -- 172.21.15.39:0/222334725 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0009550 con 0x7f8bfc1070d0 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.162+0000 7f8c00a8e700 1 --2- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc106900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0006af0 tx=0x7f8bf0011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.162+0000 7f8bf9ffb700 1 -- 172.21.15.39:0/222334725 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bf00037d0 con 0x7f8bfc1070d0 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.162+0000 7f8bf9ffb700 1 -- 172.21.15.39:0/222334725 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bf001b440 con 0x7f8bfc1070d0 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.162+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc105080 con 0x7f8bfc1070d0 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.162+0000 7f8bf9ffb700 1 -- 172.21.15.39:0/222334725 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bf0023400 con 0x7f8bfc1070d0 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.162+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc105520 con 0x7f8bfc1070d0 2024-05-24T11:31:47.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.163+0000 7f8bf9ffb700 1 -- 172.21.15.39:0/222334725 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f8bf0023670 con 0x7f8bfc1070d0 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.163+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc10a7e0 con 0x7f8bfc1070d0 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.163+0000 7f8bf9ffb700 1 --2- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8bec038750 0x7f8bec03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.163+0000 7f8bf9ffb700 1 -- 172.21.15.39:0/222334725 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8bf004c9f0 con 0x7f8bfc1070d0 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.164+0000 7f8bfbfff700 1 --2- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8bec038750 0x7f8bec03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.164+0000 7f8bfbfff700 1 --2- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8bec038750 0x7f8bec03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006fd0 tx=0x7f8be8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.168+0000 7f8bf9ffb700 1 -- 172.21.15.39:0/222334725 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8bf002c910 con 0x7f8bfc1070d0 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.368+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f8bfc10aa20 con 0x7f8bfc1070d0 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.368+0000 7f8bf9ffb700 1 -- 172.21.15.39:0/222334725 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8bf002bb40 con 0x7f8bfc1070d0 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8bec038750 msgr2=0x7f8bec03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 --2- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8bec038750 0x7f8bec03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006fd0 tx=0x7f8be8006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 msgr2=0x7f8bfc106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 --2- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc106900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0006af0 tx=0x7f8bf0011ed0 comp rx=0 tx=0).stop 2024-05-24T11:31:47.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 shutdown_connections 2024-05-24T11:31:47.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 --2- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8bec038750 0x7f8bec03ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:47.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 --2- 172.21.15.39:0/222334725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bfc1070d0 0x7f8bfc106900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:47.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 >> 172.21.15.39:0/222334725 conn(0x7f8bfc100b20 msgr2=0x7f8bfc101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:47.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 shutdown_connections 2024-05-24T11:31:47.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:47.370+0000 7f8c02cf2700 1 -- 172.21.15.39:0/222334725 wait complete. 2024-05-24T11:31:48.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.118+0000 7f0725552700 1 Processor -- start 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.118+0000 7f0725552700 1 -- start start 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.118+0000 7f0725552700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720109510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.118+0000 7f0725552700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0720079ec0 con 0x7f0720107120 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.119+0000 7f071effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.119+0000 7f071effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35052/0 (socket says 172.21.15.39:35052) 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.119+0000 7f071effd700 1 -- 172.21.15.39:0/2331984973 learned_addr learned my addr 172.21.15.39:0/2331984973 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.119+0000 7f071effd700 1 -- 172.21.15.39:0/2331984973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720109a50 con 0x7f0720107120 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.119+0000 7f071effd700 1 --2- 172.21.15.39:0/2331984973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720109510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07080098a0 tx=0x7f0708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727737ae17f5dff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.119+0000 7f071dffb700 1 -- 172.21.15.39:0/2331984973 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0708004160 con 0x7f0720107120 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.119+0000 7f071dffb700 1 -- 172.21.15.39:0/2331984973 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07080042c0 con 0x7f0720107120 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.119+0000 7f071dffb700 1 -- 172.21.15.39:0/2331984973 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07080118e0 con 0x7f0720107120 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.120+0000 7f0725552700 1 -- 172.21.15.39:0/2331984973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 msgr2=0x7f0720109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:48.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.120+0000 7f0725552700 1 --2- 172.21.15.39:0/2331984973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720109510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f07080098a0 tx=0x7f0708009bb0 comp rx=0 tx=0).stop 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.120+0000 7f0725552700 1 -- 172.21.15.39:0/2331984973 shutdown_connections 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.120+0000 7f0725552700 1 --2- 172.21.15.39:0/2331984973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720109510 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.120+0000 7f0725552700 1 -- 172.21.15.39:0/2331984973 >> 172.21.15.39:0/2331984973 conn(0x7f0720100bb0 msgr2=0x7f0720102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.120+0000 7f0725552700 1 -- 172.21.15.39:0/2331984973 shutdown_connections 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.120+0000 7f0725552700 1 -- 172.21.15.39:0/2331984973 wait complete. 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.120+0000 7f0725552700 1 Processor -- start 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.121+0000 7f0725552700 1 -- start start 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.121+0000 7f0725552700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720106900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.121+0000 7f0725552700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0720106e40 con 0x7f0720107120 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.121+0000 7f071effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720106900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.121+0000 7f071effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720106900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35056/0 (socket says 172.21.15.39:35056) 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.121+0000 7f071effd700 1 -- 172.21.15.39:0/738753550 learned_addr learned my addr 172.21.15.39:0/738753550 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.121+0000 7f071effd700 1 -- 172.21.15.39:0/738753550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0708009550 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.121+0000 7f071effd700 1 --2- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720106900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07080046e0 tx=0x7f07080119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.122+0000 7f0717fff700 1 -- 172.21.15.39:0/738753550 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07080037d0 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.122+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07201050d0 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.122+0000 7f0717fff700 1 -- 172.21.15.39:0/738753550 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f070801b440 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.122+0000 7f0717fff700 1 -- 172.21.15.39:0/738753550 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0708023400 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.122+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0720105570 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.123+0000 7f0717fff700 1 -- 172.21.15.39:0/738753550 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f0708003930 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.123+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07200625d0 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.123+0000 7f0717fff700 1 --2- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f070c0387f0 0x7f070c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.123+0000 7f0717fff700 1 -- 172.21.15.39:0/738753550 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f070804c7b0 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.128+0000 7f071e7fc700 1 --2- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f070c0387f0 0x7f070c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.128+0000 7f0717fff700 1 -- 172.21.15.39:0/738753550 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f07080195b0 con 0x7f0720107120 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.128+0000 7f071e7fc700 1 --2- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f070c0387f0 0x7f070c03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0710006fd0 tx=0x7f0710006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:48.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.443+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f072010a830 con 0x7f0720107120 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.450+0000 7f0717fff700 1 -- 172.21.15.39:0/738753550 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f07080193d0 con 0x7f0720107120 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.452+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f070c0387f0 msgr2=0x7f070c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.452+0000 7f0725552700 1 --2- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f070c0387f0 0x7f070c03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0710006fd0 tx=0x7f0710006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.452+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 msgr2=0x7f0720106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.452+0000 7f0725552700 1 --2- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720106900 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07080046e0 tx=0x7f07080119a0 comp rx=0 tx=0).stop 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.452+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 shutdown_connections 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.452+0000 7f0725552700 1 --2- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f070c0387f0 0x7f070c03acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.452+0000 7f0725552700 1 --2- 172.21.15.39:0/738753550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0720107120 0x7f0720106900 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.452+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 >> 172.21.15.39:0/738753550 conn(0x7f0720100bb0 msgr2=0x7f0720101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.452+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 shutdown_connections 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-24T11:31:48.453+0000 7f0725552700 1 -- 172.21.15.39:0/738753550 wait complete. 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-24T11:31:48.816 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout:Or, if you are only running a single cluster on this host: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: ceph telemetry on 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout:For more information see: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:48.817 INFO:teuthology.orchestra.run.smithi039.stdout:Bootstrap complete. 2024-05-24T11:31:48.845 INFO:tasks.cephadm:Fetching config... 2024-05-24T11:31:48.845 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:31:48.845 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-24T11:31:48.863 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-24T11:31:48.863 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:31:48.863 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-24T11:31:48.922 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-24T11:31:48.922 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:31:48.922 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-24T11:31:48.991 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-24T11:31:48.991 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:31:48.991 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-24T11:31:49.049 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-24T11:31:49.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1H4LQC8DpFBgI21N9m9sNd2eyc/cpdHTBN+gSyP3pno3I51itjjlPAAdRgaaJme6RiJQtiz7p36JzYw+Rg17MwzhlkDtARPQ8UoEPHmutZHYH2tHF5NzqOvJlI84M5nrdZTuSdzzYJsohhAPIozC0666fGcwGGMa4Z9M+++ctlhfdglfGWpM17iMgG7RB0JPWdoO/mHbQ7ruG7wsASs58a/1YyOu3QVWmfnU+wmvAXXR/7cus/MfouvrLUXTmUif2Pg4xYedVgfvTMfTsozFJqJjT2kHXDxLX5PUQ/poM0I/P0qjaCprlrGBEY9XefFTzJLslsDtILqzuIYVaY8agenFbKL5MTGeRzlK6mJx1iv9iCXChVI+MfaDvBAZwMf78AsmdlwDExF8L7x7EhnCSPfKdH5AcZphtTIfQX2hfP6ou1M/IWrmJ6by8akalPsXYvfHYlOnRcSHco8qyuzIZU6P64vW+gYlLSmjs++Vg3QoctJpPNsuKLzzQ+mxxCSc= ceph-f68008c4-19c0-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-24T11:31:49.050 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:48 smithi039 ceph-mon[42919]: mgrmap e13: x(active, since 6s) 2024-05-24T11:31:49.050 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:48 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/738753550' entity='client.admin' 2024-05-24T11:31:49.131 INFO:teuthology.orchestra.run.smithi039.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1H4LQC8DpFBgI21N9m9sNd2eyc/cpdHTBN+gSyP3pno3I51itjjlPAAdRgaaJme6RiJQtiz7p36JzYw+Rg17MwzhlkDtARPQ8UoEPHmutZHYH2tHF5NzqOvJlI84M5nrdZTuSdzzYJsohhAPIozC0666fGcwGGMa4Z9M+++ctlhfdglfGWpM17iMgG7RB0JPWdoO/mHbQ7ruG7wsASs58a/1YyOu3QVWmfnU+wmvAXXR/7cus/MfouvrLUXTmUif2Pg4xYedVgfvTMfTsozFJqJjT2kHXDxLX5PUQ/poM0I/P0qjaCprlrGBEY9XefFTzJLslsDtILqzuIYVaY8agenFbKL5MTGeRzlK6mJx1iv9iCXChVI+MfaDvBAZwMf78AsmdlwDExF8L7x7EhnCSPfKdH5AcZphtTIfQX2hfP6ou1M/IWrmJ6by8akalPsXYvfHYlOnRcSHco8qyuzIZU6P64vW+gYlLSmjs++Vg3QoctJpPNsuKLzzQ+mxxCSc= ceph-f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:31:49.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1H4LQC8DpFBgI21N9m9sNd2eyc/cpdHTBN+gSyP3pno3I51itjjlPAAdRgaaJme6RiJQtiz7p36JzYw+Rg17MwzhlkDtARPQ8UoEPHmutZHYH2tHF5NzqOvJlI84M5nrdZTuSdzzYJsohhAPIozC0666fGcwGGMa4Z9M+++ctlhfdglfGWpM17iMgG7RB0JPWdoO/mHbQ7ruG7wsASs58a/1YyOu3QVWmfnU+wmvAXXR/7cus/MfouvrLUXTmUif2Pg4xYedVgfvTMfTsozFJqJjT2kHXDxLX5PUQ/poM0I/P0qjaCprlrGBEY9XefFTzJLslsDtILqzuIYVaY8agenFbKL5MTGeRzlK6mJx1iv9iCXChVI+MfaDvBAZwMf78AsmdlwDExF8L7x7EhnCSPfKdH5AcZphtTIfQX2hfP6ou1M/IWrmJ6by8akalPsXYvfHYlOnRcSHco8qyuzIZU6P64vW+gYlLSmjs++Vg3QoctJpPNsuKLzzQ+mxxCSc= ceph-f68008c4-19c0-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-24T11:31:49.193 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1H4LQC8DpFBgI21N9m9sNd2eyc/cpdHTBN+gSyP3pno3I51itjjlPAAdRgaaJme6RiJQtiz7p36JzYw+Rg17MwzhlkDtARPQ8UoEPHmutZHYH2tHF5NzqOvJlI84M5nrdZTuSdzzYJsohhAPIozC0666fGcwGGMa4Z9M+++ctlhfdglfGWpM17iMgG7RB0JPWdoO/mHbQ7ruG7wsASs58a/1YyOu3QVWmfnU+wmvAXXR/7cus/MfouvrLUXTmUif2Pg4xYedVgfvTMfTsozFJqJjT2kHXDxLX5PUQ/poM0I/P0qjaCprlrGBEY9XefFTzJLslsDtILqzuIYVaY8agenFbKL5MTGeRzlK6mJx1iv9iCXChVI+MfaDvBAZwMf78AsmdlwDExF8L7x7EhnCSPfKdH5AcZphtTIfQX2hfP6ou1M/IWrmJ6by8akalPsXYvfHYlOnRcSHco8qyuzIZU6P64vW+gYlLSmjs++Vg3QoctJpPNsuKLzzQ+mxxCSc= ceph-f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:31:49.208 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1H4LQC8DpFBgI21N9m9sNd2eyc/cpdHTBN+gSyP3pno3I51itjjlPAAdRgaaJme6RiJQtiz7p36JzYw+Rg17MwzhlkDtARPQ8UoEPHmutZHYH2tHF5NzqOvJlI84M5nrdZTuSdzzYJsohhAPIozC0666fGcwGGMa4Z9M+++ctlhfdglfGWpM17iMgG7RB0JPWdoO/mHbQ7ruG7wsASs58a/1YyOu3QVWmfnU+wmvAXXR/7cus/MfouvrLUXTmUif2Pg4xYedVgfvTMfTsozFJqJjT2kHXDxLX5PUQ/poM0I/P0qjaCprlrGBEY9XefFTzJLslsDtILqzuIYVaY8agenFbKL5MTGeRzlK6mJx1iv9iCXChVI+MfaDvBAZwMf78AsmdlwDExF8L7x7EhnCSPfKdH5AcZphtTIfQX2hfP6ou1M/IWrmJ6by8akalPsXYvfHYlOnRcSHco8qyuzIZU6P64vW+gYlLSmjs++Vg3QoctJpPNsuKLzzQ+mxxCSc= ceph-f68008c4-19c0-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-24T11:31:49.256 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1H4LQC8DpFBgI21N9m9sNd2eyc/cpdHTBN+gSyP3pno3I51itjjlPAAdRgaaJme6RiJQtiz7p36JzYw+Rg17MwzhlkDtARPQ8UoEPHmutZHYH2tHF5NzqOvJlI84M5nrdZTuSdzzYJsohhAPIozC0666fGcwGGMa4Z9M+++ctlhfdglfGWpM17iMgG7RB0JPWdoO/mHbQ7ruG7wsASs58a/1YyOu3QVWmfnU+wmvAXXR/7cus/MfouvrLUXTmUif2Pg4xYedVgfvTMfTsozFJqJjT2kHXDxLX5PUQ/poM0I/P0qjaCprlrGBEY9XefFTzJLslsDtILqzuIYVaY8agenFbKL5MTGeRzlK6mJx1iv9iCXChVI+MfaDvBAZwMf78AsmdlwDExF8L7x7EhnCSPfKdH5AcZphtTIfQX2hfP6ou1M/IWrmJ6by8akalPsXYvfHYlOnRcSHco8qyuzIZU6P64vW+gYlLSmjs++Vg3QoctJpPNsuKLzzQ+mxxCSc= ceph-f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:31:49.269 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-24T11:31:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:31:50.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.869+0000 7f87e7fc8700 1 -- 172.21.15.39:0/3483082517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 msgr2=0x7f87e01037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.869+0000 7f87e7fc8700 1 --2- 172.21.15.39:0/3483082517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 0x7f87e01037b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009910 tx=0x7f87dc009c20 comp rx=0 tx=0).stop 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.869+0000 7f87e7fc8700 1 -- 172.21.15.39:0/3483082517 shutdown_connections 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.869+0000 7f87e7fc8700 1 --2- 172.21.15.39:0/3483082517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 0x7f87e01037b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.869+0000 7f87e7fc8700 1 -- 172.21.15.39:0/3483082517 >> 172.21.15.39:0/3483082517 conn(0x7f87e00fadd0 msgr2=0x7f87e00fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.869+0000 7f87e7fc8700 1 -- 172.21.15.39:0/3483082517 shutdown_connections 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.869+0000 7f87e7fc8700 1 -- 172.21.15.39:0/3483082517 wait complete. 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.870+0000 7f87e7fc8700 1 Processor -- start 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.870+0000 7f87e7fc8700 1 -- start start 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.870+0000 7f87e7fc8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 0x7f87e0100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.870+0000 7f87e7fc8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e0101130 con 0x7f87e01013c0 2024-05-24T11:31:50.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.870+0000 7f87e5d64700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 0x7f87e0100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:50.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.870+0000 7f87e5d64700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 0x7f87e0100bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35080/0 (socket says 172.21.15.39:35080) 2024-05-24T11:31:50.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.870+0000 7f87e5d64700 1 -- 172.21.15.39:0/478580330 learned_addr learned my addr 172.21.15.39:0/478580330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:50.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.871+0000 7f87e5d64700 1 -- 172.21.15.39:0/478580330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0095f0 con 0x7f87e01013c0 2024-05-24T11:31:50.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.871+0000 7f87e5d64700 1 --2- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 0x7f87e0100bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87dc005950 tx=0x7f87dc005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:50.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.871+0000 7f87d6ffd700 1 -- 172.21.15.39:0/478580330 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87dc00b960 con 0x7f87e01013c0 2024-05-24T11:31:50.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.871+0000 7f87d6ffd700 1 -- 172.21.15.39:0/478580330 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87dc00f460 con 0x7f87e01013c0 2024-05-24T11:31:50.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.871+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e00ff370 con 0x7f87e01013c0 2024-05-24T11:31:50.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.871+0000 7f87d6ffd700 1 -- 172.21.15.39:0/478580330 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87dc017440 con 0x7f87e01013c0 2024-05-24T11:31:50.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.871+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e00ff810 con 0x7f87e01013c0 2024-05-24T11:31:50.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.872+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e0106e30 con 0x7f87e01013c0 2024-05-24T11:31:50.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.872+0000 7f87d6ffd700 1 -- 172.21.15.39:0/478580330 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f87dc0176b0 con 0x7f87e01013c0 2024-05-24T11:31:50.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.872+0000 7f87d6ffd700 1 --2- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f87cc0387f0 0x7f87cc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:50.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.872+0000 7f87d6ffd700 1 -- 172.21.15.39:0/478580330 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f87dc04c820 con 0x7f87e01013c0 2024-05-24T11:31:50.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.873+0000 7f87e5563700 1 --2- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f87cc0387f0 0x7f87cc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:50.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.873+0000 7f87e5563700 1 --2- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f87cc0387f0 0x7f87cc03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f87d0006fd0 tx=0x7f87d0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:50.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:50.877+0000 7f87d6ffd700 1 -- 172.21.15.39:0/478580330 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f87dc0163a0 con 0x7f87e01013c0 2024-05-24T11:31:51.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.078+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f87e00625d0 con 0x7f87e01013c0 2024-05-24T11:31:51.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.082+0000 7f87d6ffd700 1 -- 172.21.15.39:0/478580330 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f87dc027030 con 0x7f87e01013c0 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f87cc0387f0 msgr2=0x7f87cc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 --2- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f87cc0387f0 0x7f87cc03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f87d0006fd0 tx=0x7f87d0006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 msgr2=0x7f87e0100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 --2- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 0x7f87e0100bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87dc005950 tx=0x7f87dc005810 comp rx=0 tx=0).stop 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 shutdown_connections 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 --2- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f87cc0387f0 0x7f87cc03acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 --2- 172.21.15.39:0/478580330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e01013c0 0x7f87e0100bf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 >> 172.21.15.39:0/478580330 conn(0x7f87e00fadd0 msgr2=0x7f87e00fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 shutdown_connections 2024-05-24T11:31:51.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:51.084+0000 7f87e7fc8700 1 -- 172.21.15.39:0/478580330 wait complete. 2024-05-24T11:31:51.461 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-24T11:31:51.462 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-24T11:31:51.699 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:31:52.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:52 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/478580330' entity='client.admin' 2024-05-24T11:31:52.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.839+0000 7fcc4b277700 1 -- 172.21.15.39:0/4075852672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 msgr2=0x7fcc440ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.839+0000 7fcc4b277700 1 --2- 172.21.15.39:0/4075852672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 0x7fcc440ffc50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009910 tx=0x7fcc40009c20 comp rx=0 tx=0).stop 2024-05-24T11:31:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.839+0000 7fcc4b277700 1 -- 172.21.15.39:0/4075852672 shutdown_connections 2024-05-24T11:31:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.839+0000 7fcc4b277700 1 --2- 172.21.15.39:0/4075852672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 0x7fcc440ffc50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.839+0000 7fcc4b277700 1 -- 172.21.15.39:0/4075852672 >> 172.21.15.39:0/4075852672 conn(0x7fcc440fadd0 msgr2=0x7fcc440fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.839+0000 7fcc4b277700 1 -- 172.21.15.39:0/4075852672 shutdown_connections 2024-05-24T11:31:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.839+0000 7fcc4b277700 1 -- 172.21.15.39:0/4075852672 wait complete. 2024-05-24T11:31:52.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.839+0000 7fcc4b277700 1 Processor -- start 2024-05-24T11:31:52.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.839+0000 7fcc4b277700 1 -- start start 2024-05-24T11:31:52.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc4b277700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 0x7fcc44194bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc4b277700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc44195110 con 0x7fcc440ff830 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc49013700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 0x7fcc44194bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc49013700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 0x7fcc44194bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35088/0 (socket says 172.21.15.39:35088) 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc49013700 1 -- 172.21.15.39:0/3371721554 learned_addr learned my addr 172.21.15.39:0/3371721554 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc49013700 1 -- 172.21.15.39:0/3371721554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400095f0 con 0x7fcc440ff830 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc49013700 1 --2- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 0x7fcc44194bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005790 tx=0x7fcc400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc3a7fc700 1 -- 172.21.15.39:0/3371721554 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc4000b960 con 0x7fcc440ff830 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc3a7fc700 1 -- 172.21.15.39:0/3371721554 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc4000f460 con 0x7fcc440ff830 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc44195310 con 0x7fcc440ff830 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc3a7fc700 1 -- 172.21.15.39:0/3371721554 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc400175d0 con 0x7fcc440ff830 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.840+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc441957b0 con 0x7fcc440ff830 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.841+0000 7fcc3a7fc700 1 -- 172.21.15.39:0/3371721554 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fcc40017810 con 0x7fcc440ff830 2024-05-24T11:31:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.841+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4418e400 con 0x7fcc440ff830 2024-05-24T11:31:52.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.841+0000 7fcc3a7fc700 1 --2- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcc300387a0 0x7fcc3003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:52.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.841+0000 7fcc3a7fc700 1 -- 172.21.15.39:0/3371721554 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc4004cb00 con 0x7fcc440ff830 2024-05-24T11:31:52.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.842+0000 7fcc48812700 1 --2- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcc300387a0 0x7fcc3003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:52.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.842+0000 7fcc48812700 1 --2- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcc300387a0 0x7fcc3003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcc34006fd0 tx=0x7fcc34006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:52.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:52.846+0000 7fcc3a7fc700 1 -- 172.21.15.39:0/3371721554 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcc400163a0 con 0x7fcc440ff830 2024-05-24T11:31:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.055+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc440613e0 con 0x7fcc300387a0 2024-05-24T11:31:53.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.058+0000 7fcc3a7fc700 1 -- 172.21.15.39:0/3371721554 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcc440613e0 con 0x7fcc300387a0 2024-05-24T11:31:53.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcc300387a0 msgr2=0x7fcc3003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 --2- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcc300387a0 0x7fcc3003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcc34006fd0 tx=0x7fcc34006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 msgr2=0x7fcc44194bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 --2- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 0x7fcc44194bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005790 tx=0x7fcc400047b0 comp rx=0 tx=0).stop 2024-05-24T11:31:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 shutdown_connections 2024-05-24T11:31:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 --2- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcc300387a0 0x7fcc3003ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 --2- 172.21.15.39:0/3371721554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc440ff830 0x7fcc44194bd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 >> 172.21.15.39:0/3371721554 conn(0x7fcc440fadd0 msgr2=0x7fcc440fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 shutdown_connections 2024-05-24T11:31:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:53.061+0000 7fcc4b277700 1 -- 172.21.15.39:0/3371721554 wait complete. 2024-05-24T11:31:53.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:53.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:53.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:31:53.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:53.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:31:53.437 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:53.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:53.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:31:53.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:31:53.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:31:53.439 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi053 2024-05-24T11:31:53.439 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:31:53.439 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.conf 2024-05-24T11:31:53.461 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:31:53.461 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:31:53.523 INFO:tasks.cephadm:Adding host smithi053 to orchestrator... 2024-05-24T11:31:53.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch host add smithi053 2024-05-24T11:31:53.754 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:31:54.573 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:54 smithi039 ceph-mon[42919]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:54.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:54 smithi039 ceph-mon[42919]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-24T11:31:54.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:54 smithi039 ceph-mon[42919]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:31:54.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:54 smithi039 ceph-mon[42919]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:31:54.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:54 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:54.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:54 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:54.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:54 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:54.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.750+0000 7f13196c2700 1 -- 172.21.15.39:0/1160724466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 msgr2=0x7f1314101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:54.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.750+0000 7f13196c2700 1 --2- 172.21.15.39:0/1160724466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 0x7f1314101e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1304009910 tx=0x7f1304009c20 comp rx=0 tx=0).stop 2024-05-24T11:31:54.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.750+0000 7f13196c2700 1 -- 172.21.15.39:0/1160724466 shutdown_connections 2024-05-24T11:31:54.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.750+0000 7f13196c2700 1 --2- 172.21.15.39:0/1160724466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 0x7f1314101e70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:54.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.750+0000 7f13196c2700 1 -- 172.21.15.39:0/1160724466 >> 172.21.15.39:0/1160724466 conn(0x7f13140fb300 msgr2=0x7f13140fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:54.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.751+0000 7f13196c2700 1 -- 172.21.15.39:0/1160724466 shutdown_connections 2024-05-24T11:31:54.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.751+0000 7f13196c2700 1 -- 172.21.15.39:0/1160724466 wait complete. 2024-05-24T11:31:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.751+0000 7f13196c2700 1 Processor -- start 2024-05-24T11:31:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.751+0000 7f13196c2700 1 -- start start 2024-05-24T11:31:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.751+0000 7f13196c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 0x7f1314198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.751+0000 7f13196c2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1314198850 con 0x7f1314101a90 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f1312ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 0x7f1314198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f1312ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 0x7f1314198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35100/0 (socket says 172.21.15.39:35100) 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f1312ffd700 1 -- 172.21.15.39:0/1038736036 learned_addr learned my addr 172.21.15.39:0/1038736036 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f1312ffd700 1 -- 172.21.15.39:0/1038736036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13040095f0 con 0x7f1314101a90 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f1312ffd700 1 --2- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 0x7f1314198310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f13040055f0 tx=0x7f13040056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f12fbfff700 1 -- 172.21.15.39:0/1038736036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f130401d070 con 0x7f1314101a90 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f12fbfff700 1 -- 172.21.15.39:0/1038736036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f130400b750 con 0x7f1314101a90 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1314198a50 con 0x7f1314101a90 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f12fbfff700 1 -- 172.21.15.39:0/1038736036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13040037b0 con 0x7f1314101a90 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.752+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1314198e70 con 0x7f1314101a90 2024-05-24T11:31:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.753+0000 7f12fbfff700 1 -- 172.21.15.39:0/1038736036 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f1304004390 con 0x7f1314101a90 2024-05-24T11:31:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.753+0000 7f12fbfff700 1 --2- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12fc0387a0 0x7f12fc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.753+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1314069620 con 0x7f1314101a90 2024-05-24T11:31:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.753+0000 7f12fbfff700 1 -- 172.21.15.39:0/1038736036 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f130404c480 con 0x7f1314101a90 2024-05-24T11:31:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.753+0000 7f13127fc700 1 --2- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12fc0387a0 0x7f12fc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.754+0000 7f13127fc700 1 --2- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12fc0387a0 0x7f12fc03ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1308006fd0 tx=0x7f1308006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:54.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.758+0000 7f12fbfff700 1 -- 172.21.15.39:0/1038736036 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1304003df0 con 0x7f1314101a90 2024-05-24T11:31:54.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:54.967+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}) v1 -- 0x7f1314107e20 con 0x7f12fc0387a0 2024-05-24T11:31:55.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:55 smithi039 ceph-mon[42919]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:31:56.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:56 smithi039 ceph-mon[42919]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:31:56.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:56 smithi039 ceph-mon[42919]: Deploying cephadm binary to smithi053 2024-05-24T11:31:57.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.412+0000 7f12fbfff700 1 -- 172.21.15.39:0/1038736036 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f1314107e20 con 0x7f12fc0387a0 2024-05-24T11:31:57.415 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi053' with addr '172.21.15.53' 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12fc0387a0 msgr2=0x7f12fc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 --2- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12fc0387a0 0x7f12fc03ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1308006fd0 tx=0x7f1308006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 msgr2=0x7f1314198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 --2- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 0x7f1314198310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f13040055f0 tx=0x7f13040056d0 comp rx=0 tx=0).stop 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 shutdown_connections 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 --2- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12fc0387a0 0x7f12fc03ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 --2- 172.21.15.39:0/1038736036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1314101a90 0x7f1314198310 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 >> 172.21.15.39:0/1038736036 conn(0x7f13140fb300 msgr2=0x7f1314106c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 shutdown_connections 2024-05-24T11:31:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:57.414+0000 7f13196c2700 1 -- 172.21.15.39:0/1038736036 wait complete. 2024-05-24T11:31:57.787 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-24T11:31:58.013 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:31:58.788 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:58 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:58.788 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:58 smithi039 ceph-mon[42919]: Added host smithi053 2024-05-24T11:31:58.788 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:58 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:31:58.789 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:58 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:58.789 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:58 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:31:58.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.965+0000 7f0d275fa700 1 -- 172.21.15.39:0/621651562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 msgr2=0x7f0d20105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:58.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.965+0000 7f0d275fa700 1 --2- 172.21.15.39:0/621651562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 0x7f0d20105c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009910 tx=0x7f0d1c009c20 comp rx=0 tx=0).stop 2024-05-24T11:31:58.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.965+0000 7f0d275fa700 1 -- 172.21.15.39:0/621651562 shutdown_connections 2024-05-24T11:31:58.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.965+0000 7f0d275fa700 1 --2- 172.21.15.39:0/621651562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 0x7f0d20105c00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.965+0000 7f0d275fa700 1 -- 172.21.15.39:0/621651562 >> 172.21.15.39:0/621651562 conn(0x7f0d20075350 msgr2=0x7f0d20075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.965+0000 7f0d275fa700 1 -- 172.21.15.39:0/621651562 shutdown_connections 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.965+0000 7f0d275fa700 1 -- 172.21.15.39:0/621651562 wait complete. 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.966+0000 7f0d275fa700 1 Processor -- start 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.966+0000 7f0d275fa700 1 -- start start 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.966+0000 7f0d275fa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 0x7f0d20198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.966+0000 7f0d275fa700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d201988c0 con 0x7f0d20105820 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.966+0000 7f0d25396700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 0x7f0d20198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.966+0000 7f0d25396700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 0x7f0d20198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34260/0 (socket says 172.21.15.39:34260) 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.966+0000 7f0d25396700 1 -- 172.21.15.39:0/3192013540 learned_addr learned my addr 172.21.15.39:0/3192013540 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.966+0000 7f0d25396700 1 -- 172.21.15.39:0/3192013540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0095f0 con 0x7f0d20105820 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.967+0000 7f0d25396700 1 --2- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 0x7f0d20198380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0055f0 tx=0x7f0d1c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:58.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.967+0000 7f0d167fc700 1 -- 172.21.15.39:0/3192013540 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d1c01d070 con 0x7f0d20105820 2024-05-24T11:31:58.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.967+0000 7f0d167fc700 1 -- 172.21.15.39:0/3192013540 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d1c00b750 con 0x7f0d20105820 2024-05-24T11:31:58.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.967+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d20198ac0 con 0x7f0d20105820 2024-05-24T11:31:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.967+0000 7f0d167fc700 1 -- 172.21.15.39:0/3192013540 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d1c0037b0 con 0x7f0d20105820 2024-05-24T11:31:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.967+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d20198ee0 con 0x7f0d20105820 2024-05-24T11:31:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.968+0000 7f0d167fc700 1 -- 172.21.15.39:0/3192013540 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f0d1c004390 con 0x7f0d20105820 2024-05-24T11:31:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.968+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d20100ef0 con 0x7f0d20105820 2024-05-24T11:31:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.968+0000 7f0d167fc700 1 --2- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0d0c0387f0 0x7f0d0c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:31:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.968+0000 7f0d167fc700 1 -- 172.21.15.39:0/3192013540 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0d1c04c3f0 con 0x7f0d20105820 2024-05-24T11:31:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.968+0000 7f0d24b95700 1 --2- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0d0c0387f0 0x7f0d0c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:31:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.969+0000 7f0d24b95700 1 --2- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0d0c0387f0 0x7f0d0c03acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0d10006fd0 tx=0x7f0d10006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:31:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:58.973+0000 7f0d167fc700 1 -- 172.21.15.39:0/3192013540 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0d1c00b8c0 con 0x7f0d20105820 2024-05-24T11:31:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.181+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d20109a50 con 0x7f0d0c0387f0 2024-05-24T11:31:59.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.182+0000 7f0d167fc700 1 -- 172.21.15.39:0/3192013540 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f0d20109a50 con 0x7f0d0c0387f0 2024-05-24T11:31:59.185 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:31:59.185 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}] 2024-05-24T11:31:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0d0c0387f0 msgr2=0x7f0d0c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 --2- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0d0c0387f0 0x7f0d0c03acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0d10006fd0 tx=0x7f0d10006e40 comp rx=0 tx=0).stop 2024-05-24T11:31:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 msgr2=0x7f0d20198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:31:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 --2- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 0x7f0d20198380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0055f0 tx=0x7f0d1c0056d0 comp rx=0 tx=0).stop 2024-05-24T11:31:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 shutdown_connections 2024-05-24T11:31:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 --2- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0d0c0387f0 0x7f0d0c03acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 --2- 172.21.15.39:0/3192013540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d20105820 0x7f0d20198380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:31:59.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 >> 172.21.15.39:0/3192013540 conn(0x7f0d20075350 msgr2=0x7f0d200680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:31:59.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 shutdown_connections 2024-05-24T11:31:59.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:31:59.184+0000 7f0d275fa700 1 -- 172.21.15.39:0/3192013540 wait complete. 2024-05-24T11:31:59.542 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-05-24T11:31:59.543 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:31:59.543 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-05-24T11:31:59.565 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:31:59.566 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:31:59.627 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-05-24T11:31:59.627 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch host add smithi137 2024-05-24T11:31:59.854 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:32:00.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:00.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:31:59 smithi039 ceph-mon[42919]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.765+0000 7f9c5cd53700 1 -- 172.21.15.39:0/2107004864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 msgr2=0x7f9c58101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.765+0000 7f9c5cd53700 1 --2- 172.21.15.39:0/2107004864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 0x7f9c58101e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009910 tx=0x7f9c4c009c20 comp rx=0 tx=0).stop 2024-05-24T11:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.765+0000 7f9c5cd53700 1 -- 172.21.15.39:0/2107004864 shutdown_connections 2024-05-24T11:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.765+0000 7f9c5cd53700 1 --2- 172.21.15.39:0/2107004864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 0x7f9c58101e70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.765+0000 7f9c5cd53700 1 -- 172.21.15.39:0/2107004864 >> 172.21.15.39:0/2107004864 conn(0x7f9c580fb2e0 msgr2=0x7f9c580fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.765+0000 7f9c5cd53700 1 -- 172.21.15.39:0/2107004864 shutdown_connections 2024-05-24T11:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.765+0000 7f9c5cd53700 1 -- 172.21.15.39:0/2107004864 wait complete. 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.766+0000 7f9c5cd53700 1 Processor -- start 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.766+0000 7f9c5cd53700 1 -- start start 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.766+0000 7f9c5cd53700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 0x7f9c58196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.766+0000 7f9c5cd53700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c581966d0 con 0x7f9c58101a90 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.766+0000 7f9c5659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 0x7f9c58196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.766+0000 7f9c5659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 0x7f9c58196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34278/0 (socket says 172.21.15.39:34278) 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.766+0000 7f9c5659c700 1 -- 172.21.15.39:0/1707555169 learned_addr learned my addr 172.21.15.39:0/1707555169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.766+0000 7f9c5659c700 1 -- 172.21.15.39:0/1707555169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0095f0 con 0x7f9c58101a90 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.766+0000 7f9c5659c700 1 --2- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 0x7f9c58196190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0055f0 tx=0x7f9c4c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.767+0000 7f9c477fe700 1 -- 172.21.15.39:0/1707555169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c4c01d070 con 0x7f9c58101a90 2024-05-24T11:32:00.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.767+0000 7f9c477fe700 1 -- 172.21.15.39:0/1707555169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c4c00b750 con 0x7f9c58101a90 2024-05-24T11:32:00.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.767+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c581968d0 con 0x7f9c58101a90 2024-05-24T11:32:00.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.767+0000 7f9c477fe700 1 -- 172.21.15.39:0/1707555169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c4c0037b0 con 0x7f9c58101a90 2024-05-24T11:32:00.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.767+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c58196cf0 con 0x7f9c58101a90 2024-05-24T11:32:00.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.767+0000 7f9c477fe700 1 -- 172.21.15.39:0/1707555169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f9c4c003a20 con 0x7f9c58101a90 2024-05-24T11:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.768+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c58069260 con 0x7f9c58101a90 2024-05-24T11:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.768+0000 7f9c477fe700 1 --2- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c400387f0 0x7f9c4003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.768+0000 7f9c477fe700 1 -- 172.21.15.39:0/1707555169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9c4c04c640 con 0x7f9c58101a90 2024-05-24T11:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.768+0000 7f9c55d9b700 1 --2- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c400387f0 0x7f9c4003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.768+0000 7f9c55d9b700 1 --2- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c400387f0 0x7f9c4003acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c48006fd0 tx=0x7f9c48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.773+0000 7f9c477fe700 1 -- 172.21.15.39:0/1707555169 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9c4c02abe0 con 0x7f9c58101a90 2024-05-24T11:32:00.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:00.980+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c581975e0 con 0x7f9c400387f0 2024-05-24T11:32:02.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: Deploying cephadm binary to smithi137 2024-05-24T11:32:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:02 smithi039 ceph-mon[42919]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:03.295 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-05-24T11:32:03.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.292+0000 7f9c477fe700 1 -- 172.21.15.39:0/1707555169 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9c581975e0 con 0x7f9c400387f0 2024-05-24T11:32:03.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c400387f0 msgr2=0x7f9c4003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:03.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 --2- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c400387f0 0x7f9c4003acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c48006fd0 tx=0x7f9c48006e40 comp rx=0 tx=0).stop 2024-05-24T11:32:03.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 msgr2=0x7f9c58196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:03.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 --2- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 0x7f9c58196190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0055f0 tx=0x7f9c4c0056d0 comp rx=0 tx=0).stop 2024-05-24T11:32:03.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 shutdown_connections 2024-05-24T11:32:03.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 --2- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c400387f0 0x7f9c4003acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:03.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 --2- 172.21.15.39:0/1707555169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c58101a90 0x7f9c58196190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:03.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 >> 172.21.15.39:0/1707555169 conn(0x7f9c580fb2e0 msgr2=0x7f9c58104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:03.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 shutdown_connections 2024-05-24T11:32:03.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:03.295+0000 7f9c5cd53700 1 -- 172.21.15.39:0/1707555169 wait complete. 2024-05-24T11:32:03.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:03 smithi039 ceph-mon[42919]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:03.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:03 smithi039 ceph-mon[42919]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:32:03.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:03.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:03.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:03.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:03.675 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-24T11:32:03.901 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:32:04.721 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:04 smithi039 ceph-mon[42919]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:32:04.722 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:04 smithi039 ceph-mon[42919]: Added host smithi137 2024-05-24T11:32:04.722 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:04 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:04.722 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:04 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:04.722 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:04 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:04.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.897+0000 7f5011534700 1 -- 172.21.15.39:0/1865036130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 msgr2=0x7f500c105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:04.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.897+0000 7f5011534700 1 --2- 172.21.15.39:0/1865036130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 0x7f500c105bc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009910 tx=0x7f4ff4009c20 comp rx=0 tx=0).stop 2024-05-24T11:32:04.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.897+0000 7f5011534700 1 -- 172.21.15.39:0/1865036130 shutdown_connections 2024-05-24T11:32:04.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.897+0000 7f5011534700 1 --2- 172.21.15.39:0/1865036130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 0x7f500c105bc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:04.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.897+0000 7f5011534700 1 -- 172.21.15.39:0/1865036130 >> 172.21.15.39:0/1865036130 conn(0x7f500c0fb310 msgr2=0x7f500c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:04.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.897+0000 7f5011534700 1 -- 172.21.15.39:0/1865036130 shutdown_connections 2024-05-24T11:32:04.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.897+0000 7f5011534700 1 -- 172.21.15.39:0/1865036130 wait complete. 2024-05-24T11:32:04.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.898+0000 7f5011534700 1 Processor -- start 2024-05-24T11:32:04.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.898+0000 7f5011534700 1 -- start start 2024-05-24T11:32:04.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.898+0000 7f5011534700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 0x7f500c0fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:04.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.898+0000 7f5011534700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0fd630 con 0x7f500c1057e0 2024-05-24T11:32:04.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.898+0000 7f500affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 0x7f500c0fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:04.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.898+0000 7f500affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 0x7f500c0fb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34288/0 (socket says 172.21.15.39:34288) 2024-05-24T11:32:04.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.898+0000 7f500affd700 1 -- 172.21.15.39:0/3012527070 learned_addr learned my addr 172.21.15.39:0/3012527070 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:32:04.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.899+0000 7f500affd700 1 -- 172.21.15.39:0/3012527070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40095f0 con 0x7f500c1057e0 2024-05-24T11:32:04.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.899+0000 7f500affd700 1 --2- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 0x7f500c0fb7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4005660 tx=0x7f4ff4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:04.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.899+0000 7f5003fff700 1 -- 172.21.15.39:0/3012527070 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ff401d070 con 0x7f500c1057e0 2024-05-24T11:32:04.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.899+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c0fbd00 con 0x7f500c1057e0 2024-05-24T11:32:04.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.899+0000 7f5003fff700 1 -- 172.21.15.39:0/3012527070 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ff400b6f0 con 0x7f500c1057e0 2024-05-24T11:32:04.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.899+0000 7f5003fff700 1 -- 172.21.15.39:0/3012527070 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ff40037f0 con 0x7f500c1057e0 2024-05-24T11:32:04.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.899+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c0fc180 con 0x7f500c1057e0 2024-05-24T11:32:04.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.900+0000 7f5003fff700 1 -- 172.21.15.39:0/3012527070 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f4ff4003ab0 con 0x7f500c1057e0 2024-05-24T11:32:04.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.900+0000 7f5003fff700 1 --2- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ff80387a0 0x7f4ff803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:04.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.900+0000 7f5003fff700 1 -- 172.21.15.39:0/3012527070 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4ff404c440 con 0x7f500c1057e0 2024-05-24T11:32:04.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.900+0000 7f500a7fc700 1 --2- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ff80387a0 0x7f4ff803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:04.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.900+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c100f20 con 0x7f500c1057e0 2024-05-24T11:32:04.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.900+0000 7f500a7fc700 1 --2- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ff80387a0 0x7f4ff803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc006fd0 tx=0x7f4ffc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:04.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:04.905+0000 7f5003fff700 1 -- 172.21.15.39:0/3012527070 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4ff4026020 con 0x7f500c1057e0 2024-05-24T11:32:05.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.114+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f500c0fccf0 con 0x7f4ff80387a0 2024-05-24T11:32:05.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.115+0000 7f5003fff700 1 -- 172.21.15.39:0/3012527070 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f500c0fccf0 con 0x7f4ff80387a0 2024-05-24T11:32:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:32:05.117 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ff80387a0 msgr2=0x7f4ff803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 --2- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ff80387a0 0x7f4ff803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc006fd0 tx=0x7f4ffc006e40 comp rx=0 tx=0).stop 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 msgr2=0x7f500c0fb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 --2- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 0x7f500c0fb7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4005660 tx=0x7f4ff4005740 comp rx=0 tx=0).stop 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 shutdown_connections 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 --2- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ff80387a0 0x7f4ff803ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 --2- 172.21.15.39:0/3012527070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f500c1057e0 0x7f500c0fb7c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 >> 172.21.15.39:0/3012527070 conn(0x7f500c0fb310 msgr2=0x7f500c068ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 shutdown_connections 2024-05-24T11:32:05.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:05.117+0000 7f5011534700 1 -- 172.21.15.39:0/3012527070 wait complete. 2024-05-24T11:32:05.501 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-24T11:32:05.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-24T11:32:05.724 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:32:06.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:05 smithi039 ceph-mon[42919]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:06.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:06.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.706+0000 7ff512e1d700 1 -- 172.21.15.39:0/3640079220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 msgr2=0x7ff50c105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:06.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.706+0000 7ff512e1d700 1 --2- 172.21.15.39:0/3640079220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 0x7ff50c105c00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff500009910 tx=0x7ff500009c20 comp rx=0 tx=0).stop 2024-05-24T11:32:06.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.706+0000 7ff512e1d700 1 -- 172.21.15.39:0/3640079220 shutdown_connections 2024-05-24T11:32:06.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.706+0000 7ff512e1d700 1 --2- 172.21.15.39:0/3640079220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 0x7ff50c105c00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:06.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.706+0000 7ff512e1d700 1 -- 172.21.15.39:0/3640079220 >> 172.21.15.39:0/3640079220 conn(0x7ff50c075350 msgr2=0x7ff50c075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:06.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.706+0000 7ff512e1d700 1 -- 172.21.15.39:0/3640079220 shutdown_connections 2024-05-24T11:32:06.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.706+0000 7ff512e1d700 1 -- 172.21.15.39:0/3640079220 wait complete. 2024-05-24T11:32:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.707+0000 7ff512e1d700 1 Processor -- start 2024-05-24T11:32:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.707+0000 7ff512e1d700 1 -- start start 2024-05-24T11:32:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.707+0000 7ff512e1d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 0x7ff50c196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.707+0000 7ff512e1d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c069090 con 0x7ff50c105820 2024-05-24T11:32:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff510bb9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 0x7ff50c196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff510bb9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 0x7ff50c196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34316/0 (socket says 172.21.15.39:34316) 2024-05-24T11:32:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff510bb9700 1 -- 172.21.15.39:0/1238516684 learned_addr learned my addr 172.21.15.39:0/1238516684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:32:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff510bb9700 1 -- 172.21.15.39:0/1238516684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5000095f0 con 0x7ff50c105820 2024-05-24T11:32:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff510bb9700 1 --2- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 0x7ff50c196f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff5000055f0 tx=0x7ff5000056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:06.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff509ffb700 1 -- 172.21.15.39:0/1238516684 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff50001d070 con 0x7ff50c105820 2024-05-24T11:32:06.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff509ffb700 1 -- 172.21.15.39:0/1238516684 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff50000b750 con 0x7ff50c105820 2024-05-24T11:32:06.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c197510 con 0x7ff50c105820 2024-05-24T11:32:06.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff509ffb700 1 -- 172.21.15.39:0/1238516684 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5000037b0 con 0x7ff50c105820 2024-05-24T11:32:06.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.708+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c197990 con 0x7ff50c105820 2024-05-24T11:32:06.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.709+0000 7ff509ffb700 1 -- 172.21.15.39:0/1238516684 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7ff500004390 con 0x7ff50c105820 2024-05-24T11:32:06.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.709+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c100ef0 con 0x7ff50c105820 2024-05-24T11:32:06.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.709+0000 7ff509ffb700 1 --2- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff4fc0387a0 0x7ff4fc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:06.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.710+0000 7ff509ffb700 1 -- 172.21.15.39:0/1238516684 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff50004c4a0 con 0x7ff50c105820 2024-05-24T11:32:06.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.710+0000 7ff50bfff700 1 --2- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff4fc0387a0 0x7ff4fc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:06.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.710+0000 7ff50bfff700 1 --2- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff4fc0387a0 0x7ff4fc03ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8006fd0 tx=0x7ff4f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:06.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.714+0000 7ff509ffb700 1 -- 172.21.15.39:0/1238516684 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff500029360 con 0x7ff50c105820 2024-05-24T11:32:06.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:06 smithi039 ceph-mon[42919]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:32:06.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:06.926+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff50c197bd0 con 0x7ff50c105820 2024-05-24T11:32:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.765+0000 7ff509ffb700 1 -- 172.21.15.39:0/1238516684 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff500026020 con 0x7ff50c105820 2024-05-24T11:32:07.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff4fc0387a0 msgr2=0x7ff4fc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:07.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 --2- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff4fc0387a0 0x7ff4fc03ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8006fd0 tx=0x7ff4f8006e40 comp rx=0 tx=0).stop 2024-05-24T11:32:07.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 msgr2=0x7ff50c196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:07.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 --2- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 0x7ff50c196f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff5000055f0 tx=0x7ff5000056d0 comp rx=0 tx=0).stop 2024-05-24T11:32:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 shutdown_connections 2024-05-24T11:32:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 --2- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff4fc0387a0 0x7ff4fc03ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 --2- 172.21.15.39:0/1238516684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c105820 0x7ff50c196f70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 >> 172.21.15.39:0/1238516684 conn(0x7ff50c075350 msgr2=0x7ff50c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 shutdown_connections 2024-05-24T11:32:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:07.768+0000 7ff512e1d700 1 -- 172.21.15.39:0/1238516684 wait complete. 2024-05-24T11:32:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-05-24T11:32:08.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:07 smithi039 ceph-mon[42919]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:08.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:07 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1238516684' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-24T11:32:08.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:07 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:08.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:07 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:08.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:07 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:08.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:07 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:08.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:07 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:32:08.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:07 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:08.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:07 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:08.184 INFO:tasks.cephadm:Adding mon.a on smithi039 2024-05-24T11:32:08.184 INFO:tasks.cephadm:Adding mon.b on smithi053 2024-05-24T11:32:08.184 INFO:tasks.cephadm:Adding mon.c on smithi137 2024-05-24T11:32:08.184 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi039:172.21.15.39=a;smithi053:172.21.15.53=b;smithi137:172.21.15.137=c' 2024-05-24T11:32:08.427 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:09.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:08 smithi039 ceph-mon[42919]: Updating smithi137:/etc/ceph/ceph.conf 2024-05-24T11:32:09.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:08 smithi039 ceph-mon[42919]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:09.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:08 smithi039 ceph-mon[42919]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:32:09.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:08 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1238516684' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-24T11:32:09.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:08 smithi039 ceph-mon[42919]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:09.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:09.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:09.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:09.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.341+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/3457846934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 msgr2=0x7f2ba4105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:09.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.341+0000 7f2ba8dbe700 1 --2- 172.21.15.137:0/3457846934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 0x7f2ba4105c00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009910 tx=0x7f2b8c009c20 comp rx=0 tx=0).stop 2024-05-24T11:32:09.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.341+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/3457846934 shutdown_connections 2024-05-24T11:32:09.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.341+0000 7f2ba8dbe700 1 --2- 172.21.15.137:0/3457846934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 0x7f2ba4105c00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:09.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.341+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/3457846934 >> 172.21.15.137:0/3457846934 conn(0x7f2ba4075350 msgr2=0x7f2ba4075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:09.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.341+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/3457846934 shutdown_connections 2024-05-24T11:32:09.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.341+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/3457846934 wait complete. 2024-05-24T11:32:09.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.342+0000 7f2ba8dbe700 1 Processor -- start 2024-05-24T11:32:09.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.342+0000 7f2ba8dbe700 1 -- start start 2024-05-24T11:32:09.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.342+0000 7f2ba8dbe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 0x7f2ba4195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:09.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.342+0000 7f2ba8dbe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba4195560 con 0x7f2ba4105820 2024-05-24T11:32:09.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.342+0000 7f2ba259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 0x7f2ba4195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:09.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.342+0000 7f2ba259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 0x7f2ba4195020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.137:41700/0 (socket says 172.21.15.137:41700) 2024-05-24T11:32:09.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.343+0000 7f2ba259c700 1 -- 172.21.15.137:0/2432123732 learned_addr learned my addr 172.21.15.137:0/2432123732 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:32:09.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.343+0000 7f2ba259c700 1 -- 172.21.15.137:0/2432123732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c0095f0 con 0x7f2ba4105820 2024-05-24T11:32:09.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.343+0000 7f2ba259c700 1 --2- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 0x7f2ba4195020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0055f0 tx=0x7f2b8c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:09.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.343+0000 7f2b9b7fe700 1 -- 172.21.15.137:0/2432123732 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b8c01d070 con 0x7f2ba4105820 2024-05-24T11:32:09.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.343+0000 7f2b9b7fe700 1 -- 172.21.15.137:0/2432123732 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b8c00b750 con 0x7f2ba4105820 2024-05-24T11:32:09.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.343+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba4195760 con 0x7f2ba4105820 2024-05-24T11:32:09.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.343+0000 7f2b9b7fe700 1 -- 172.21.15.137:0/2432123732 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b8c0037b0 con 0x7f2ba4105820 2024-05-24T11:32:09.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.343+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba4195b80 con 0x7f2ba4105820 2024-05-24T11:32:09.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.344+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba4100ef0 con 0x7f2ba4105820 2024-05-24T11:32:09.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.344+0000 7f2b9b7fe700 1 -- 172.21.15.137:0/2432123732 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f2b8c003a20 con 0x7f2ba4105820 2024-05-24T11:32:09.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.345+0000 7f2b9b7fe700 1 --2- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2b900387f0 0x7f2b9003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:09.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.345+0000 7f2b9b7fe700 1 -- 172.21.15.137:0/2432123732 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2b8c04c480 con 0x7f2ba4105820 2024-05-24T11:32:09.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.345+0000 7f2ba1d9b700 1 --2- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2b900387f0 0x7f2b9003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:09.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.345+0000 7f2ba1d9b700 1 --2- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2b900387f0 0x7f2b9003acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006fd0 tx=0x7f2b94006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:09.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.350+0000 7f2b9b7fe700 1 -- 172.21.15.137:0/2432123732 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2b8c029770 con 0x7f2ba4105820 2024-05-24T11:32:09.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.550+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi053:172.21.15.53=b;smithi137:172.21.15.137=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ba4109aa0 con 0x7f2b900387f0 2024-05-24T11:32:09.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.559+0000 7f2b9b7fe700 1 -- 172.21.15.137:0/2432123732 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2ba4109aa0 con 0x7f2b900387f0 2024-05-24T11:32:09.562 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mon update... 2024-05-24T11:32:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2b900387f0 msgr2=0x7f2b9003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 --2- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2b900387f0 0x7f2b9003acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006fd0 tx=0x7f2b94006e40 comp rx=0 tx=0).stop 2024-05-24T11:32:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 msgr2=0x7f2ba4195020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 --2- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 0x7f2ba4195020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0055f0 tx=0x7f2b8c0056d0 comp rx=0 tx=0).stop 2024-05-24T11:32:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 shutdown_connections 2024-05-24T11:32:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 --2- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2b900387f0 0x7f2b9003acb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 --2- 172.21.15.137:0/2432123732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ba4105820 0x7f2ba4195020 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 >> 172.21.15.137:0/2432123732 conn(0x7f2ba4075350 msgr2=0x7f2ba41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 shutdown_connections 2024-05-24T11:32:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:09.561+0000 7f2ba8dbe700 1 -- 172.21.15.137:0/2432123732 wait complete. 2024-05-24T11:32:09.968 DEBUG:teuthology.orchestra.run.smithi053:mon.b> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.b.service 2024-05-24T11:32:09.971 DEBUG:teuthology.orchestra.run.smithi137:mon.c> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.c.service 2024-05-24T11:32:09.973 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-24T11:32:09.973 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-24T11:32:10.005 INFO:journalctl@ceph.mon.c.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:32:10.010 INFO:journalctl@ceph.mon.b.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:32:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:09 smithi039 ceph-mon[42919]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:32:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:09 smithi039 ceph-mon[42919]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:10.294 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:11.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:10 smithi039 ceph-mon[42919]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi053:172.21.15.53=b;smithi137:172.21.15.137=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:11.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:10 smithi039 ceph-mon[42919]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi053:172.21.15.53=b;smithi137:172.21.15.137=c;count:3 2024-05-24T11:32:11.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:10 smithi039 ceph-mon[42919]: Deploying daemon mon.c on smithi137 2024-05-24T11:32:11.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.576+0000 7f51d2551700 1 -- 172.21.15.137:0/2082657911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 msgr2=0x7f51cc106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:11.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.576+0000 7f51d2551700 1 --2- 172.21.15.137:0/2082657911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 0x7f51cc106c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009910 tx=0x7f51b4009c20 comp rx=0 tx=0).stop 2024-05-24T11:32:11.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.576+0000 7f51d2551700 1 -- 172.21.15.137:0/2082657911 shutdown_connections 2024-05-24T11:32:11.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.576+0000 7f51d2551700 1 --2- 172.21.15.137:0/2082657911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 0x7f51cc106c90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:11.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.576+0000 7f51d2551700 1 -- 172.21.15.137:0/2082657911 >> 172.21.15.137:0/2082657911 conn(0x7f51cc0fc3a0 msgr2=0x7f51cc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:11.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.576+0000 7f51d2551700 1 -- 172.21.15.137:0/2082657911 shutdown_connections 2024-05-24T11:32:11.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.576+0000 7f51d2551700 1 -- 172.21.15.137:0/2082657911 wait complete. 2024-05-24T11:32:11.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.576+0000 7f51d2551700 1 Processor -- start 2024-05-24T11:32:11.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.576+0000 7f51d2551700 1 -- start start 2024-05-24T11:32:11.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.577+0000 7f51d2551700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 0x7f51cc196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:11.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.577+0000 7f51d2551700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc1965c0 con 0x7f51cc1068b0 2024-05-24T11:32:11.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.577+0000 7f51cbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 0x7f51cc196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:11.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.577+0000 7f51cbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 0x7f51cc196080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.137:41722/0 (socket says 172.21.15.137:41722) 2024-05-24T11:32:11.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.577+0000 7f51cbfff700 1 -- 172.21.15.137:0/3157732499 learned_addr learned my addr 172.21.15.137:0/3157732499 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:32:11.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.578+0000 7f51cbfff700 1 -- 172.21.15.137:0/3157732499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b40095f0 con 0x7f51cc1068b0 2024-05-24T11:32:11.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.578+0000 7f51cbfff700 1 --2- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 0x7f51cc196080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f51b40098e0 tx=0x7f51b4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:11.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.578+0000 7f51c97fa700 1 -- 172.21.15.137:0/3157732499 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51b401d070 con 0x7f51cc1068b0 2024-05-24T11:32:11.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.578+0000 7f51c97fa700 1 -- 172.21.15.137:0/3157732499 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51b4009e60 con 0x7f51cc1068b0 2024-05-24T11:32:11.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.578+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc196820 con 0x7f51cc1068b0 2024-05-24T11:32:11.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.578+0000 7f51c97fa700 1 -- 172.21.15.137:0/3157732499 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51b4022b30 con 0x7f51cc1068b0 2024-05-24T11:32:11.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.578+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc196ca0 con 0x7f51cc1068b0 2024-05-24T11:32:11.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.579+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51cc101ff0 con 0x7f51cc1068b0 2024-05-24T11:32:11.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.579+0000 7f51c97fa700 1 -- 172.21.15.137:0/3157732499 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f51b4022470 con 0x7f51cc1068b0 2024-05-24T11:32:11.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.579+0000 7f51c97fa700 1 --2- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51b80387a0 0x7f51b803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:11.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.579+0000 7f51c97fa700 1 -- 172.21.15.137:0/3157732499 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f51b404ccc0 con 0x7f51cc1068b0 2024-05-24T11:32:11.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.579+0000 7f51cb7fe700 1 --2- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51b80387a0 0x7f51b803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:11.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.580+0000 7f51cb7fe700 1 --2- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51b80387a0 0x7f51b803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51bc006fd0 tx=0x7f51bc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:11.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.584+0000 7f51c97fa700 1 -- 172.21.15.137:0/3157732499 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f51b402ba00 con 0x7f51cc1068b0 2024-05-24T11:32:11.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.887+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f51cc10aa60 con 0x7f51cc1068b0 2024-05-24T11:32:11.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.888+0000 7f51c97fa700 1 -- 172.21.15.137:0/3157732499 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f51b4027020 con 0x7f51cc1068b0 2024-05-24T11:32:11.892 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:32:11.892 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":1,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","modified":"2024-05-24T11:30:53.236187Z","created":"2024-05-24T11:30:53.236187Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-24T11:32:11.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51b80387a0 msgr2=0x7f51b803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 --2- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51b80387a0 0x7f51b803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51bc006fd0 tx=0x7f51bc006e40 comp rx=0 tx=0).stop 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 msgr2=0x7f51cc196080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 --2- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 0x7f51cc196080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f51b40098e0 tx=0x7f51b4005710 comp rx=0 tx=0).stop 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 shutdown_connections 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 --2- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51b80387a0 0x7f51b803ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 --2- 172.21.15.137:0/3157732499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51cc1068b0 0x7f51cc196080 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 >> 172.21.15.137:0/3157732499 conn(0x7f51cc0fc3a0 msgr2=0x7f51cc108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 shutdown_connections 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:11.890+0000 7f51d2551700 1 -- 172.21.15.137:0/3157732499 wait complete. 2024-05-24T11:32:11.893 INFO:teuthology.orchestra.run.smithi137.stderr:dumped monmap epoch 1 2024-05-24T11:32:12.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:11 smithi039 ceph-mon[42919]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:12.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 systemd[1]: Started Ceph mon.c for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:32:12.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: RocksDB version: 7.9.2 2024-05-24T11:32:12.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Git sha 0 2024-05-24T11:32:12.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Compile date 2024-05-19 15:03:31 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: DB SUMMARY 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: DB Session ID: 80L80TB7W7D21VXW5GOK 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: CURRENT file: CURRENT 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: IDENTITY file: IDENTITY 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.error_if_exists: 0 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.create_if_missing: 0 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.paranoid_checks: 1 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-24T11:32:12.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.env: 0x558679961820 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.fs: PosixFileSystem 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.info_log: 0x55867bbb1520 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_file_opening_threads: 16 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.statistics: (nil) 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.use_fsync: 0 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_log_file_size: 0 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.keep_log_file_num: 1000 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.recycle_log_file_num: 0 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.allow_fallocate: 1 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.allow_mmap_reads: 0 2024-05-24T11:32:12.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.allow_mmap_writes: 0 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.use_direct_reads: 0 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.create_missing_column_families: 0 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.db_log_dir: 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.wal_dir: 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.advise_random_on_open: 1 2024-05-24T11:32:12.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.db_write_buffer_size: 0 2024-05-24T11:32:12.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.write_buffer_manager: 0x55867ae46320 2024-05-24T11:32:12.750 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-24T11:32:12.750 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-24T11:32:12.750 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-24T11:32:12.750 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.rate_limiter: (nil) 2024-05-24T11:32:12.750 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-24T11:32:12.750 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.wal_recovery_mode: 2 2024-05-24T11:32:12.750 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.enable_thread_tracking: 0 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.enable_pipelined_write: 0 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.unordered_write: 0 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.row_cache: None 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.wal_filter: None 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.allow_ingest_behind: 0 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.two_write_queues: 0 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.manual_wal_flush: 0 2024-05-24T11:32:12.751 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.wal_compression: 0 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.atomic_flush: 0 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.log_readahead_size: 0 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.best_efforts_recovery: 0 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.allow_data_in_errors: 0 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.db_host_id: __hostname__ 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_background_jobs: 2 2024-05-24T11:32:12.752 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_background_compactions: -1 2024-05-24T11:32:12.753 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_subcompactions: 1 2024-05-24T11:32:12.754 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-24T11:32:12.754 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-24T11:32:12.754 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-24T11:32:12.754 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_total_wal_size: 0 2024-05-24T11:32:12.754 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-24T11:32:12.754 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-24T11:32:12.754 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-24T11:32:12.754 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_open_files: -1 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bytes_per_sync: 0 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_readahead_size: 0 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_background_flushes: -1 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Compression algorithms supported: 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: kZSTD supported: 0 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: kXpressCompression supported: 0 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: kLZ4HCCompression supported: 1 2024-05-24T11:32:12.755 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: kZlibCompression supported: 1 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: kSnappyCompression supported: 1 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: kLZ4Compression supported: 1 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: kBZip2Compression supported: 0 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.merge_operator: 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_filter: None 2024-05-24T11:32:12.756 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_filter_factory: None 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.sst_partitioner_factory: None 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55867bbb1620) 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout: cache_index_and_filter_blocks: 1 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout: pin_top_level_index_and_filter: 1 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout: index_type: 0 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout: data_block_index_type: 0 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout: index_shortening: 1 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-24T11:32:12.757 INFO:journalctl@ceph.mon.c.smithi137.stdout: checksum: 4 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: no_block_cache: 0 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_cache: 0x55867aec91f0 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_cache_name: BinnedLRUCache 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_cache_options: 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: capacity : 536870912 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: num_shard_bits : 4 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: strict_capacity_limit : 0 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: high_pri_pool_ratio: 0.000 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_cache_compressed: (nil) 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: persistent_cache: (nil) 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_size: 4096 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_size_deviation: 10 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_restart_interval: 16 2024-05-24T11:32:12.758 INFO:journalctl@ceph.mon.c.smithi137.stdout: index_block_restart_interval: 1 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: metadata_block_size: 4096 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: partition_filters: 0 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: use_delta_encoding: 1 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: filter_policy: bloomfilter 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: whole_key_filtering: 1 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: verify_compression: 0 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: read_amp_bytes_per_bit: 0 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: format_version: 5 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: enable_index_compression: 1 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_align: 0 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: max_auto_readahead_size: 262144 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: prepopulate_block_cache: 0 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: initial_auto_readahead_size: 8192 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout: num_file_reads_for_auto_readahead: 2 2024-05-24T11:32:12.759 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.write_buffer_size: 33554432 2024-05-24T11:32:12.760 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_write_buffer_number: 2 2024-05-24T11:32:12.760 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression: NoCompression 2024-05-24T11:32:12.760 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression: Disabled 2024-05-24T11:32:12.760 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.prefix_extractor: nullptr 2024-05-24T11:32:12.760 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-24T11:32:12.760 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.num_levels: 7 2024-05-24T11:32:12.760 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-24T11:32:12.760 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-24T11:32:12.760 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression_opts.level: 32767 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression_opts.strategy: 0 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-24T11:32:12.761 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression_opts.enabled: false 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.target_file_size_base: 67108864 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-24T11:32:12.762 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.arena_block_size: 1048576 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.disable_auto_compactions: 0 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-24T11:32:12.763 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.table_properties_collectors: 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.inplace_update_support: 0 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-24T11:32:12.764 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.bloom_locality: 0 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.max_successive_merges: 0 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.paranoid_file_checks: 0 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.force_consistency_checks: 1 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.report_bg_io_stats: 0 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.ttl: 2592000 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.enable_blob_files: false 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.min_blob_size: 0 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.blob_file_size: 268435456 2024-05-24T11:32:12.765 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.blob_file_starting_level: 0 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 37e4c03d-f10f-46d0-93a4-a68440bb95f0 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716550332716396, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716550332716826, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716550332, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "37e4c03d-f10f-46d0-93a4-a68440bb95f0", "db_session_id": "80L80TB7W7D21VXW5GOK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-24T11:32:12.766 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716550332716893, "job": 1, "event": "recovery_finished"} 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55867af66000 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: DB pointer 0x55867af54000 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout: ** DB Stats ** 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-24T11:32:12.767 INFO:journalctl@ceph.mon.c.smithi137.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-24T11:32:12.768 INFO:journalctl@ceph.mon.c.smithi137.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-24T11:32:12.768 INFO:journalctl@ceph.mon.c.smithi137.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-24T11:32:12.768 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-05-24T11:32:12.768 INFO:journalctl@ceph.mon.c.smithi137.stdout: ** Compaction Stats [default] ** 2024-05-24T11:32:12.768 INFO:journalctl@ceph.mon.c.smithi137.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: ** Compaction Stats [default] ** 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-24T11:32:12.769 INFO:journalctl@ceph.mon.c.smithi137.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: Block cache BinnedLRUCache@0x55867aec91f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: using public_addr v2:172.21.15.137:0/0 -> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-05-24T11:32:12.770 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: starting mon.c rank -1 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(???) e0 preinit fsid f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).mds e1 new map 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout: e1 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout: legacy client fscid: -1 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout: No filesystems configured 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-24T11:32:12.771 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-24T11:32:12.772 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-24T11:32:12.772 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-24T11:32:12.772 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-24T11:32:12.772 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-24T11:32:12.772 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-24T11:32:12.772 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-24T11:32:12.772 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:12.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:12 smithi137 ceph-mon[47907]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-24T11:32:13.270 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-24T11:32:13.271 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-24T11:32:13.497 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:32:18.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.068+0000 7f9676cc0700 1 -- 172.21.15.137:0/4192335986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96600035c0 msgr2=0x7f9660005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:18.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.068+0000 7f9676cc0700 1 --2- 172.21.15.137:0/4192335986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96600035c0 0x7f9660005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9670100920 tx=0x7f9664014ec0 comp rx=0 tx=0).stop 2024-05-24T11:32:18.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.068+0000 7f9676cc0700 1 -- 172.21.15.137:0/4192335986 shutdown_connections 2024-05-24T11:32:18.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.068+0000 7f9676cc0700 1 --2- 172.21.15.137:0/4192335986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96600035c0 0x7f9660005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:18.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.068+0000 7f9676cc0700 1 -- 172.21.15.137:0/4192335986 >> 172.21.15.137:0/4192335986 conn(0x7f96700fc500 msgr2=0x7f96700fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:18.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.068+0000 7f9676cc0700 1 -- 172.21.15.137:0/4192335986 shutdown_connections 2024-05-24T11:32:18.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.068+0000 7f9676cc0700 1 -- 172.21.15.137:0/4192335986 wait complete. 2024-05-24T11:32:18.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f9676cc0700 1 Processor -- start 2024-05-24T11:32:18.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f9676cc0700 1 -- start start 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f9676cc0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 0x7f9670196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f9676cc0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96701966d0 0x7f967019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f9676cc0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9670196c60 con 0x7f96701966d0 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f9676cc0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9670196dd0 con 0x7f96600035c0 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f9674a5c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 0x7f9670196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f9674a5c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 0x7f9670196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39924/0 (socket says 172.21.15.137:39924) 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f9674a5c700 1 -- 172.21.15.137:0/931521701 learned_addr learned my addr 172.21.15.137:0/931521701 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.069+0000 7f966ffff700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96701966d0 0x7f967019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f9674a5c700 1 -- 172.21.15.137:0/931521701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 msgr2=0x7f9670196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f9674a5c700 1 -- 172.21.15.137:0/931521701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 msgr2=0x7f9670196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-24T11:32:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f9674a5c700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 0x7f9670196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f9674a5c700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 0x7f9670196190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f966ffff700 1 -- 172.21.15.137:0/931521701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 msgr2=0x7f9670196190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f966ffff700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 0x7f9670196190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f966ffff700 1 -- 172.21.15.137:0/931521701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664014910 con 0x7f96701966d0 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f966ffff700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96701966d0 0x7f967019aa80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b4d0 tx=0x7f965c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f966dffb700 1 -- 172.21.15.137:0/931521701 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f965c011630 con 0x7f96701966d0 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967019b020 con 0x7f96701966d0 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f966dffb700 1 -- 172.21.15.137:0/931521701 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f965c011c20 con 0x7f96701966d0 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.070+0000 7f966dffb700 1 -- 172.21.15.137:0/931521701 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f965c010c60 con 0x7f96701966d0 2024-05-24T11:32:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.071+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967019b570 con 0x7f96701966d0 2024-05-24T11:32:18.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.071+0000 7f966dffb700 1 -- 172.21.15.137:0/931521701 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f965c01a400 con 0x7f96701966d0 2024-05-24T11:32:18.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.072+0000 7f966dffb700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f96540385e0 0x7f965403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:18.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.072+0000 7f966dffb700 1 -- 172.21.15.137:0/931521701 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f965c04aeb0 con 0x7f96701966d0 2024-05-24T11:32:18.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.072+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967018ff10 con 0x7f96701966d0 2024-05-24T11:32:18.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.072+0000 7f9674a5c700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f96540385e0 0x7f965403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:18.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.073+0000 7f9674a5c700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f96540385e0 0x7f965403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9664005d10 tx=0x7f9664005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:18.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.078+0000 7f966dffb700 1 -- 172.21.15.137:0/931521701 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f965c010dc0 con 0x7f96701966d0 2024-05-24T11:32:18.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:18.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: mon.a calling monitor election 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: mon.c calling monitor election 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-24T11:32:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: monmap e2: 2 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],c=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:32:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: fsmap 2024-05-24T11:32:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: mgrmap e13: x(active, since 35s) 2024-05-24T11:32:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: overall HEALTH_OK 2024-05-24T11:32:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:17 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:18.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:18.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:32:18.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: mon.a calling monitor election 2024-05-24T11:32:18.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:18.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: mon.c calling monitor election 2024-05-24T11:32:18.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:18.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:18.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-24T11:32:18.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: monmap e2: 2 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],c=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:32:18.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: fsmap 2024-05-24T11:32:18.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:18.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: mgrmap e13: x(active, since 35s) 2024-05-24T11:32:18.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: overall HEALTH_OK 2024-05-24T11:32:18.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:18.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:18.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:18.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:18.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:17 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:18.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.350+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f96700625d0 con 0x7f96701966d0 2024-05-24T11:32:18.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.351+0000 7f966dffb700 1 -- 172.21.15.137:0/931521701 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f965c0172c0 con 0x7f96701966d0 2024-05-24T11:32:18.354 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:32:18.354 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":2,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","modified":"2024-05-24T11:32:12.745534Z","created":"2024-05-24T11:30:53.236187Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-24T11:32:18.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f96540385e0 msgr2=0x7f965403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:18.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f96540385e0 0x7f965403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9664005d10 tx=0x7f9664005c80 comp rx=0 tx=0).stop 2024-05-24T11:32:18.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96701966d0 msgr2=0x7f967019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:18.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96701966d0 0x7f967019aa80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b4d0 tx=0x7f965c00b890 comp rx=0 tx=0).stop 2024-05-24T11:32:18.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 shutdown_connections 2024-05-24T11:32:18.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f96540385e0 0x7f965403aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:18.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96600035c0 0x7f9670196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:18.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 --2- 172.21.15.137:0/931521701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96701966d0 0x7f967019aa80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:18.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 >> 172.21.15.137:0/931521701 conn(0x7f96700fc500 msgr2=0x7f96700fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:18.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 shutdown_connections 2024-05-24T11:32:18.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:18.353+0000 7f9676cc0700 1 -- 172.21.15.137:0/931521701 wait complete. 2024-05-24T11:32:18.357 INFO:teuthology.orchestra.run.smithi137.stderr:dumped monmap epoch 2 2024-05-24T11:32:18.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:18 smithi137 ceph-mon[47907]: Deploying daemon mon.b on smithi053 2024-05-24T11:32:18.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:18 smithi137 ceph-mon[47907]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:18.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:18 smithi137 ceph-mon[47907]: from='client.? 172.21.15.137:0/931521701' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-24T11:32:18.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:18 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:19.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:18 smithi039 ceph-mon[42919]: Deploying daemon mon.b on smithi053 2024-05-24T11:32:19.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:18 smithi039 ceph-mon[42919]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:19.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:18 smithi039 ceph-mon[42919]: from='client.? 172.21.15.137:0/931521701' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-24T11:32:19.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:18 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:19.679 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-24T11:32:19.679 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-24T11:32:19.908 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:32:20.077 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:32:19 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:32:19.743+0000 7fb8b5aa0700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-24T11:32:20.637 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 systemd[1]: Started Ceph mon.b for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).mds e1 new map 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout: e1 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout: legacy client fscid: -1 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout: 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout: No filesystems configured 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-24T11:32:20.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-24T11:32:20.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-24T11:32:20.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-24T11:32:20.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-24T11:32:20.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-24T11:32:20.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-24T11:32:20.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-24T11:32:20.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-24T11:32:20.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:20.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:20.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:32:20.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.a calling monitor election 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.c calling monitor election 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: monmap e2: 2 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],c=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: fsmap 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mgrmap e13: x(active, since 35s) 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: overall HEALTH_OK 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: Deploying daemon mon.b on smithi053 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='client.? 172.21.15.137:0/931521701' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:20 smithi053 ceph-mon[47448]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-24T11:32:25.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: mon.c calling monitor election 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: mon.a calling monitor election 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: fsmap 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: mgrmap e13: x(active, since 43s) 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: mon.b (rank 2) addr [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] is down (out of quorum) 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:25.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:25.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:26.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:26.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: mon.c calling monitor election 2024-05-24T11:32:26.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:32:26.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: mon.a calling monitor election 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: fsmap 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: mgrmap e13: x(active, since 43s) 2024-05-24T11:32:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-24T11:32:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-24T11:32:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-24T11:32:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: mon.b (rank 2) addr [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] is down (out of quorum) 2024-05-24T11:32:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:26.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.762+0000 7f44ad64c700 1 -- 172.21.15.137:0/3219355837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 msgr2=0x7f4490008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:26.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.762+0000 7f44ad64c700 1 --2- 172.21.15.137:0/3219355837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 0x7f4490008710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f44a8101a90 tx=0x7f449800fe90 comp rx=0 tx=0).stop 2024-05-24T11:32:26.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.762+0000 7f44ad64c700 1 -- 172.21.15.137:0/3219355837 shutdown_connections 2024-05-24T11:32:26.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.762+0000 7f44ad64c700 1 --2- 172.21.15.137:0/3219355837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 0x7f4490008710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:26.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.762+0000 7f44ad64c700 1 -- 172.21.15.137:0/3219355837 >> 172.21.15.137:0/3219355837 conn(0x7f44a80fc480 msgr2=0x7f44a80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:26.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.762+0000 7f44ad64c700 1 -- 172.21.15.137:0/3219355837 shutdown_connections 2024-05-24T11:32:26.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.762+0000 7f44ad64c700 1 -- 172.21.15.137:0/3219355837 wait complete. 2024-05-24T11:32:26.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.763+0000 7f44ad64c700 1 Processor -- start 2024-05-24T11:32:26.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.763+0000 7f44ad64c700 1 -- start start 2024-05-24T11:32:26.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.763+0000 7f44ad64c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 0x7f44a8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:26.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.763+0000 7f44ad64c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 0x7f44a819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:26.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.763+0000 7f44ad64c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a819b1c0 0x7f44a819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:26.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.763+0000 7f44ad64c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8196fc0 con 0x7f44900062a0 2024-05-24T11:32:26.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.763+0000 7f44ad64c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a81a2f30 con 0x7f44a819b1c0 2024-05-24T11:32:26.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.763+0000 7f44ad64c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a81a3070 con 0x7f44a81968d0 2024-05-24T11:32:26.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.763+0000 7f44a67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 0x7f44a819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:26.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 0x7f44a8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:26.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 0x7f44a8196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.137:59680/0 (socket says 172.21.15.137:59680) 2024-05-24T11:32:26.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a67fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 0x7f44a819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49408/0 (socket says 172.21.15.137:49408) 2024-05-24T11:32:26.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a6ffd700 1 -- 172.21.15.137:0/1427630272 learned_addr learned my addr 172.21.15.137:0/1427630272 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:32:26.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a77fe700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a819b1c0 0x7f44a819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:26.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a67fc700 1 -- 172.21.15.137:0/1427630272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 msgr2=0x7f44a819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-24T11:32:26.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a67fc700 1 -- 172.21.15.137:0/1427630272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 msgr2=0x7f44a819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-24T11:32:26.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a67fc700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 0x7f44a819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-24T11:32:26.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a67fc700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 0x7f44a819ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:32:26.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a77fe700 1 -- 172.21.15.137:0/1427630272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a819b1c0 msgr2=0x7f44a819b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-24T11:32:26.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a77fe700 1 -- 172.21.15.137:0/1427630272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a819b1c0 msgr2=0x7f44a819b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-24T11:32:26.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a77fe700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a819b1c0 0x7f44a819b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-24T11:32:26.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a77fe700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a819b1c0 0x7f44a819b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:32:26.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a6ffd700 1 -- 172.21.15.137:0/1427630272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 msgr2=0x7f44a819ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:32:26.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a6ffd700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 0x7f44a819ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:26.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a6ffd700 1 -- 172.21.15.137:0/1427630272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a819b1c0 msgr2=0x7f44a819b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:32:26.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a6ffd700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a819b1c0 0x7f44a819b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:26.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a6ffd700 1 -- 172.21.15.137:0/1427630272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449800f8e0 con 0x7f44900062a0 2024-05-24T11:32:26.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44a6ffd700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 0x7f44a8196390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4498010010 tx=0x7f449801dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:26.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f448ffff700 1 -- 172.21.15.137:0/1427630272 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498011440 con 0x7f44900062a0 2024-05-24T11:32:26.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a81a32f0 con 0x7f44900062a0 2024-05-24T11:32:26.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f448ffff700 1 -- 172.21.15.137:0/1427630272 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4498003680 con 0x7f44900062a0 2024-05-24T11:32:26.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f448ffff700 1 -- 172.21.15.137:0/1427630272 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498011870 con 0x7f44900062a0 2024-05-24T11:32:26.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.764+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a81a37f0 con 0x7f44900062a0 2024-05-24T11:32:26.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.765+0000 7f448ffff700 1 -- 172.21.15.137:0/1427630272 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f44980119d0 con 0x7f44900062a0 2024-05-24T11:32:26.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.765+0000 7f448ffff700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4494038c20 0x7f449403b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:26.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.765+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a8102110 con 0x7f44900062a0 2024-05-24T11:32:26.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.766+0000 7f448ffff700 1 -- 172.21.15.137:0/1427630272 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4498000e40 con 0x7f44900062a0 2024-05-24T11:32:26.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.766+0000 7f44a67fc700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4494038c20 0x7f449403b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:26.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.766+0000 7f44a67fc700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4494038c20 0x7f449403b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f449c009850 tx=0x7f449c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:26.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:26.771+0000 7f448ffff700 1 -- 172.21.15.137:0/1427630272 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f449802f360 con 0x7f44900062a0 2024-05-24T11:32:26.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:26.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:26.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:26.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:26.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:26.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:26.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:26.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:27.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:27.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:27.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.087+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f44a80625d0 con 0x7f44900062a0 2024-05-24T11:32:27.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.087+0000 7f448ffff700 1 -- 172.21.15.137:0/1427630272 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7f4498037500 con 0x7f44900062a0 2024-05-24T11:32:27.090 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-05-24T11:32:27.090 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":3,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","modified":"2024-05-24T11:32:20.638678Z","created":"2024-05-24T11:30:53.236187Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4494038c20 msgr2=0x7f449403b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4494038c20 0x7f449403b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f449c009850 tx=0x7f449c008040 comp rx=0 tx=0).stop 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 msgr2=0x7f44a8196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 0x7f44a8196390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4498010010 tx=0x7f449801dae0 comp rx=0 tx=0).stop 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 shutdown_connections 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4494038c20 0x7f449403b0e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44a819b1c0 0x7f44a819b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44900062a0 0x7f44a8196390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 --2- 172.21.15.137:0/1427630272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a81968d0 0x7f44a819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 >> 172.21.15.137:0/1427630272 conn(0x7f44a80fc480 msgr2=0x7f44a81024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:27.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 shutdown_connections 2024-05-24T11:32:27.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:27.090+0000 7f44ad64c700 1 -- 172.21.15.137:0/1427630272 wait complete. 2024-05-24T11:32:27.094 INFO:teuthology.orchestra.run.smithi137.stderr:dumped monmap epoch 3 2024-05-24T11:32:27.461 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-24T11:32:27.461 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-24T11:32:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:32:27.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: mon.c calling monitor election 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: mon.a calling monitor election 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:27.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: fsmap 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: mgrmap e13: x(active, since 43s) 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: mon.b (rank 2) addr [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] is down (out of quorum) 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:28.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: mon.b calling monitor election 2024-05-24T11:32:28.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:28.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:28.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:28.417 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: Reconfiguring mon.a (unknown last config time)... 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: Reconfiguring daemon mon.a on smithi039 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: mon.b calling monitor election 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: mon.c calling monitor election 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: mon.a calling monitor election 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: fsmap 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: mgrmap e13: x(active, since 45s) 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: Cluster is now healthy 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: overall HEALTH_OK 2024-05-24T11:32:28.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:28.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:28.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:28.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-24T11:32:28.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:28 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:28.680 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: mon.b calling monitor election 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: Reconfiguring mon.a (unknown last config time)... 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: Reconfiguring daemon mon.a on smithi039 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: mon.b calling monitor election 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: mon.c calling monitor election 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: mon.a calling monitor election 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: fsmap 2024-05-24T11:32:28.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:28.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: mgrmap e13: x(active, since 45s) 2024-05-24T11:32:28.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-24T11:32:28.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: Cluster is now healthy 2024-05-24T11:32:28.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: overall HEALTH_OK 2024-05-24T11:32:28.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:28.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:28.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:28.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-24T11:32:28.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:28 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:28.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: mon.b calling monitor election 2024-05-24T11:32:28.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:28.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:28.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: Reconfiguring mon.a (unknown last config time)... 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: Reconfiguring daemon mon.a on smithi039 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: mon.b calling monitor election 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: mon.c calling monitor election 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: mon.a calling monitor election 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0],c=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: fsmap 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: osdmap e4: 0 total, 0 up, 0 in 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: mgrmap e13: x(active, since 45s) 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: Cluster is now healthy 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: overall HEALTH_OK 2024-05-24T11:32:28.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:28.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:28.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:28.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-24T11:32:28.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:28 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:29.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.084+0000 7f4f8bad1700 1 -- 172.21.15.39:0/2475062133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84105f00 msgr2=0x7f4f841062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:29.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.084+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/2475062133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84105f00 0x7f4f841062e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f800098f0 tx=0x7f4f80009c00 comp rx=0 tx=0).stop 2024-05-24T11:32:29.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.085+0000 7f4f8bad1700 1 -- 172.21.15.39:0/2475062133 shutdown_connections 2024-05-24T11:32:29.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.085+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/2475062133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f84100890 0x7f4f8410cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:29.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.085+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/2475062133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84105f00 0x7f4f841062e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:29.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.085+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/2475062133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f840ffed0 0x7f4f84100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:29.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.085+0000 7f4f8bad1700 1 -- 172.21.15.39:0/2475062133 >> 172.21.15.39:0/2475062133 conn(0x7f4f84075f80 msgr2=0x7f4f84076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:29.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.085+0000 7f4f8bad1700 1 -- 172.21.15.39:0/2475062133 shutdown_connections 2024-05-24T11:32:29.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.085+0000 7f4f8bad1700 1 -- 172.21.15.39:0/2475062133 wait complete. 2024-05-24T11:32:29.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.085+0000 7f4f8bad1700 1 Processor -- start 2024-05-24T11:32:29.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8bad1700 1 -- start start 2024-05-24T11:32:29.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8bad1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f840ffed0 0x7f4f84101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:29.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8bad1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f84100890 0x7f4f84102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:29.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8bad1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f84105f00 0x7f4f84102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:29.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8bad1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f841a56b0 con 0x7f4f840ffed0 2024-05-24T11:32:29.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8bad1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f841a5820 con 0x7f4f84100890 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8bad1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f841a59e0 con 0x7f4f84105f00 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8986d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f840ffed0 0x7f4f84101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8986d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f840ffed0 0x7f4f84101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38912/0 (socket says 172.21.15.39:38912) 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8986d700 1 -- 172.21.15.39:0/3779019442 learned_addr learned my addr 172.21.15.39:0/3779019442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8906c700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f84100890 0x7f4f84102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8a06e700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f84105f00 0x7f4f84102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8986d700 1 -- 172.21.15.39:0/3779019442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f84105f00 msgr2=0x7f4f84102a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8986d700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f84105f00 0x7f4f84102a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8986d700 1 -- 172.21.15.39:0/3779019442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f84100890 msgr2=0x7f4f84102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8986d700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f84100890 0x7f4f84102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8986d700 1 -- 172.21.15.39:0/3779019442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80009520 con 0x7f4f840ffed0 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8a06e700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f84105f00 0x7f4f84102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:32:29.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f8986d700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f840ffed0 0x7f4f84101fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000fa00 tx=0x7f4f80010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.086+0000 7f4f7affd700 1 -- 172.21.15.39:0/3779019442 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8001c070 con 0x7f4f840ffed0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.087+0000 7f4f7affd700 1 -- 172.21.15.39:0/3779019442 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f8000ba70 con 0x7f4f840ffed0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.087+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f841a5c60 con 0x7f4f840ffed0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.087+0000 7f4f7affd700 1 -- 172.21.15.39:0/3779019442 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80004930 con 0x7f4f840ffed0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.087+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f841a60d0 con 0x7f4f840ffed0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.087+0000 7f4f7affd700 1 -- 172.21.15.39:0/3779019442 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f4f80020070 con 0x7f4f840ffed0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.088+0000 7f4f7affd700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4f700389d0 0x7f4f7003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.088+0000 7f4f7affd700 1 -- 172.21.15.39:0/3779019442 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4f8004bdf0 con 0x7f4f840ffed0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.088+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f841015b0 con 0x7f4f840ffed0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.088+0000 7f4f8906c700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4f700389d0 0x7f4f7003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:29.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.088+0000 7f4f8906c700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4f700389d0 0x7f4f7003ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f84103710 tx=0x7f4f74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:29.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.093+0000 7f4f7affd700 1 -- 172.21.15.39:0/3779019442 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4f80027020 con 0x7f4f840ffed0 2024-05-24T11:32:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.296+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4f84103560 con 0x7f4f840ffed0 2024-05-24T11:32:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.296+0000 7f4f7affd700 1 -- 172.21.15.39:0/3779019442 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f4f80027130 con 0x7f4f840ffed0 2024-05-24T11:32:29.300 INFO:teuthology.orchestra.run.smithi039.stdout:# minimal ceph.conf for f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:32:29.300 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-05-24T11:32:29.300 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:32:29.301 INFO:teuthology.orchestra.run.smithi039.stdout: mon_host = [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-05-24T11:32:29.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4f700389d0 msgr2=0x7f4f7003ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:29.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4f700389d0 0x7f4f7003ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f84103710 tx=0x7f4f74009260 comp rx=0 tx=0).stop 2024-05-24T11:32:29.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f840ffed0 msgr2=0x7f4f84101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:29.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f840ffed0 0x7f4f84101fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000fa00 tx=0x7f4f80010f80 comp rx=0 tx=0).stop 2024-05-24T11:32:29.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 shutdown_connections 2024-05-24T11:32:29.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4f700389d0 0x7f4f7003ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:29.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f84105f00 0x7f4f84102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:29.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f840ffed0 0x7f4f84101fe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:29.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 --2- 172.21.15.39:0/3779019442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f84100890 0x7f4f84102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:29.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 >> 172.21.15.39:0/3779019442 conn(0x7f4f84075f80 msgr2=0x7f4f841043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:29.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 shutdown_connections 2024-05-24T11:32:29.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:29.298+0000 7f4f8bad1700 1 -- 172.21.15.39:0/3779019442 wait complete. 2024-05-24T11:32:29.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:29.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: Reconfiguring mon.b (monmap changed)... 2024-05-24T11:32:29.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: Reconfiguring daemon mon.b on smithi053 2024-05-24T11:32:29.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:29.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:29.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:29.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:29.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-24T11:32:29.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:29.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:29 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3779019442' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:29.673 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-24T11:32:29.673 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:32:29.673 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.conf 2024-05-24T11:32:29.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:29.704 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: Reconfiguring mon.b (monmap changed)... 2024-05-24T11:32:29.704 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: Reconfiguring daemon mon.b on smithi053 2024-05-24T11:32:29.704 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:29.704 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:29.704 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:29.704 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:29.704 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-24T11:32:29.704 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:29.704 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:29 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3779019442' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:29.705 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:32:29.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:32:29.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:29.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: Reconfiguring mon.b (monmap changed)... 2024-05-24T11:32:29.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: Reconfiguring daemon mon.b on smithi053 2024-05-24T11:32:29.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:32:29.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:29.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:29.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-24T11:32:29.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-24T11:32:29.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:29.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:29 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3779019442' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:29.773 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:32:29.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-05-24T11:32:29.813 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:32:29.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:32:29.890 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:32:29.890 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-05-24T11:32:29.921 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:32:29.921 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:32:29.989 INFO:tasks.cephadm:Adding mgr.x on smithi039 2024-05-24T11:32:29.989 INFO:tasks.cephadm:Adding mgr.y on smithi053 2024-05-24T11:32:29.989 INFO:tasks.cephadm:Adding mgr.z on smithi137 2024-05-24T11:32:29.989 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi039=x;smithi053=y;smithi137=z' 2024-05-24T11:32:30.245 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:32:30.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:30 smithi137 ceph-mon[47907]: Reconfiguring mon.c (monmap changed)... 2024-05-24T11:32:30.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:30 smithi137 ceph-mon[47907]: Reconfiguring daemon mon.c on smithi137 2024-05-24T11:32:30.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:30.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:30.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:30.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:30.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:30.631 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:30.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:30 smithi053 ceph-mon[47448]: Reconfiguring mon.c (monmap changed)... 2024-05-24T11:32:30.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:30 smithi053 ceph-mon[47448]: Reconfiguring daemon mon.c on smithi137 2024-05-24T11:32:30.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:30.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:30.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:30.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:30.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:30.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:30.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:30 smithi039 ceph-mon[42919]: Reconfiguring mon.c (monmap changed)... 2024-05-24T11:32:30.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:30 smithi039 ceph-mon[42919]: Reconfiguring daemon mon.c on smithi137 2024-05-24T11:32:30.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:30.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:30.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:30.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:30.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:30.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.144+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1390401297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 msgr2=0x7f1de01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.144+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1390401297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 0x7f1de01014e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009990 tx=0x7f1dd4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1390401297 shutdown_connections 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1390401297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de0101a20 0x7f1de0108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1390401297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de00fff00 0x7f1de0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1390401297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 0x7f1de01014e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1390401297 >> 172.21.15.137:0/1390401297 conn(0x7f1de00fb4c0 msgr2=0x7f1de00fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1390401297 shutdown_connections 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1390401297 wait complete. 2024-05-24T11:32:31.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 Processor -- start 2024-05-24T11:32:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 -- start start 2024-05-24T11:32:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de00fff00 0x7f1de0069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.145+0000 7f1de5b5c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 0x7f1de019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1de5b5c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de0101a20 0x7f1de019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1de5b5c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019e8c0 con 0x7f1de00fff00 2024-05-24T11:32:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1de5b5c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019ea30 con 0x7f1de0101a20 2024-05-24T11:32:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1de5b5c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019ebf0 con 0x7f1de0101060 2024-05-24T11:32:31.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 0x7f1de019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 0x7f1de019bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52704/0 (socket says 172.21.15.137:52704) 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddeffd700 1 -- 172.21.15.137:0/1574666274 learned_addr learned my addr 172.21.15.137:0/1574666274 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddf7fe700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de00fff00 0x7f1de0069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddeffd700 1 -- 172.21.15.137:0/1574666274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de0101a20 msgr2=0x7f1de019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddeffd700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de0101a20 0x7f1de019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddeffd700 1 -- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de00fff00 msgr2=0x7f1de0069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddeffd700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de00fff00 0x7f1de0069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddeffd700 1 -- 172.21.15.137:0/1574666274 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd40095f0 con 0x7f1de0101060 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddf7fe700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de00fff00 0x7f1de0069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddeffd700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 0x7f1de019bb90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4000c00 tx=0x7f1dd4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddcff9700 1 -- 172.21.15.137:0/1574666274 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd401d070 con 0x7f1de0101060 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddcff9700 1 -- 172.21.15.137:0/1574666274 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dd4004100 con 0x7f1de0101060 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de019ee70 con 0x7f1de0101060 2024-05-24T11:32:31.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1ddcff9700 1 -- 172.21.15.137:0/1574666274 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4022e40 con 0x7f1de0101060 2024-05-24T11:32:31.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.146+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de019f2e0 con 0x7f1de0101060 2024-05-24T11:32:31.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.147+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de0107420 con 0x7f1de0101060 2024-05-24T11:32:31.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.148+0000 7f1ddcff9700 1 -- 172.21.15.137:0/1574666274 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f1dd4003c70 con 0x7f1de0101060 2024-05-24T11:32:31.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.148+0000 7f1ddcff9700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1dc80389d0 0x7f1dc803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:31.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.148+0000 7f1ddcff9700 1 -- 172.21.15.137:0/1574666274 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1dd404cde0 con 0x7f1de0101060 2024-05-24T11:32:31.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.148+0000 7f1ddf7fe700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1dc80389d0 0x7f1dc803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:31.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.149+0000 7f1ddf7fe700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1dc80389d0 0x7f1dc803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0005950 tx=0x7f1dd0005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:31.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.152+0000 7f1ddcff9700 1 -- 172.21.15.137:0/1574666274 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1dd400ee40 con 0x7f1de0101060 2024-05-24T11:32:31.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.354+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi053=y;smithi137=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f1de002ce10 con 0x7f1dc80389d0 2024-05-24T11:32:31.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.362+0000 7f1ddcff9700 1 -- 172.21.15.137:0/1574666274 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1de002ce10 con 0x7f1dc80389d0 2024-05-24T11:32:31.366 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mgr update... 2024-05-24T11:32:31.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.364+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1dc80389d0 msgr2=0x7f1dc803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.364+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1dc80389d0 0x7f1dc803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0005950 tx=0x7f1dd0005f50 comp rx=0 tx=0).stop 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.364+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 msgr2=0x7f1de019bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.364+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 0x7f1de019bb90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4000c00 tx=0x7f1dd4005ef0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.365+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 shutdown_connections 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.365+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1dc80389d0 0x7f1dc803ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.365+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1de0101a20 0x7f1de019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.365+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1de00fff00 0x7f1de0069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.365+0000 7f1de5b5c700 1 --2- 172.21.15.137:0/1574666274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1de0101060 0x7f1de019bb90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.365+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 >> 172.21.15.137:0/1574666274 conn(0x7f1de00fb4c0 msgr2=0x7f1de0104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.365+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 shutdown_connections 2024-05-24T11:32:31.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:32:31.365+0000 7f1de5b5c700 1 -- 172.21.15.137:0/1574666274 wait complete. 2024-05-24T11:32:31.642 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:31 smithi137 ceph-mon[47907]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:31.642 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:31 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:31.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:31 smithi053 ceph-mon[47448]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:31.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:31 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:31.766 DEBUG:teuthology.orchestra.run.smithi053:mgr.y> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-24T11:32:31.769 DEBUG:teuthology.orchestra.run.smithi137:mgr.z> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-24T11:32:31.771 INFO:tasks.cephadm:Deploying OSDs... 2024-05-24T11:32:31.773 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:32:31.773 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-05-24T11:32:31.795 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-24T11:32:31.795 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-05-24T11:32:31.800 INFO:journalctl@ceph.mgr.z.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:32:31.804 INFO:journalctl@ceph.mgr.y.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:32:31.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:31 smithi039 ceph-mon[42919]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:31.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:31 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:31.845 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-24T11:32:31.846 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:31.846 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 182047 Links: 1 2024-05-24T11:32:31.846 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:31.846 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:31.846 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-24 11:31:51.392520056 +0000 2024-05-24T11:32:31.846 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-24 11:24:59.017920260 +0000 2024-05-24T11:32:31.846 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-24 11:24:59.017920260 +0000 2024-05-24T11:32:31.846 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-05-24T11:32:31.846 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-24T11:32:31.916 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-24T11:32:31.917 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-24T11:32:31.917 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000369581 s, 1.4 MB/s 2024-05-24T11:32:31.918 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-24T11:32:31.979 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-05-24T11:32:32.039 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-24T11:32:32.039 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:32.039 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 189554 Links: 1 2024-05-24T11:32:32.039 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:32.039 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:32.039 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-24 11:31:51.436519229 +0000 2024-05-24T11:32:32.040 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-24 11:24:59.381913435 +0000 2024-05-24T11:32:32.040 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-24 11:24:59.381913435 +0000 2024-05-24T11:32:32.040 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-05-24T11:32:32.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-24T11:32:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-24T11:32:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-24T11:32:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000271325 s, 1.9 MB/s 2024-05-24T11:32:32.110 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-24T11:32:32.171 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-05-24T11:32:32.229 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-24T11:32:32.230 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:32.230 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 183229 Links: 1 2024-05-24T11:32:32.230 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:32.230 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:32.230 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-24 11:31:51.481518384 +0000 2024-05-24T11:32:32.230 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-24 11:24:59.754906441 +0000 2024-05-24T11:32:32.230 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-24 11:24:59.754906441 +0000 2024-05-24T11:32:32.230 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-05-24T11:32:32.230 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-24T11:32:32.297 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-24T11:32:32.298 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-24T11:32:32.298 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000297997 s, 1.7 MB/s 2024-05-24T11:32:32.299 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-24T11:32:32.358 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-05-24T11:32:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-24T11:32:32.417 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:32.417 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 186709 Links: 1 2024-05-24T11:32:32.417 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:32.417 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:32.417 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-24 11:31:51.526517538 +0000 2024-05-24T11:32:32.418 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-24 11:25:00.120899578 +0000 2024-05-24T11:32:32.418 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-24 11:25:00.120899578 +0000 2024-05-24T11:32:32.418 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-05-24T11:32:32.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-24T11:32:32.484 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-24T11:32:32.484 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-24T11:32:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000281641 s, 1.8 MB/s 2024-05-24T11:32:32.486 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-24T11:32:32.544 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-05-24T11:32:32.544 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-05-24T11:32:32.557 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi053=y;smithi137=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:32.557 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: Saving service mgr spec with placement smithi039=x;smithi053=y;smithi137=z;count:3 2024-05-24T11:32:32.557 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:32.557 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:32.558 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:32.558 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:32.558 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-24T11:32:32.558 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-24T11:32:32.558 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-24T11:32:32.558 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:32.558 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mon[47907]: Deploying daemon mgr.z on smithi137 2024-05-24T11:32:32.567 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-24T11:32:32.567 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-05-24T11:32:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi053=y;smithi137=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: Saving service mgr spec with placement smithi039=x;smithi053=y;smithi137=z;count:3 2024-05-24T11:32:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-24T11:32:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-24T11:32:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-24T11:32:32.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:32.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:32 smithi039 ceph-mon[42919]: Deploying daemon mgr.z on smithi137 2024-05-24T11:32:32.629 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-24T11:32:32.630 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:32.630 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 190599 Links: 1 2024-05-24T11:32:32.630 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:32.630 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:32.630 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-24 11:32:20.181394543 +0000 2024-05-24T11:32:32.630 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-24 11:25:18.215978440 +0000 2024-05-24T11:32:32.630 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-24 11:25:18.215978440 +0000 2024-05-24T11:32:32.630 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-05-24T11:32:32.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-24T11:32:32.700 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi053=y;smithi137=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:32.700 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: Saving service mgr spec with placement smithi039=x;smithi053=y;smithi137=z;count:3 2024-05-24T11:32:32.700 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:32.700 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:32.700 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:32.700 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:32.700 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-24T11:32:32.700 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-24T11:32:32.701 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-24T11:32:32.701 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:32.701 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:32 smithi053 ceph-mon[47448]: Deploying daemon mgr.z on smithi137 2024-05-24T11:32:32.703 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-24T11:32:32.703 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-24T11:32:32.703 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000112895 s, 4.5 MB/s 2024-05-24T11:32:32.704 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-24T11:32:32.767 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-05-24T11:32:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-24T11:32:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:32.830 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 183980 Links: 1 2024-05-24T11:32:32.830 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:32.830 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:32.830 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-24 11:32:20.181394543 +0000 2024-05-24T11:32:32.830 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-24 11:25:18.601970512 +0000 2024-05-24T11:32:32.830 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-24 11:25:18.601970512 +0000 2024-05-24T11:32:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-05-24T11:32:32.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-24T11:32:32.883 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:32 smithi137 systemd[1]: Starting Ceph mgr.z for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:32:32.902 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-24T11:32:32.902 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-24T11:32:32.903 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000127193 s, 4.0 MB/s 2024-05-24T11:32:32.904 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-24T11:32:32.967 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-05-24T11:32:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-24T11:32:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:33.031 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 188805 Links: 1 2024-05-24T11:32:33.031 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:33.031 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:33.031 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-24 11:32:20.181394543 +0000 2024-05-24T11:32:33.031 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-24 11:25:18.951963324 +0000 2024-05-24T11:32:33.031 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-24 11:25:18.951963324 +0000 2024-05-24T11:32:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-05-24T11:32:33.032 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-24T11:32:33.104 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-24T11:32:33.104 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-24T11:32:33.105 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000120131 s, 4.3 MB/s 2024-05-24T11:32:33.106 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-24T11:32:33.168 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-05-24T11:32:33.230 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-24T11:32:33.230 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:33.230 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 184076 Links: 1 2024-05-24T11:32:33.230 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:33.230 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:33.231 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-05-24 11:32:20.182394522 +0000 2024-05-24T11:32:33.231 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-05-24 11:25:19.300956157 +0000 2024-05-24T11:32:33.231 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-05-24 11:25:19.300956157 +0000 2024-05-24T11:32:33.231 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-05-24T11:32:33.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-24T11:32:33.243 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:32 smithi137 ceph-mgr[49786]: -- 172.21.15.137:0/1983069908 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aed82ac1a0 con 0x55aed8282c00 2024-05-24T11:32:33.243 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:33 smithi137 bash[49747]: 897822afa79b7eb15f12e8d05c73e3529dbb4b60327af285ef5aa86029b35f54 2024-05-24T11:32:33.243 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:33 smithi137 systemd[1]: Started Ceph mgr.z for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:32:33.299 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-05-24T11:32:33.299 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-05-24T11:32:33.299 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000115446 s, 4.4 MB/s 2024-05-24T11:32:33.301 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-24T11:32:33.361 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-05-24T11:32:33.361 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-05-24T11:32:33.380 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-24T11:32:33.380 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-05-24T11:32:33.437 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-24T11:32:33.438 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:33.438 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 188742 Links: 1 2024-05-24T11:32:33.438 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:33.438 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:33.438 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-05-24 11:32:32.601933802 +0000 2024-05-24T11:32:33.438 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-05-24 11:24:34.731529343 +0000 2024-05-24T11:32:33.438 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-05-24 11:24:34.731529343 +0000 2024-05-24T11:32:33.438 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-05-24T11:32:33.438 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-24T11:32:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-05-24T11:32:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-05-24T11:32:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.00027994 s, 1.8 MB/s 2024-05-24T11:32:33.508 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-24T11:32:33.566 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-05-24T11:32:33.624 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-24T11:32:33.624 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:33.624 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 185196 Links: 1 2024-05-24T11:32:33.625 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:33.625 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:33.625 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-05-24 11:32:32.602933785 +0000 2024-05-24T11:32:33.625 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-05-24 11:24:35.064523583 +0000 2024-05-24T11:32:33.625 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-05-24 11:24:35.064523583 +0000 2024-05-24T11:32:33.625 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-05-24T11:32:33.625 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-24T11:32:33.675 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:33.385+0000 7fe652f31200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:33.486+0000 7fe652f31200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:33.580+0000 7fe652f31200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:33.671+0000 7fe652f31200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:33 smithi137 ceph-mon[47907]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:33 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:33 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:33 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:33 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:33 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:33 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-24T11:32:33.676 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:33 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:33.689 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:33 smithi053 ceph-mon[47448]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:33.689 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:33 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:33.689 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:33 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:33.689 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:33 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:33.689 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:33 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-24T11:32:33.689 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:33 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-24T11:32:33.689 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:33 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-24T11:32:33.689 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:33 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-05-24T11:32:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-05-24T11:32:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000201435 s, 2.5 MB/s 2024-05-24T11:32:33.693 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-24T11:32:33.751 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-05-24T11:32:33.809 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-24T11:32:33.809 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:33.809 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 186074 Links: 1 2024-05-24T11:32:33.809 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:33.809 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:33.809 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-05-24 11:32:32.602933785 +0000 2024-05-24T11:32:33.809 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-05-24 11:24:35.401517754 +0000 2024-05-24T11:32:33.809 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-05-24 11:24:35.401517754 +0000 2024-05-24T11:32:33.809 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-05-24T11:32:33.810 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-24T11:32:33.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:33 smithi039 ceph-mon[42919]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:33.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:33 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:33.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:33 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:33.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:33 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:33.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:33 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-24T11:32:33.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:33 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-24T11:32:33.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:33 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-24T11:32:33.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:33 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:33.874 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-05-24T11:32:33.874 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-05-24T11:32:33.874 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000224069 s, 2.3 MB/s 2024-05-24T11:32:33.875 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-24T11:32:33.933 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-05-24T11:32:33.991 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-24T11:32:33.991 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-24T11:32:33.992 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 188867 Links: 1 2024-05-24T11:32:33.992 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-24T11:32:33.992 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-05-24T11:32:33.992 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-05-24 11:32:32.602933785 +0000 2024-05-24T11:32:33.992 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-05-24 11:24:35.762511509 +0000 2024-05-24T11:32:33.992 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-05-24 11:24:35.762511509 +0000 2024-05-24T11:32:33.992 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-05-24T11:32:33.992 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-24T11:32:34.057 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-05-24T11:32:34.058 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-05-24T11:32:34.058 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000221958 s, 2.3 MB/s 2024-05-24T11:32:34.059 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-24T11:32:34.123 INFO:tasks.cephadm:Deploying osd.0 on smithi039 with /dev/vg_nvme/lv_4... 2024-05-24T11:32:34.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-24T11:32:34.359 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:32:34.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:34 smithi053 ceph-mon[47448]: Deploying daemon mgr.y on smithi053 2024-05-24T11:32:34.602 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:34 smithi137 ceph-mon[47907]: Deploying daemon mgr.y on smithi053 2024-05-24T11:32:34.602 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:34.346+0000 7fe652f31200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:32:34.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:34 smithi039 ceph-mon[42919]: Deploying daemon mgr.y on smithi053 2024-05-24T11:32:34.956 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:34.598+0000 7fe652f31200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-24T11:32:34.956 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:34.685+0000 7fe652f31200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-24T11:32:34.956 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:34.794+0000 7fe652f31200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-24T11:32:34.956 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:34.951+0000 7fe652f31200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-24T11:32:34.987 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:34 smithi053 systemd[1]: Starting Ceph mgr.y for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:32:35.243 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:35.079+0000 7fe652f31200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-24T11:32:35.243 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:35.165+0000 7fe652f31200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-24T11:32:35.419 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:34 smithi053 bash[48924]: ba706faa775d7ad2fc7f6264d3efb4206e88bb7105fa4474274e73cc1f7448de 2024-05-24T11:32:35.420 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:34 smithi053 systemd[1]: Started Ceph mgr.y for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:32:35.420 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:34 smithi053 ceph-mgr[48962]: -- 172.21.15.53:0/3992137238 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562b832f21a0 con 0x562b832c8800 2024-05-24T11:32:35.694 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:35 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:35.458+0000 7f4f58046200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-24T11:32:35.694 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:35 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:35.568+0000 7f4f58046200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-24T11:32:35.694 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:35 smithi053 ceph-mon[47448]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:35.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.695 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:35.720 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-24T11:32:35.720 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-24T11:32:35.720 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-24T11:32:35.720 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-24T11:32:35.721 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00865993 s, 1.2 GB/s 2024-05-24T11:32:35.721 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-24T11:32:35.743 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:35.571+0000 7fe652f31200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-24T11:32:35.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:35 smithi137 ceph-mon[47907]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:35.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:35.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:35 smithi039 ceph-mon[42919]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:35.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:35.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:35.987 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:35 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:35.688+0000 7f4f58046200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-24T11:32:35.987 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:35 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:35.789+0000 7f4f58046200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-24T11:32:36.065 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_4 2024-05-24T11:32:36.292 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:32:36.302 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:36.009+0000 7fe652f31200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-24T11:32:36.302 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:36.205+0000 7fe652f31200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-24T11:32:36.302 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:36.298+0000 7fe652f31200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-24T11:32:36.832 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:36 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:36.545+0000 7f4f58046200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:32:37.233 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:36 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:36.829+0000 7f4f58046200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-24T11:32:37.233 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:36 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:36.926+0000 7f4f58046200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-24T11:32:37.233 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:37 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:37.049+0000 7f4f58046200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-24T11:32:37.233 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:37 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:37.229+0000 7f4f58046200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-24T11:32:37.233 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:37.233 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.233 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.233 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.233 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.234 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:37.234 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:37.234 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.234 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: Reconfiguring mgr.x (unknown last config time)... 2024-05-24T11:32:37.234 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-24T11:32:37.234 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-24T11:32:37.234 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:37.234 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: Reconfiguring daemon mgr.x on smithi039 2024-05-24T11:32:37.234 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.317 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:37.069+0000 7fe652f31200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-24T11:32:37.317 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:37.165+0000 7fe652f31200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-24T11:32:37.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:37.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:37.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:37.318 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.318 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: Reconfiguring mgr.x (unknown last config time)... 2024-05-24T11:32:37.318 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-24T11:32:37.318 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-24T11:32:37.318 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:37.318 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: Reconfiguring daemon mgr.x on smithi039 2024-05-24T11:32:37.318 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:37.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:37.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:37.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: Reconfiguring mgr.x (unknown last config time)... 2024-05-24T11:32:37.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-24T11:32:37.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-24T11:32:37.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:37.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: Reconfiguring daemon mgr.x on smithi039 2024-05-24T11:32:37.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:37.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.360+0000 7f8d337ce700 1 -- 172.21.15.39:0/3636258883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c108ee0 msgr2=0x7f8d2c10b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:37.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.360+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3636258883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c108ee0 0x7f8d2c10b370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400b0a0 tx=0x7f8d2400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:32:37.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.360+0000 7f8d337ce700 1 -- 172.21.15.39:0/3636258883 shutdown_connections 2024-05-24T11:32:37.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.360+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3636258883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c108ee0 0x7f8d2c10b370 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:37.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.360+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3636258883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d2c074f70 0x7f8d2c0733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:37.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.360+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3636258883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d2c0739a0 0x7f8d2c073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:37.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.360+0000 7f8d337ce700 1 -- 172.21.15.39:0/3636258883 >> 172.21.15.39:0/3636258883 conn(0x7f8d2c0fc5a0 msgr2=0x7f8d2c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:37.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.360+0000 7f8d337ce700 1 -- 172.21.15.39:0/3636258883 shutdown_connections 2024-05-24T11:32:37.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.360+0000 7f8d337ce700 1 -- 172.21.15.39:0/3636258883 wait complete. 2024-05-24T11:32:37.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d337ce700 1 Processor -- start 2024-05-24T11:32:37.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d337ce700 1 -- start start 2024-05-24T11:32:37.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d337ce700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c0739a0 0x7f8d2c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d337ce700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d2c074f70 0x7f8d2c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d337ce700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d2c108ee0 0x7f8d2c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d337ce700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1a0c90 con 0x7f8d2c0739a0 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d337ce700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1a0dd0 con 0x7f8d2c074f70 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d337ce700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1a0f60 con 0x7f8d2c108ee0 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d3156a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c0739a0 0x7f8d2c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d3156a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c0739a0 0x7f8d2c1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34558/0 (socket says 172.21.15.39:34558) 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d3156a700 1 -- 172.21.15.39:0/3354601578 learned_addr learned my addr 172.21.15.39:0/3354601578 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d30d69700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d2c074f70 0x7f8d2c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.361+0000 7f8d31d6b700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d2c108ee0 0x7f8d2c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d3156a700 1 -- 172.21.15.39:0/3354601578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d2c108ee0 msgr2=0x7f8d2c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d3156a700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d2c108ee0 0x7f8d2c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:37.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d3156a700 1 -- 172.21.15.39:0/3354601578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d2c074f70 msgr2=0x7f8d2c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d3156a700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d2c074f70 0x7f8d2c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d3156a700 1 -- 172.21.15.39:0/3354601578 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24009d00 con 0x7f8d2c0739a0 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d30d69700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d2c074f70 0x7f8d2c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d3156a700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c0739a0 0x7f8d2c1991a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800d980 tx=0x7f8d2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d227fc700 1 -- 172.21.15.39:0/3354601578 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d280098d0 con 0x7f8d2c0739a0 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d227fc700 1 -- 172.21.15.39:0/3354601578 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d28010460 con 0x7f8d2c0739a0 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c1a1240 con 0x7f8d2c0739a0 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d227fc700 1 -- 172.21.15.39:0/3354601578 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2800f620 con 0x7f8d2c0739a0 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.362+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c1a1790 con 0x7f8d2c0739a0 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.363+0000 7f8d227fc700 1 -- 172.21.15.39:0/3354601578 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f8d2800f780 con 0x7f8d2c0739a0 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.363+0000 7f8d227fc700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d180389d0 0x7f8d1803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:37.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.363+0000 7f8d227fc700 1 -- 172.21.15.39:0/3354601578 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8d2804b000 con 0x7f8d2c0739a0 2024-05-24T11:32:37.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.363+0000 7f8d30d69700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d180389d0 0x7f8d1803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:37.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.363+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c107cf0 con 0x7f8d2c0739a0 2024-05-24T11:32:37.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.364+0000 7f8d30d69700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d180389d0 0x7f8d1803ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005950 tx=0x7f8d1c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:37.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.368+0000 7f8d227fc700 1 -- 172.21.15.39:0/3354601578 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d280105d0 con 0x7f8d2c0739a0 2024-05-24T11:32:37.487 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:37 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:37.373+0000 7f4f58046200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-24T11:32:37.487 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:37 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:37.469+0000 7f4f58046200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-24T11:32:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:37.584+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d2c0613e0 con 0x7f8d180389d0 2024-05-24T11:32:37.743 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:37.313+0000 7fe652f31200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-24T11:32:38.237 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:37 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:37.896+0000 7f4f58046200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-24T11:32:38.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:38.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:32:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:32:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:38.243 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:37.951+0000 7fe652f31200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-24T11:32:38.243 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:38 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:38.134+0000 7fe652f31200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-24T11:32:38.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:32:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:32:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:38.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:38.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:32:38.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:32:38.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:38.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:38.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:38.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:38.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:38.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:38.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:38.660 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:38 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:38.336+0000 7f4f58046200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-24T11:32:38.661 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:38 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:38.553+0000 7f4f58046200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-24T11:32:38.743 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:38 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:38.330+0000 7fe652f31200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-24T11:32:39.554 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:39 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:39.263+0000 7fe652f31200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-24T11:32:39.806 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:39 smithi053 ceph-mon[47448]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:39.806 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:39 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4032640242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "010b8819-8872-412c-b489-b1b1389d3e7f"}]: dispatch 2024-05-24T11:32:39.806 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:39 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4032640242' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "010b8819-8872-412c-b489-b1b1389d3e7f"}]': finished 2024-05-24T11:32:39.806 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:39 smithi053 ceph-mon[47448]: osdmap e5: 1 total, 0 up, 1 in 2024-05-24T11:32:39.806 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:39.807 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:38 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:38.657+0000 7f4f58046200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-24T11:32:39.809 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:39 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:39.525+0000 7f4f58046200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-24T11:32:39.809 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:39 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:39.633+0000 7f4f58046200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-24T11:32:39.809 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:39 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:39.801+0000 7f4f58046200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-24T11:32:39.834 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:39 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:39.551+0000 7fe652f31200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:32:39.834 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:39 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:39.639+0000 7fe652f31200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-24T11:32:39.835 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:39 smithi137 ceph-mon[47907]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:39.835 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:39 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4032640242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "010b8819-8872-412c-b489-b1b1389d3e7f"}]: dispatch 2024-05-24T11:32:39.835 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:39 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4032640242' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "010b8819-8872-412c-b489-b1b1389d3e7f"}]': finished 2024-05-24T11:32:39.835 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:39 smithi137 ceph-mon[47907]: osdmap e5: 1 total, 0 up, 1 in 2024-05-24T11:32:39.835 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:40.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:39 smithi039 ceph-mon[42919]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:40.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:39 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4032640242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "010b8819-8872-412c-b489-b1b1389d3e7f"}]: dispatch 2024-05-24T11:32:40.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:39 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4032640242' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "010b8819-8872-412c-b489-b1b1389d3e7f"}]': finished 2024-05-24T11:32:40.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:39 smithi039 ceph-mon[42919]: osdmap e5: 1 total, 0 up, 1 in 2024-05-24T11:32:40.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:40.243 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:32:39 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:32:39.830+0000 7fe652f31200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-24T11:32:40.947 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:40 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:40.522+0000 7f4f58046200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-24T11:32:40.947 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:40 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:40.726+0000 7f4f58046200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-24T11:32:40.947 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:40 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:40.943+0000 7f4f58046200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-24T11:32:41.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:41 smithi053 ceph-mon[47448]: Standby manager daemon z started 2024-05-24T11:32:41.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:41 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-24T11:32:41.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:41 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:32:41.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:41 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-24T11:32:41.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:41 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:32:41.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:41 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2692461778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:32:41.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:41.240+0000 7f8d227fc700 1 -- 172.21.15.39:0/3354601578 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 14) v1 ==== 90707+0+0 (secure 0 0 0) 0x7f8d28009a30 con 0x7f8d2c0739a0 2024-05-24T11:32:41.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:41 smithi137 ceph-mon[47907]: Standby manager daemon z started 2024-05-24T11:32:41.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:41 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-24T11:32:41.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:41 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:32:41.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:41 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-24T11:32:41.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:41 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:32:41.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:41 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2692461778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:32:41.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:41 smithi039 ceph-mon[42919]: Standby manager daemon z started 2024-05-24T11:32:41.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:41 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-24T11:32:41.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:41 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:32:41.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:41 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-24T11:32:41.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:41 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.137:0/554918927' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:32:41.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:41 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2692461778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:32:42.322 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:42 smithi053 ceph-mon[47448]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:42.322 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:42 smithi053 ceph-mon[47448]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-24T11:32:42.322 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:42 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-24T11:32:42.322 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:42 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:42.000+0000 7f4f58046200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-24T11:32:42.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:42 smithi137 ceph-mon[47907]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:42.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:42 smithi137 ceph-mon[47907]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-24T11:32:42.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:42 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-24T11:32:42.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:42 smithi039 ceph-mon[42919]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:42.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:42 smithi039 ceph-mon[42919]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-24T11:32:42.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-24T11:32:42.634 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:42 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:42.318+0000 7f4f58046200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:32:42.635 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:42 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:42.417+0000 7f4f58046200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-24T11:32:42.635 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:32:42 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:32:42.631+0000 7f4f58046200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-24T11:32:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:43.143+0000 7f8d227fc700 1 -- 172.21.15.39:0/3354601578 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f8d2800ec50 con 0x7f8d2c0739a0 2024-05-24T11:32:43.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:43 smithi053 ceph-mon[47448]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:43.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:43 smithi053 ceph-mon[47448]: Standby manager daemon y started 2024-05-24T11:32:43.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:43 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-24T11:32:43.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:43 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:32:43.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:43 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-24T11:32:43.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:43 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:32:43.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:43 smithi137 ceph-mon[47907]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:43.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:43 smithi137 ceph-mon[47907]: Standby manager daemon y started 2024-05-24T11:32:43.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:43 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-24T11:32:43.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:43 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:32:43.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:43 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-24T11:32:43.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:43 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:32:43.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:43 smithi039 ceph-mon[42919]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:43.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:43 smithi039 ceph-mon[42919]: Standby manager daemon y started 2024-05-24T11:32:43.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:43 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-24T11:32:43.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:43 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:32:43.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:43 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-24T11:32:43.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:43 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.53:0/3648497582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:32:44.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:44 smithi039 ceph-mon[42919]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-05-24T11:32:44.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-24T11:32:44.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:44 smithi053 ceph-mon[47448]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-05-24T11:32:44.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-24T11:32:44.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:44 smithi137 ceph-mon[47907]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-05-24T11:32:44.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-24T11:32:45.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:45 smithi053 ceph-mon[47448]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:45.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:45 smithi137 ceph-mon[47907]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:45.531 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:45 smithi039 ceph-mon[42919]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:47.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:47 smithi039 ceph-mon[42919]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:47.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-24T11:32:47.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:47.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:47 smithi039 ceph-mon[42919]: Deploying daemon osd.0 on smithi039 2024-05-24T11:32:47.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:47 smithi053 ceph-mon[47448]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:47.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-24T11:32:47.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:47.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:47 smithi053 ceph-mon[47448]: Deploying daemon osd.0 on smithi039 2024-05-24T11:32:47.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:47 smithi137 ceph-mon[47907]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:47.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-24T11:32:47.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:47.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:47 smithi137 ceph-mon[47907]: Deploying daemon osd.0 on smithi039 2024-05-24T11:32:49.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:49 smithi053 ceph-mon[47448]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:49.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:49 smithi137 ceph-mon[47907]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:49.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:49 smithi039 ceph-mon[42919]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:51.609 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:51 smithi039 ceph-mon[42919]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:51.609 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:51 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:51.609 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:51 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:51.609 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:51 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:51.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:51 smithi053 ceph-mon[47448]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:51 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:51 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:51 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:51.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:51 smithi137 ceph-mon[47907]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:51.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:51 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:51.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:51 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:51.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:51 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.786+0000 7f8d227fc700 1 -- 172.21.15.39:0/3354601578 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8d2c0613e0 con 0x7f8d180389d0 2024-05-24T11:32:52.789 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 0 on host 'smithi039' 2024-05-24T11:32:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d180389d0 msgr2=0x7f8d1803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d180389d0 0x7f8d1803ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005950 tx=0x7f8d1c00b410 comp rx=0 tx=0).stop 2024-05-24T11:32:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c0739a0 msgr2=0x7f8d2c1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c0739a0 0x7f8d2c1991a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800d980 tx=0x7f8d2800dc90 comp rx=0 tx=0).stop 2024-05-24T11:32:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 shutdown_connections 2024-05-24T11:32:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d180389d0 0x7f8d1803ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d2c108ee0 0x7f8d2c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d2c0739a0 0x7f8d2c1991a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 --2- 172.21.15.39:0/3354601578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d2c074f70 0x7f8d2c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:52.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 >> 172.21.15.39:0/3354601578 conn(0x7f8d2c0fc5a0 msgr2=0x7f8d2c104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:52.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 shutdown_connections 2024-05-24T11:32:52.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:32:52.788+0000 7f8d337ce700 1 -- 172.21.15.39:0/3354601578 wait complete. 2024-05-24T11:32:52.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:52 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:52.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:52 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:52.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:52 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:52.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:52 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:52.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:52 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:52.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:52 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:52.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:52 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:52.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:52 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:52.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:52 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:52.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:52 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:52.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:52 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:52.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:52 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:52.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:52 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:52.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:52 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:52.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:52 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:53.176 DEBUG:teuthology.orchestra.run.smithi039:osd.0> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.0.service 2024-05-24T11:32:53.178 INFO:tasks.cephadm:Deploying osd.1 on smithi053 with /dev/vg_nvme/lv_4... 2024-05-24T11:32:53.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-24T11:32:53.213 INFO:journalctl@ceph.osd.0.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:32:53.428 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:32:53.851 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:53 smithi053 ceph-mon[47448]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:53.851 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:53.851 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:53.851 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:54.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:53 smithi039 ceph-mon[42919]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:54.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:54.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:54.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:54.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:53 smithi137 ceph-mon[47907]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:54.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:54.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:54.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:32:54.577 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:32:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-0[60527]: 2024-05-24T11:32:54.276+0000 7f1bff0917c0 -1 osd.0 0 log_to_monitors true 2024-05-24T11:32:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-24T11:32:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-24T11:32:54.732 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-24T11:32:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-24T11:32:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00894908 s, 1.2 GB/s 2024-05-24T11:32:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-24T11:32:55.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:54 smithi039 ceph-mon[42919]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-24T11:32:55.080 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:54 smithi053 ceph-mon[47448]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-24T11:32:55.238 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-05-24T11:32:55.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:54 smithi137 ceph-mon[47907]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-24T11:32:55.486 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:32:55.934 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:55.934 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-24T11:32:55.934 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: osdmap e6: 1 total, 0 up, 1 in 2024-05-24T11:32:55.934 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:55.935 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:32:55.935 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:55.935 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:55.935 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:32:55.935 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:55.935 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:55.935 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:55.935 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:55 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:56.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:56.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-24T11:32:56.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: osdmap e6: 1 total, 0 up, 1 in 2024-05-24T11:32:56.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:56.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:32:56.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:56.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:56.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:32:56.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:56.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:56.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:56.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:55 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:56.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:56.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: osdmap e6: 1 total, 0 up, 1 in 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:32:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:55 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:32:56.472 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.467+0000 7efc69c41700 1 -- 172.21.15.53:0/2592758092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc64101a90 msgr2=0x7efc64103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.467+0000 7efc69c41700 1 --2- 172.21.15.53:0/2592758092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc64101a90 0x7efc64103e80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efc54009990 tx=0x7efc54009ca0 comp rx=0 tx=0).stop 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.468+0000 7efc69c41700 1 -- 172.21.15.53:0/2592758092 shutdown_connections 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.468+0000 7efc69c41700 1 --2- 172.21.15.53:0/2592758092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc64106cf0 0x7efc64109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.468+0000 7efc69c41700 1 --2- 172.21.15.53:0/2592758092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc64101a90 0x7efc64103e80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.468+0000 7efc69c41700 1 --2- 172.21.15.53:0/2592758092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641043c0 0x7efc641067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.468+0000 7efc69c41700 1 -- 172.21.15.53:0/2592758092 >> 172.21.15.53:0/2592758092 conn(0x7efc640fb4a0 msgr2=0x7efc640fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.468+0000 7efc69c41700 1 -- 172.21.15.53:0/2592758092 shutdown_connections 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.468+0000 7efc69c41700 1 -- 172.21.15.53:0/2592758092 wait complete. 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.468+0000 7efc69c41700 1 Processor -- start 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc69c41700 1 -- start start 2024-05-24T11:32:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc69c41700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc64101a90 0x7efc64196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc69c41700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641043c0 0x7efc64197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc69c41700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc64106cf0 0x7efc6419e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc69c41700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419e960 con 0x7efc64106cf0 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc69c41700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419eaa0 con 0x7efc641043c0 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc69c41700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6419ec30 con 0x7efc64101a90 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc62ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641043c0 0x7efc64197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc62ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641043c0 0x7efc64197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41406/0 (socket says 172.21.15.53:41406) 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc62ffd700 1 -- 172.21.15.53:0/981439360 learned_addr learned my addr 172.21.15.53:0/981439360 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc63fff700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc64106cf0 0x7efc6419e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc637fe700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc64101a90 0x7efc64196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc62ffd700 1 -- 172.21.15.53:0/981439360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc64101a90 msgr2=0x7efc64196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc62ffd700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc64101a90 0x7efc64196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:56.474 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc62ffd700 1 -- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc64106cf0 msgr2=0x7efc6419e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:32:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc62ffd700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc64106cf0 0x7efc6419e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:32:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc62ffd700 1 -- 172.21.15.53:0/981439360 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540095f0 con 0x7efc641043c0 2024-05-24T11:32:56.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.469+0000 7efc62ffd700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641043c0 0x7efc64197360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc5800e9a0 tx=0x7efc5800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.470+0000 7efc60ff9700 1 -- 172.21.15.53:0/981439360 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58009790 con 0x7efc641043c0 2024-05-24T11:32:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.470+0000 7efc60ff9700 1 -- 172.21.15.53:0/981439360 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc58010430 con 0x7efc641043c0 2024-05-24T11:32:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.470+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6419ef10 con 0x7efc641043c0 2024-05-24T11:32:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.470+0000 7efc60ff9700 1 -- 172.21.15.53:0/981439360 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5800cea0 con 0x7efc641043c0 2024-05-24T11:32:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.470+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6419f380 con 0x7efc641043c0 2024-05-24T11:32:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.471+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc640fd020 con 0x7efc641043c0 2024-05-24T11:32:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.472+0000 7efc60ff9700 1 -- 172.21.15.53:0/981439360 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7efc580105a0 con 0x7efc641043c0 2024-05-24T11:32:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.472+0000 7efc60ff9700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efc4c0a1090 0x7efc4c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:32:56.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.473+0000 7efc60ff9700 1 -- 172.21.15.53:0/981439360 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7efc58014070 con 0x7efc641043c0 2024-05-24T11:32:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.473+0000 7efc637fe700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efc4c0a1090 0x7efc4c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:32:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.473+0000 7efc637fe700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efc4c0a1090 0x7efc4c0a3550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc54005910 tx=0x7efc5400b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:32:56.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.477+0000 7efc60ff9700 1 -- 172.21.15.53:0/981439360 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efc58052d60 con 0x7efc641043c0 2024-05-24T11:32:56.702 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:32:56.698+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7efc6402ce10 con 0x7efc4c0a1090 2024-05-24T11:32:57.076 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:32:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-0[60527]: 2024-05-24T11:32:56.703+0000 7f1bf56f7700 -1 osd.0 0 waiting for initial osdmap 2024-05-24T11:32:57.076 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:32:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-0[60527]: 2024-05-24T11:32:56.707+0000 7f1bec4e2700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:32:57.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi039 2024-05-24T11:32:57.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi039 to 17114M 2024-05-24T11:32:57.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:32:57.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: osdmap e7: 1 total, 0 up, 1 in 2024-05-24T11:32:57.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:57.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:57.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:32:57.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' 2024-05-24T11:32:57.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:32:57.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:57.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:57.123 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi039 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi039 to 17114M 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: osdmap e7: 1 total, 0 up, 1 in 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:57.124 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:57.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi039 2024-05-24T11:32:57.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi039 to 17114M 2024-05-24T11:32:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:32:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: osdmap e7: 1 total, 0 up, 1 in 2024-05-24T11:32:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:32:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: from='osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952]' entity='osd.0' 2024-05-24T11:32:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:32:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:32:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:57.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:57 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:32:57.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:57 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:32:57.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:57 smithi053 ceph-mon[47448]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:57.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:57 smithi053 ceph-mon[47448]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:57.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:57 smithi053 ceph-mon[47448]: osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] boot 2024-05-24T11:32:57.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:57 smithi053 ceph-mon[47448]: osdmap e8: 1 total, 1 up, 1 in 2024-05-24T11:32:57.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:57 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:58.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:57 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:32:58.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:57 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:32:58.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:57 smithi039 ceph-mon[42919]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:58.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:57 smithi039 ceph-mon[42919]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:58.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:57 smithi039 ceph-mon[42919]: osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] boot 2024-05-24T11:32:58.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:57 smithi039 ceph-mon[42919]: osdmap e8: 1 total, 1 up, 1 in 2024-05-24T11:32:58.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:57 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:58.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:57 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:32:58.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:57 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:32:58.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:57 smithi137 ceph-mon[47907]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-24T11:32:58.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:57 smithi137 ceph-mon[47907]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:32:58.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:57 smithi137 ceph-mon[47907]: osd.0 [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] boot 2024-05-24T11:32:58.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:57 smithi137 ceph-mon[47907]: osdmap e8: 1 total, 1 up, 1 in 2024-05-24T11:32:58.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:57 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:32:59.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:58 smithi039 ceph-mon[42919]: from='client.? 172.21.15.53:0/1273159571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9572166c-feb0-4545-b576-9909a66bcdee"}]: dispatch 2024-05-24T11:32:59.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:58 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9572166c-feb0-4545-b576-9909a66bcdee"}]: dispatch 2024-05-24T11:32:59.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:58 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9572166c-feb0-4545-b576-9909a66bcdee"}]': finished 2024-05-24T11:32:59.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:58 smithi039 ceph-mon[42919]: osdmap e9: 2 total, 1 up, 2 in 2024-05-24T11:32:59.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:58 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:32:59.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:58 smithi053 ceph-mon[47448]: from='client.? 172.21.15.53:0/1273159571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9572166c-feb0-4545-b576-9909a66bcdee"}]: dispatch 2024-05-24T11:32:59.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:58 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9572166c-feb0-4545-b576-9909a66bcdee"}]: dispatch 2024-05-24T11:32:59.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:58 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9572166c-feb0-4545-b576-9909a66bcdee"}]': finished 2024-05-24T11:32:59.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:58 smithi053 ceph-mon[47448]: osdmap e9: 2 total, 1 up, 2 in 2024-05-24T11:32:59.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:58 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:32:59.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:58 smithi137 ceph-mon[47907]: from='client.? 172.21.15.53:0/1273159571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9572166c-feb0-4545-b576-9909a66bcdee"}]: dispatch 2024-05-24T11:32:59.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:58 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9572166c-feb0-4545-b576-9909a66bcdee"}]: dispatch 2024-05-24T11:32:59.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:58 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9572166c-feb0-4545-b576-9909a66bcdee"}]': finished 2024-05-24T11:32:59.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:58 smithi137 ceph-mon[47907]: osdmap e9: 2 total, 1 up, 2 in 2024-05-24T11:32:59.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:58 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:00.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:59 smithi039 ceph-mon[42919]: pgmap v36: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:00.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:32:59 smithi039 ceph-mon[42919]: from='client.? 172.21.15.53:0/3663150073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:00.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:59 smithi053 ceph-mon[47448]: pgmap v36: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:00.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:32:59 smithi053 ceph-mon[47448]: from='client.? 172.21.15.53:0/3663150073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:00.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:59 smithi137 ceph-mon[47907]: pgmap v36: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:00.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:32:59 smithi137 ceph-mon[47907]: from='client.? 172.21.15.53:0/3663150073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:01.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:00 smithi039 ceph-mon[42919]: pgmap v38: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:01.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:00 smithi053 ceph-mon[47448]: pgmap v38: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:01.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:00 smithi137 ceph-mon[47907]: pgmap v38: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:03.163 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:02 smithi053 ceph-mon[47448]: pgmap v39: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:03.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:02 smithi137 ceph-mon[47907]: pgmap v39: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:03.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:02 smithi039 ceph-mon[42919]: pgmap v39: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:05.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:04 smithi053 ceph-mon[47448]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:05.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:04 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-24T11:33:05.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:04 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:05.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:04 smithi053 ceph-mon[47448]: Deploying daemon osd.1 on smithi053 2024-05-24T11:33:05.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:04 smithi137 ceph-mon[47907]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:05.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:04 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-24T11:33:05.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:04 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:05.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:04 smithi137 ceph-mon[47907]: Deploying daemon osd.1 on smithi053 2024-05-24T11:33:05.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:04 smithi039 ceph-mon[42919]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:05.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:04 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-24T11:33:05.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:04 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:05.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:04 smithi039 ceph-mon[42919]: Deploying daemon osd.1 on smithi053 2024-05-24T11:33:07.061 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:06 smithi053 ceph-mon[47448]: pgmap v41: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:07.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:06 smithi137 ceph-mon[47907]: pgmap v41: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:07.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:06 smithi039 ceph-mon[42919]: pgmap v41: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:09.902 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:09 smithi053 ceph-mon[47448]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:09.902 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:09.902 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:09.902 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:09.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:09 smithi137 ceph-mon[47907]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:09.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:09.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:09.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:09 smithi039 ceph-mon[42919]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:10.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:10.866 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.862+0000 7efc60ff9700 1 -- 172.21.15.53:0/981439360 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efc6402ce10 con 0x7efc4c0a1090 2024-05-24T11:33:10.866 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 1 on host 'smithi053' 2024-05-24T11:33:10.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efc4c0a1090 msgr2=0x7efc4c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:10.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efc4c0a1090 0x7efc4c0a3550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc54005910 tx=0x7efc5400b610 comp rx=0 tx=0).stop 2024-05-24T11:33:10.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641043c0 msgr2=0x7efc64197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:10.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641043c0 0x7efc64197360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc5800e9a0 tx=0x7efc5800ecb0 comp rx=0 tx=0).stop 2024-05-24T11:33:10.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 shutdown_connections 2024-05-24T11:33:10.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efc4c0a1090 0x7efc4c0a3550 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:10.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc64101a90 0x7efc64196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:10.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc64106cf0 0x7efc6419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:10.869 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 --2- 172.21.15.53:0/981439360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641043c0 0x7efc64197360 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:10.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.865+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 >> 172.21.15.53:0/981439360 conn(0x7efc640fb4a0 msgr2=0x7efc64103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:33:10.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.866+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 shutdown_connections 2024-05-24T11:33:10.870 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:10.866+0000 7efc69c41700 1 -- 172.21.15.53:0/981439360 wait complete. 2024-05-24T11:33:11.149 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:10 smithi053 ceph-mon[47448]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:11.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:11.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:11.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.150 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:11.243 DEBUG:teuthology.orchestra.run.smithi053:osd.1> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.1.service 2024-05-24T11:33:11.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:10 smithi137 ceph-mon[47907]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:11.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:11.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:11.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:11.247 INFO:tasks.cephadm:Deploying osd.2 on smithi137 with /dev/vg_nvme/lv_4... 2024-05-24T11:33:11.247 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-24T11:33:11.283 INFO:journalctl@ceph.osd.1.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:33:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:10 smithi039 ceph-mon[42919]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:11.473 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:33:12.721 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-24T11:33:12.721 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-24T11:33:12.721 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-05-24T11:33:12.721 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-05-24T11:33:12.721 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130599 s, 803 MB/s 2024-05-24T11:33:12.721 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-05-24T11:33:12.893 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-1[52465]: 2024-05-24T11:33:12.555+0000 7fc3167e37c0 -1 osd.1 0 log_to_monitors true 2024-05-24T11:33:13.092 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:12 smithi137 ceph-mon[47907]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:13.092 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:12 smithi137 ceph-mon[47907]: from='osd.1 [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-24T11:33:13.093 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:12 smithi137 ceph-mon[47907]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-24T11:33:13.094 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-05-24T11:33:13.197 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:12 smithi053 ceph-mon[47448]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:13.197 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:12 smithi053 ceph-mon[47448]: from='osd.1 [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-24T11:33:13.197 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:12 smithi053 ceph-mon[47448]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-24T11:33:13.309 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:33:13.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:12 smithi039 ceph-mon[42919]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:13.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:12 smithi039 ceph-mon[42919]: from='osd.1 [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-24T11:33:13.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:12 smithi039 ceph-mon[42919]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-24T11:33:13.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-24T11:33:13.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: osdmap e10: 2 total, 1 up, 2 in 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='osd.1 [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi053 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi053 to 17118M 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:14.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-24T11:33:14.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: osdmap e10: 2 total, 1 up, 2 in 2024-05-24T11:33:14.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:14.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='osd.1 [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:33:14.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:33:14.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi053 2024-05-24T11:33:14.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:14.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi053 to 17118M 2024-05-24T11:33:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:14.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.249+0000 7f4afbe99700 1 -- 172.21.15.137:0/2313291449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100150 msgr2=0x7f4af41005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:14.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.249+0000 7f4afbe99700 1 --2- 172.21.15.137:0/2313291449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100150 0x7f4af41005d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4af00098f0 tx=0x7f4af0009c00 comp rx=0 tx=0).stop 2024-05-24T11:33:14.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.249+0000 7f4afbe99700 1 -- 172.21.15.137:0/2313291449 shutdown_connections 2024-05-24T11:33:14.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.249+0000 7f4afbe99700 1 --2- 172.21.15.137:0/2313291449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af4100b10 0x7f4af41083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:14.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.249+0000 7f4afbe99700 1 --2- 172.21.15.137:0/2313291449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af4101580 0x7f4af41019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:14.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.249+0000 7f4afbe99700 1 --2- 172.21.15.137:0/2313291449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100150 0x7f4af41005d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:14.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.249+0000 7f4afbe99700 1 -- 172.21.15.137:0/2313291449 >> 172.21.15.137:0/2313291449 conn(0x7f4af40fac40 msgr2=0x7f4af40fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:33:14.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.250+0000 7f4afbe99700 1 -- 172.21.15.137:0/2313291449 shutdown_connections 2024-05-24T11:33:14.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.250+0000 7f4afbe99700 1 -- 172.21.15.137:0/2313291449 wait complete. 2024-05-24T11:33:14.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.250+0000 7f4afbe99700 1 Processor -- start 2024-05-24T11:33:14.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.250+0000 7f4afbe99700 1 -- start start 2024-05-24T11:33:14.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.250+0000 7f4afbe99700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af4100150 0x7f4af40ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:14.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.250+0000 7f4afbe99700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100b10 0x7f4af419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:14.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.250+0000 7f4afbe99700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af4101580 0x7f4af419e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:14.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4afbe99700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af419e9a0 con 0x7f4af4100150 2024-05-24T11:33:14.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4afbe99700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af419eb10 con 0x7f4af4101580 2024-05-24T11:33:14.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4afbe99700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af419ecd0 con 0x7f4af4100b10 2024-05-24T11:33:14.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9434700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100b10 0x7f4af419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:14.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9434700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100b10 0x7f4af419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58990/0 (socket says 172.21.15.137:58990) 2024-05-24T11:33:14.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9434700 1 -- 172.21.15.137:0/732295654 learned_addr learned my addr 172.21.15.137:0/732295654 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:33:14.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9c35700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af4100150 0x7f4af40ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:14.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4afa436700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af4101580 0x7f4af419e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:14.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9434700 1 -- 172.21.15.137:0/732295654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af4101580 msgr2=0x7f4af419e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:14.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9434700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af4101580 0x7f4af419e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9434700 1 -- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af4100150 msgr2=0x7f4af40ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9434700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af4100150 0x7f4af40ffb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9434700 1 -- 172.21.15.137:0/732295654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af0009520 con 0x7f4af4100b10 2024-05-24T11:33:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4af9434700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100b10 0x7f4af419bc30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009810 tx=0x7f4af000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:33:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4aeaffd700 1 -- 172.21.15.137:0/732295654 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af001c070 con 0x7f4af4100b10 2024-05-24T11:33:14.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4aeaffd700 1 -- 172.21.15.137:0/732295654 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4af000f7b0 con 0x7f4af4100b10 2024-05-24T11:33:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4aeaffd700 1 -- 172.21.15.137:0/732295654 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af00038e0 con 0x7f4af4100b10 2024-05-24T11:33:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af419ef50 con 0x7f4af4100b10 2024-05-24T11:33:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.251+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af419f2b0 con 0x7f4af4100b10 2024-05-24T11:33:14.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.252+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af41074e0 con 0x7f4af4100b10 2024-05-24T11:33:14.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.253+0000 7f4aeaffd700 1 -- 172.21.15.137:0/732295654 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f4af0020030 con 0x7f4af4100b10 2024-05-24T11:33:14.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.254+0000 7f4aeaffd700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ae00a1110 0x7f4ae00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:14.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.254+0000 7f4aeaffd700 1 -- 172.21.15.137:0/732295654 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f4af00cbff0 con 0x7f4af4100b10 2024-05-24T11:33:14.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.254+0000 7f4af9c35700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ae00a1110 0x7f4ae00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:14.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.254+0000 7f4af9c35700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ae00a1110 0x7f4ae00a35d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009c60 tx=0x7f4ae4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:33:14.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.258+0000 7f4aeaffd700 1 -- 172.21.15.137:0/732295654 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4af0097180 con 0x7f4af4100b10 2024-05-24T11:33:14.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: osdmap e10: 2 total, 1 up, 2 in 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='osd.1 [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi053 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi053 to 17118M 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:14.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:14.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:14.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:14.456+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4af402ceb0 con 0x7f4ae00a1110 2024-05-24T11:33:15.123 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:15.123 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:33:15.123 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: osdmap e11: 2 total, 1 up, 2 in 2024-05-24T11:33:15.123 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:15.123 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:15.123 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:33:15.123 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:33:15.123 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:33:15.124 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:15.124 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:14 smithi137 ceph-mon[47907]: from='osd.1 ' entity='osd.1' 2024-05-24T11:33:15.237 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:14 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-1[52465]: 2024-05-24T11:33:14.769+0000 7fc30ce49700 -1 osd.1 0 waiting for initial osdmap 2024-05-24T11:33:15.238 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:14 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-1[52465]: 2024-05-24T11:33:14.773+0000 7fc303c34700 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:33:15.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:15.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:33:15.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: osdmap e11: 2 total, 1 up, 2 in 2024-05-24T11:33:15.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:15.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:15.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:33:15.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:33:15.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:33:15.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:15.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:14 smithi053 ceph-mon[47448]: from='osd.1 ' entity='osd.1' 2024-05-24T11:33:15.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:15.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:33:15.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: osdmap e11: 2 total, 1 up, 2 in 2024-05-24T11:33:15.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:15.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:15.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:33:15.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:33:15.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:33:15.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:15.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:14 smithi039 ceph-mon[42919]: from='osd.1 ' entity='osd.1' 2024-05-24T11:33:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:15 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:33:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:15 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:33:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:15 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:15 smithi053 ceph-mon[47448]: osd.1 [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] boot 2024-05-24T11:33:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:15 smithi053 ceph-mon[47448]: osdmap e12: 2 total, 2 up, 2 in 2024-05-24T11:33:16.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:15 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:15 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:33:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:15 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:33:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:15 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:15 smithi137 ceph-mon[47907]: osd.1 [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] boot 2024-05-24T11:33:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:15 smithi137 ceph-mon[47907]: osdmap e12: 2 total, 2 up, 2 in 2024-05-24T11:33:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:15 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:16.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:15 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:33:16.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:15 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:33:16.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:15 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:16.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:15 smithi039 ceph-mon[42919]: osd.1 [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] boot 2024-05-24T11:33:16.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:15 smithi039 ceph-mon[42919]: osdmap e12: 2 total, 2 up, 2 in 2024-05-24T11:33:16.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:15 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:33:17.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:16 smithi053 ceph-mon[47448]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:17.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:16 smithi053 ceph-mon[47448]: from='client.? 172.21.15.137:0/3168293080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e133e51-8deb-40e3-b5ee-813fd6dfb344"}]: dispatch 2024-05-24T11:33:17.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:16 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e133e51-8deb-40e3-b5ee-813fd6dfb344"}]: dispatch 2024-05-24T11:33:17.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:16 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e133e51-8deb-40e3-b5ee-813fd6dfb344"}]': finished 2024-05-24T11:33:17.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:16 smithi053 ceph-mon[47448]: osdmap e13: 3 total, 2 up, 3 in 2024-05-24T11:33:17.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:17.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:16 smithi053 ceph-mon[47448]: from='client.? 172.21.15.137:0/2141482845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:17.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:16 smithi137 ceph-mon[47907]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:17.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:16 smithi137 ceph-mon[47907]: from='client.? 172.21.15.137:0/3168293080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e133e51-8deb-40e3-b5ee-813fd6dfb344"}]: dispatch 2024-05-24T11:33:17.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:16 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e133e51-8deb-40e3-b5ee-813fd6dfb344"}]: dispatch 2024-05-24T11:33:17.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:16 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e133e51-8deb-40e3-b5ee-813fd6dfb344"}]': finished 2024-05-24T11:33:17.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:16 smithi137 ceph-mon[47907]: osdmap e13: 3 total, 2 up, 3 in 2024-05-24T11:33:17.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:17.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:16 smithi137 ceph-mon[47907]: from='client.? 172.21.15.137:0/2141482845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:17.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:16 smithi039 ceph-mon[42919]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-24T11:33:17.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:16 smithi039 ceph-mon[42919]: from='client.? 172.21.15.137:0/3168293080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e133e51-8deb-40e3-b5ee-813fd6dfb344"}]: dispatch 2024-05-24T11:33:17.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:16 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e133e51-8deb-40e3-b5ee-813fd6dfb344"}]: dispatch 2024-05-24T11:33:17.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:16 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e133e51-8deb-40e3-b5ee-813fd6dfb344"}]': finished 2024-05-24T11:33:17.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:16 smithi039 ceph-mon[42919]: osdmap e13: 3 total, 2 up, 3 in 2024-05-24T11:33:17.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:17.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:16 smithi039 ceph-mon[42919]: from='client.? 172.21.15.137:0/2141482845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:18.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:18 smithi053 ceph-mon[47448]: osdmap e14: 3 total, 2 up, 3 in 2024-05-24T11:33:18.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:18 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:18.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:18 smithi137 ceph-mon[47907]: osdmap e14: 3 total, 2 up, 3 in 2024-05-24T11:33:18.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:18 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:18.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:18 smithi039 ceph-mon[42919]: osdmap e14: 3 total, 2 up, 3 in 2024-05-24T11:33:18.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:18 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:19.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:19 smithi053 ceph-mon[47448]: pgmap v52: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:19.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:19 smithi137 ceph-mon[47907]: pgmap v52: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:19.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:19 smithi039 ceph-mon[42919]: pgmap v52: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:21.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:21 smithi053 ceph-mon[47448]: pgmap v53: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:21.489 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:21 smithi137 ceph-mon[47907]: pgmap v53: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:21.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:21 smithi039 ceph-mon[42919]: pgmap v53: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:23.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:23 smithi137 ceph-mon[47907]: pgmap v54: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:23.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-24T11:33:23.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:23.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:23 smithi137 ceph-mon[47907]: Deploying daemon osd.2 on smithi137 2024-05-24T11:33:23.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:23 smithi053 ceph-mon[47448]: pgmap v54: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:23.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-24T11:33:23.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:23.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:23 smithi053 ceph-mon[47448]: Deploying daemon osd.2 on smithi137 2024-05-24T11:33:23.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:23 smithi039 ceph-mon[42919]: pgmap v54: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:23.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-24T11:33:23.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:23.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:23 smithi039 ceph-mon[42919]: Deploying daemon osd.2 on smithi137 2024-05-24T11:33:25.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:25 smithi137 ceph-mon[47907]: pgmap v55: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:25.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:25 smithi039 ceph-mon[42919]: pgmap v55: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:25.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:25 smithi053 ceph-mon[47448]: pgmap v55: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:27.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:27.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:27.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:26 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:27.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:26 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:27.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:26 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:27.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:27.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:27.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:27.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.909+0000 7f4aeaffd700 1 -- 172.21.15.137:0/732295654 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4af402ceb0 con 0x7f4ae00a1110 2024-05-24T11:33:27.914 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 2 on host 'smithi137' 2024-05-24T11:33:27.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ae00a1110 msgr2=0x7f4ae00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:27.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ae00a1110 0x7f4ae00a35d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009c60 tx=0x7f4ae4009260 comp rx=0 tx=0).stop 2024-05-24T11:33:27.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100b10 msgr2=0x7f4af419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:27.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100b10 0x7f4af419bc30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009810 tx=0x7f4af000b790 comp rx=0 tx=0).stop 2024-05-24T11:33:27.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 shutdown_connections 2024-05-24T11:33:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4ae00a1110 0x7f4ae00a35d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4af4101580 0x7f4af419e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af4100150 0x7f4af40ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 --2- 172.21.15.137:0/732295654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af4100b10 0x7f4af419bc30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 >> 172.21.15.137:0/732295654 conn(0x7f4af40fac40 msgr2=0x7f4af4104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:33:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 shutdown_connections 2024-05-24T11:33:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:33:27.912+0000 7f4afbe99700 1 -- 172.21.15.137:0/732295654 wait complete. 2024-05-24T11:33:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:27 smithi039 ceph-mon[42919]: pgmap v56: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.121 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:27 smithi137 ceph-mon[47907]: pgmap v56: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:28.122 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.122 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.122 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:28.122 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:28.122 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:27 smithi053 ceph-mon[47448]: pgmap v56: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:28.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.318 DEBUG:teuthology.orchestra.run.smithi137:osd.2> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.2.service 2024-05-24T11:33:28.320 INFO:tasks.cephadm:Deploying osd.3 on smithi039 with /dev/vg_nvme/lv_3... 2024-05-24T11:33:28.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-24T11:33:28.350 INFO:journalctl@ceph.osd.2.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:33:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:33:28.983 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:28 smithi039 ceph-mon[42919]: pgmap v57: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:28.983 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:28 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.983 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:28 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:28.984 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:28 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.993 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:28 smithi137 ceph-mon[47907]: pgmap v57: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:28.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:28 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:28.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:28 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:28.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:28 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:29.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:28 smithi053 ceph-mon[47448]: pgmap v57: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:29.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:28 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:29.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:28 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:29.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:28 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-24T11:33:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-24T11:33:29.945 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-24T11:33:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-24T11:33:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00755128 s, 1.4 GB/s 2024-05-24T11:33:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-24T11:33:30.095 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:29 smithi137 ceph-mon[47907]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-24T11:33:30.095 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:29 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-2[53734]: 2024-05-24T11:33:29.808+0000 7fdfcc6da7c0 -1 osd.2 0 log_to_monitors true 2024-05-24T11:33:30.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:29 smithi053 ceph-mon[47448]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-24T11:33:30.300 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:29 smithi039 ceph-mon[42919]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-24T11:33:30.474 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_3 2024-05-24T11:33:30.707 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:33:31.183 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:31.183 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-24T11:33:31.183 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: osdmap e15: 3 total, 2 up, 3 in 2024-05-24T11:33:31.183 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:31.183 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:33:31.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi137 2024-05-24T11:33:31.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:31.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi137 to 17113M 2024-05-24T11:33:31.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:31.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:31.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: osdmap e15: 3 total, 2 up, 3 in 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi137 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi137 to 17113M 2024-05-24T11:33:31.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:31.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:31.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.244 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:30 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-2[53734]: 2024-05-24T11:33:30.895+0000 7fdfc153d700 -1 osd.2 0 waiting for initial osdmap 2024-05-24T11:33:31.244 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:30 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-2[53734]: 2024-05-24T11:33:30.902+0000 7fdfbdb33700 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:33:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-24T11:33:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: osdmap e15: 3 total, 2 up, 3 in 2024-05-24T11:33:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:33:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi137 2024-05-24T11:33:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:31.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi137 to 17113M 2024-05-24T11:33:31.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:31.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:31.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:31.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.677+0000 7f86b17f8700 1 -- 172.21.15.39:0/212963506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac101b30 msgr2=0x7f86ac103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:31.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.677+0000 7f86b17f8700 1 --2- 172.21.15.39:0/212963506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac101b30 0x7f86ac103f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f869c009990 tx=0x7f869c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:33:31.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.677+0000 7f86b17f8700 1 -- 172.21.15.39:0/212963506 shutdown_connections 2024-05-24T11:33:31.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.677+0000 7f86b17f8700 1 --2- 172.21.15.39:0/212963506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86ac106d90 0x7f86ac109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:31.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.677+0000 7f86b17f8700 1 --2- 172.21.15.39:0/212963506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac101b30 0x7f86ac103f20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:31.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.677+0000 7f86b17f8700 1 --2- 172.21.15.39:0/212963506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86ac104460 0x7f86ac106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:31.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.677+0000 7f86b17f8700 1 -- 172.21.15.39:0/212963506 >> 172.21.15.39:0/212963506 conn(0x7f86ac0fb4a0 msgr2=0x7f86ac0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:33:31.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.677+0000 7f86b17f8700 1 -- 172.21.15.39:0/212963506 shutdown_connections 2024-05-24T11:33:31.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.677+0000 7f86b17f8700 1 -- 172.21.15.39:0/212963506 wait complete. 2024-05-24T11:33:31.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.678+0000 7f86b17f8700 1 Processor -- start 2024-05-24T11:33:31.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.678+0000 7f86b17f8700 1 -- start start 2024-05-24T11:33:31.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86b17f8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86ac101b30 0x7f86ac196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:31.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86b17f8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac104460 0x7f86ac1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:31.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86b17f8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86ac106d90 0x7f86ac19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:31.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86b17f8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac19e9e0 con 0x7f86ac104460 2024-05-24T11:33:31.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86b17f8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac19eb20 con 0x7f86ac106d90 2024-05-24T11:33:31.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86b17f8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac19ecb0 con 0x7f86ac101b30 2024-05-24T11:33:31.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aa7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac104460 0x7f86ac1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:31.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aa7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac104460 0x7f86ac1973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44896/0 (socket says 172.21.15.39:44896) 2024-05-24T11:33:31.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aa7fc700 1 -- 172.21.15.39:0/1742271892 learned_addr learned my addr 172.21.15.39:0/1742271892 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:33:31.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86ab7fe700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86ac106d90 0x7f86ac19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:31.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aaffd700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86ac101b30 0x7f86ac196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:31.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aa7fc700 1 -- 172.21.15.39:0/1742271892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86ac101b30 msgr2=0x7f86ac196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:31.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aa7fc700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86ac101b30 0x7f86ac196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aa7fc700 1 -- 172.21.15.39:0/1742271892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86ac106d90 msgr2=0x7f86ac19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aa7fc700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86ac106d90 0x7f86ac19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aa7fc700 1 -- 172.21.15.39:0/1742271892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0095f0 con 0x7f86ac104460 2024-05-24T11:33:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86aa7fc700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac104460 0x7f86ac1973e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f869400e9d0 tx=0x7f869400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:33:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86a3fff700 1 -- 172.21.15.39:0/1742271892 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694009790 con 0x7f86ac104460 2024-05-24T11:33:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86a3fff700 1 -- 172.21.15.39:0/1742271892 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f869400cbd0 con 0x7f86ac104460 2024-05-24T11:33:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ac19ef90 con 0x7f86ac104460 2024-05-24T11:33:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.679+0000 7f86a3fff700 1 -- 172.21.15.39:0/1742271892 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694018680 con 0x7f86ac104460 2024-05-24T11:33:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.680+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ac19f400 con 0x7f86ac104460 2024-05-24T11:33:31.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.680+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ac0fd020 con 0x7f86ac104460 2024-05-24T11:33:31.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.681+0000 7f86a3fff700 1 -- 172.21.15.39:0/1742271892 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f8694010430 con 0x7f86ac104460 2024-05-24T11:33:31.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.682+0000 7f86a3fff700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f86980a1090 0x7f86980a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:31.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.682+0000 7f86aaffd700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f86980a1090 0x7f86980a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:31.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.682+0000 7f86a3fff700 1 -- 172.21.15.39:0/1742271892 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2571+0+0 (secure 0 0 0) 0x7f8694014070 con 0x7f86ac104460 2024-05-24T11:33:31.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.683+0000 7f86aaffd700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f86980a1090 0x7f86980a3550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f869c00b670 tx=0x7f869c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:33:31.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.686+0000 7f86a3fff700 1 -- 172.21.15.39:0/1742271892 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8694096660 con 0x7f86ac104460 2024-05-24T11:33:31.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:31.899+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f86ac02ce40 con 0x7f86980a1090 2024-05-24T11:33:32.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:31 smithi053 ceph-mon[47448]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:33:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:31 smithi053 ceph-mon[47448]: osdmap e16: 3 total, 2 up, 3 in 2024-05-24T11:33:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:31 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:31 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:32.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:31 smithi137 ceph-mon[47907]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:33:32.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:31 smithi137 ceph-mon[47907]: osdmap e16: 3 total, 2 up, 3 in 2024-05-24T11:33:32.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:31 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:32.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:31 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:32.272 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:31 smithi039 ceph-mon[42919]: from='osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:33:32.272 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:31 smithi039 ceph-mon[42919]: osdmap e16: 3 total, 2 up, 3 in 2024-05-24T11:33:32.272 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:31 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:32.272 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:31 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:33.067 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:33:33.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:33:33.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:33.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] boot 2024-05-24T11:33:33.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: osdmap e17: 3 total, 3 up, 3 in 2024-05-24T11:33:33.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:33.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:33:33.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:33:33.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:33:33.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:33.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:33:33.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:33:33.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:33.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] boot 2024-05-24T11:33:33.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: osdmap e17: 3 total, 3 up, 3 in 2024-05-24T11:33:33.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:33.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:33:33.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:33:33.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:33:33.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:33.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:33:33.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:33:33.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-24T11:33:33.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: osd.2 [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] boot 2024-05-24T11:33:33.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: osdmap e17: 3 total, 3 up, 3 in 2024-05-24T11:33:33.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:33:33.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:33:33.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:33:33.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:33:33.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:34.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:34 smithi053 ceph-mon[47448]: osdmap e18: 3 total, 3 up, 3 in 2024-05-24T11:33:34.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:34 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/684035259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29168261-54e3-4898-8f65-9b35e0e4a247"}]: dispatch 2024-05-24T11:33:34.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:34 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/684035259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29168261-54e3-4898-8f65-9b35e0e4a247"}]': finished 2024-05-24T11:33:34.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:34 smithi053 ceph-mon[47448]: osdmap e19: 4 total, 3 up, 4 in 2024-05-24T11:33:34.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:34.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:34 smithi137 ceph-mon[47907]: osdmap e18: 3 total, 3 up, 3 in 2024-05-24T11:33:34.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:34 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/684035259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29168261-54e3-4898-8f65-9b35e0e4a247"}]: dispatch 2024-05-24T11:33:34.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:34 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/684035259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29168261-54e3-4898-8f65-9b35e0e4a247"}]': finished 2024-05-24T11:33:34.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:34 smithi137 ceph-mon[47907]: osdmap e19: 4 total, 3 up, 4 in 2024-05-24T11:33:34.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:34.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:34 smithi039 ceph-mon[42919]: osdmap e18: 3 total, 3 up, 3 in 2024-05-24T11:33:34.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:34 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/684035259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29168261-54e3-4898-8f65-9b35e0e4a247"}]: dispatch 2024-05-24T11:33:34.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:34 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/684035259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29168261-54e3-4898-8f65-9b35e0e4a247"}]': finished 2024-05-24T11:33:34.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:34 smithi039 ceph-mon[42919]: osdmap e19: 4 total, 3 up, 4 in 2024-05-24T11:33:34.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:35.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:35 smithi039 ceph-mon[42919]: pgmap v64: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:35.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-24T11:33:35.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:35 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1373091128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:35.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-24T11:33:35.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:35 smithi039 ceph-mon[42919]: osdmap e20: 4 total, 3 up, 4 in 2024-05-24T11:33:35.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:35.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-24T11:33:35.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:35 smithi053 ceph-mon[47448]: pgmap v64: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:35.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-24T11:33:35.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:35 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1373091128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:35.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-24T11:33:35.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:35 smithi053 ceph-mon[47448]: osdmap e20: 4 total, 3 up, 4 in 2024-05-24T11:33:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-24T11:33:35.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:35 smithi137 ceph-mon[47907]: pgmap v64: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:35.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-24T11:33:35.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:35 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1373091128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:35.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-24T11:33:35.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:35 smithi137 ceph-mon[47907]: osdmap e20: 4 total, 3 up, 4 in 2024-05-24T11:33:35.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:35.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-24T11:33:37.076 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:33:36 smithi039 sudo[67117]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-24T11:33:37.076 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:33:36 smithi039 sudo[67117]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:37.077 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:33:36 smithi039 sudo[67117]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:37.077 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:33:36 smithi039 sudo[67117]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:37.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:37.122+0000 7f86a3fff700 1 -- 172.21.15.39:0/1742271892 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f86940106e0 con 0x7f86ac104460 2024-05-24T11:33:37.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:37 smithi053 ceph-mon[47448]: pgmap v67: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:37.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-24T11:33:37.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:37 smithi053 ceph-mon[47448]: osdmap e21: 4 total, 3 up, 4 in 2024-05-24T11:33:37.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:37.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:37 smithi053 ceph-mon[47448]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-24T11:33:37.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:37 smithi137 ceph-mon[47907]: pgmap v67: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:37.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-24T11:33:37.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:37 smithi137 ceph-mon[47907]: osdmap e21: 4 total, 3 up, 4 in 2024-05-24T11:33:37.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:37.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:37 smithi137 ceph-mon[47907]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-24T11:33:37.576 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:33:37 smithi039 sudo[67308]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-24T11:33:37.576 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:33:37 smithi039 sudo[67308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:37.577 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:33:37 smithi039 sudo[67308]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:37.577 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:33:37 smithi039 sudo[67308]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:37.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:37 smithi039 ceph-mon[42919]: pgmap v67: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:37.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-24T11:33:37.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:37 smithi039 ceph-mon[42919]: osdmap e21: 4 total, 3 up, 4 in 2024-05-24T11:33:37.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:37.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:37 smithi039 ceph-mon[42919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-24T11:33:37.988 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:37 smithi053 sudo[58081]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-24T11:33:37.988 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:37 smithi053 sudo[58081]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:37.988 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:37 smithi053 sudo[58081]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:37.988 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:37 smithi053 sudo[58081]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:38.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:38 smithi053 ceph-mon[47448]: osdmap e22: 4 total, 3 up, 4 in 2024-05-24T11:33:38.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:38.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:38 smithi053 ceph-mon[47448]: mgrmap e16: x(active, since 115s), standbys: z, y 2024-05-24T11:33:38.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:38.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:38.488 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:38 smithi053 sudo[58084]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-24T11:33:38.488 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:38 smithi053 sudo[58084]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:38.488 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:38 smithi053 sudo[58084]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:38.488 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:33:38 smithi053 sudo[58084]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:38.494 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:38 smithi137 sudo[59179]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-24T11:33:38.494 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:38 smithi137 sudo[59179]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:38.494 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:38 smithi137 sudo[59179]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:38.494 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:38 smithi137 sudo[59179]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:38.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:38 smithi137 ceph-mon[47907]: osdmap e22: 4 total, 3 up, 4 in 2024-05-24T11:33:38.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:38.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:38 smithi137 ceph-mon[47907]: mgrmap e16: x(active, since 115s), standbys: z, y 2024-05-24T11:33:38.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:38.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:38.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:38 smithi039 ceph-mon[42919]: osdmap e22: 4 total, 3 up, 4 in 2024-05-24T11:33:38.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:38.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:38 smithi039 ceph-mon[42919]: mgrmap e16: x(active, since 115s), standbys: z, y 2024-05-24T11:33:38.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:38.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:38.994 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:38 smithi137 sudo[59182]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-24T11:33:38.994 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:38 smithi137 sudo[59182]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:38.994 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:38 smithi137 sudo[59182]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:38.994 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:33:38 smithi137 sudo[59182]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:39.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 sudo[68207]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-24T11:33:39.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 sudo[68207]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:39.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 sudo[68207]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:39.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 sudo[68207]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:39.783 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:39 smithi137 ceph-mon[47907]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-24T11:33:39.784 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:39 smithi137 ceph-mon[47907]: Cluster is now healthy 2024-05-24T11:33:39.784 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:39.784 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:39 smithi137 ceph-mon[47907]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-24T11:33:39.784 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:39 smithi137 sudo[59185]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-24T11:33:39.784 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:39 smithi137 sudo[59185]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:39.784 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:39 smithi137 sudo[59185]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:39.784 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:39 smithi137 sudo[59185]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:39.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 sudo[68325]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-24T11:33:39.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 sudo[68325]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:39.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 sudo[68325]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 sudo[68325]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 ceph-mon[42919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-24T11:33:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 ceph-mon[42919]: Cluster is now healthy 2024-05-24T11:33:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:39.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:39 smithi039 ceph-mon[42919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-24T11:33:39.987 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:39 smithi053 ceph-mon[47448]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-24T11:33:39.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:39 smithi053 ceph-mon[47448]: Cluster is now healthy 2024-05-24T11:33:39.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:39.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:39 smithi053 ceph-mon[47448]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-24T11:33:40.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 sudo[59188]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-24T11:33:40.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 sudo[59188]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:40.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 sudo[59188]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:40.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 sudo[59188]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:40.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 sudo[58087]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-24T11:33:40.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 sudo[58087]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:40.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 sudo[58087]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:40.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 sudo[58087]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:41.110 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:33:41.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:33:41.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 sudo[58090]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-24T11:33:41.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 sudo[58090]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-24T11:33:41.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 sudo[58090]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-24T11:33:41.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 sudo[58090]: pam_unix(sudo:session): session closed for user root 2024-05-24T11:33:41.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:41.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-24T11:33:41.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:33:41.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:33:41.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:33:41.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-24T11:33:41.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:33:41.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:33:41.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:33:41.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-24T11:33:41.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:33:41.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:33:41.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:33:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:33:42.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:41 smithi053 ceph-mon[47448]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:42.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:41 smithi053 ceph-mon[47448]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-24T11:33:42.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:41 smithi053 ceph-mon[47448]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-24T11:33:42.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-24T11:33:42.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:42.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:41 smithi053 ceph-mon[47448]: Deploying daemon osd.3 on smithi039 2024-05-24T11:33:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:41 smithi137 ceph-mon[47907]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:41 smithi137 ceph-mon[47907]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-24T11:33:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:41 smithi137 ceph-mon[47907]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-24T11:33:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-24T11:33:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:41 smithi137 ceph-mon[47907]: Deploying daemon osd.3 on smithi039 2024-05-24T11:33:42.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:41 smithi039 ceph-mon[42919]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:42.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:41 smithi039 ceph-mon[42919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-24T11:33:42.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:41 smithi039 ceph-mon[42919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-24T11:33:42.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-24T11:33:42.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:42.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:41 smithi039 ceph-mon[42919]: Deploying daemon osd.3 on smithi039 2024-05-24T11:33:43.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:42 smithi053 ceph-mon[47448]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:43.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:42 smithi137 ceph-mon[47907]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:43.325 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:42 smithi039 ceph-mon[42919]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:45.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:44 smithi039 ceph-mon[42919]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:45.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:45.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:45.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:45.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:44 smithi053 ceph-mon[47448]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:45.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:45.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:45.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:45.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:44 smithi137 ceph-mon[47907]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:45.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:45.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:45.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:47.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.198+0000 7f86a3fff700 1 -- 172.21.15.39:0/1742271892 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f86ac02ce40 con 0x7f86980a1090 2024-05-24T11:33:47.230 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 3 on host 'smithi039' 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f86980a1090 msgr2=0x7f86980a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f86980a1090 0x7f86980a3550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f869c00b670 tx=0x7f869c005fb0 comp rx=0 tx=0).stop 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac104460 msgr2=0x7f86ac1973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac104460 0x7f86ac1973e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f869400e9d0 tx=0x7f869400ed90 comp rx=0 tx=0).stop 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 shutdown_connections 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f86980a1090 0x7f86980a3550 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86ac101b30 0x7f86ac196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86ac106d90 0x7f86ac19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 --2- 172.21.15.39:0/1742271892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac104460 0x7f86ac1973e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 >> 172.21.15.39:0/1742271892 conn(0x7f86ac0fb4a0 msgr2=0x7f86ac102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 shutdown_connections 2024-05-24T11:33:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:33:47.200+0000 7f86b17f8700 1 -- 172.21.15.39:0/1742271892 wait complete. 2024-05-24T11:33:47.605 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:47 smithi039 ceph-mon[42919]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:47.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:47.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:47.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:47.608 DEBUG:teuthology.orchestra.run.smithi039:osd.3> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.3.service 2024-05-24T11:33:47.610 INFO:tasks.cephadm:Deploying osd.4 on smithi053 with /dev/vg_nvme/lv_3... 2024-05-24T11:33:47.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-24T11:33:47.643 INFO:journalctl@ceph.osd.3.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:33:47.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:47 smithi053 ceph-mon[47448]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:47.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:47.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:47.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.646 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:47.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:47 smithi137 ceph-mon[47907]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-24T11:33:47.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:47.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:47.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:47.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:33:47.867 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:33:49.240 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-24T11:33:49.241 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-24T11:33:49.241 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-24T11:33:49.241 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-24T11:33:49.241 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00917737 s, 1.1 GB/s 2024-05-24T11:33:49.241 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-24T11:33:49.327 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 24 11:33:48 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-3[69108]: 2024-05-24T11:33:48.833+0000 7f06cd1e37c0 -1 osd.3 0 log_to_monitors true 2024-05-24T11:33:49.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:49 smithi053 ceph-mon[47448]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-24T11:33:49.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:49 smithi053 ceph-mon[47448]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-24T11:33:49.598 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-05-24T11:33:49.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:49 smithi039 ceph-mon[42919]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-24T11:33:49.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:49 smithi039 ceph-mon[42919]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-24T11:33:49.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:49 smithi137 ceph-mon[47907]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-24T11:33:49.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:49 smithi137 ceph-mon[47907]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-24T11:33:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:33:50.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-24T11:33:50.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: osdmap e23: 4 total, 3 up, 4 in 2024-05-24T11:33:50.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:50.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:33:50.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi039 2024-05-24T11:33:50.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.638 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:50.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:50.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi039 to 8557M 2024-05-24T11:33:50.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:50.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:50.639 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:50 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-24T11:33:50.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: osdmap e23: 4 total, 3 up, 4 in 2024-05-24T11:33:50.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:50.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:33:50.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi039 2024-05-24T11:33:50.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:50.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:50.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi039 to 8557M 2024-05-24T11:33:50.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:50.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:50.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:50 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-24T11:33:50.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: osdmap e23: 4 total, 3 up, 4 in 2024-05-24T11:33:50.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi039 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi039 to 8557M 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:33:50.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:50 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:33:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.826+0000 7fe063829700 1 -- 172.21.15.53:0/266500881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 msgr2=0x7fe05c103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.826+0000 7fe063829700 1 --2- 172.21.15.53:0/266500881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 0x7fe05c103f20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe058009990 tx=0x7fe058009ca0 comp rx=0 tx=0).stop 2024-05-24T11:33:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.826+0000 7fe063829700 1 -- 172.21.15.53:0/266500881 shutdown_connections 2024-05-24T11:33:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.826+0000 7fe063829700 1 --2- 172.21.15.53:0/266500881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05c106d90 0x7fe05c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.826+0000 7fe063829700 1 --2- 172.21.15.53:0/266500881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 0x7fe05c103f20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.826+0000 7fe063829700 1 --2- 172.21.15.53:0/266500881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe05c104460 0x7fe05c106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.826+0000 7fe063829700 1 -- 172.21.15.53:0/266500881 >> 172.21.15.53:0/266500881 conn(0x7fe05c0fb4a0 msgr2=0x7fe05c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:33:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.826+0000 7fe063829700 1 -- 172.21.15.53:0/266500881 shutdown_connections 2024-05-24T11:33:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.826+0000 7fe063829700 1 -- 172.21.15.53:0/266500881 wait complete. 2024-05-24T11:33:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe063829700 1 Processor -- start 2024-05-24T11:33:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe063829700 1 -- start start 2024-05-24T11:33:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe063829700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 0x7fe05c101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe063829700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe05c104460 0x7fe05c0ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe063829700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05c106d90 0x7fe05c100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe063829700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1007c0 con 0x7fe05c106d90 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe063829700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c100930 con 0x7fe05c101b30 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe063829700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1a0f40 con 0x7fe05c104460 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe0615c5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 0x7fe05c101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe061dc6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05c106d90 0x7fe05c100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe0615c5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 0x7fe05c101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35450/0 (socket says 172.21.15.53:35450) 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe0615c5700 1 -- 172.21.15.53:0/3311536963 learned_addr learned my addr 172.21.15.53:0/3311536963 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.827+0000 7fe060dc4700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe05c104460 0x7fe05c0ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe0615c5700 1 -- 172.21.15.53:0/3311536963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe05c104460 msgr2=0x7fe05c0ffae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe0615c5700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe05c104460 0x7fe05c0ffae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe0615c5700 1 -- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05c106d90 msgr2=0x7fe05c100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe0615c5700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05c106d90 0x7fe05c100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe0615c5700 1 -- 172.21.15.53:0/3311536963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0580095f0 con 0x7fe05c101b30 2024-05-24T11:33:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe061dc6700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05c106d90 0x7fe05c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:33:50.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe0615c5700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 0x7fe05c101440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe058005740 tx=0x7fe058009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:33:50.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe0527fc700 1 -- 172.21.15.53:0/3311536963 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05801d070 con 0x7fe05c101b30 2024-05-24T11:33:50.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe0527fc700 1 -- 172.21.15.53:0/3311536963 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe05800be70 con 0x7fe05c101b30 2024-05-24T11:33:50.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05c1a1170 con 0x7fe05c101b30 2024-05-24T11:33:50.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe0527fc700 1 -- 172.21.15.53:0/3311536963 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058021410 con 0x7fe05c101b30 2024-05-24T11:33:50.834 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.828+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05c1a15e0 con 0x7fe05c101b30 2024-05-24T11:33:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.829+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05c0fd020 con 0x7fe05c101b30 2024-05-24T11:33:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.830+0000 7fe0527fc700 1 -- 172.21.15.53:0/3311536963 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe05800f640 con 0x7fe05c101b30 2024-05-24T11:33:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.831+0000 7fe0527fc700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0480a11a0 0x7fe0480a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:33:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.831+0000 7fe0527fc700 1 -- 172.21.15.53:0/3311536963 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3389+0+0 (secure 0 0 0) 0x7fe0580cc850 con 0x7fe05c101b30 2024-05-24T11:33:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.831+0000 7fe060dc4700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0480a11a0 0x7fe0480a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:33:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.832+0000 7fe060dc4700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0480a11a0 0x7fe0480a3660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009a10 tx=0x7fe04c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:33:50.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:50.836+0000 7fe0527fc700 1 -- 172.21.15.53:0/3311536963 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe0580973b0 con 0x7fe05c101b30 2024-05-24T11:33:51.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:33:51.060+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe05c02ce10 con 0x7fe0480a11a0 2024-05-24T11:33:51.577 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 24 11:33:51 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-3[69108]: 2024-05-24T11:33:51.179+0000 7f06c2046700 -1 osd.3 0 waiting for initial osdmap 2024-05-24T11:33:51.577 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 24 11:33:51 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-3[69108]: 2024-05-24T11:33:51.185+0000 7f06ba634700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:33:51.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-05-24T11:33:51.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:33:51.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: osdmap e24: 4 total, 3 up, 4 in 2024-05-24T11:33:51.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:51.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:51.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:33:51.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:33:51.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' 2024-05-24T11:33:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] boot 2024-05-24T11:33:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: osdmap e25: 4 total, 4 up, 4 in 2024-05-24T11:33:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:51 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-05-24T11:33:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:33:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: osdmap e24: 4 total, 3 up, 4 in 2024-05-24T11:33:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:33:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:33:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:51.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' 2024-05-24T11:33:51.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] boot 2024-05-24T11:33:51.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: osdmap e25: 4 total, 4 up, 4 in 2024-05-24T11:33:51.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:51 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: osdmap e24: 4 total, 3 up, 4 in 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: from='osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923]' entity='osd.3' 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: osd.3 [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] boot 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: osdmap e25: 4 total, 4 up, 4 in 2024-05-24T11:33:51.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:51 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:33:52.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:52 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:33:52.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:52 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:33:52.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:52 smithi053 ceph-mon[47448]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:33:52.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:52 smithi053 ceph-mon[47448]: osdmap e26: 4 total, 4 up, 4 in 2024-05-24T11:33:52.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:52 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:33:52.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:52 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:33:52.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:52 smithi137 ceph-mon[47907]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:33:52.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:52 smithi137 ceph-mon[47907]: osdmap e26: 4 total, 4 up, 4 in 2024-05-24T11:33:52.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:52 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:33:52.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:52 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:33:52.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:52 smithi039 ceph-mon[42919]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:33:52.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:52 smithi039 ceph-mon[42919]: osdmap e26: 4 total, 4 up, 4 in 2024-05-24T11:33:53.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:53 smithi053 ceph-mon[47448]: pgmap v80: 1 pgs: 1 peering; 577 KiB data, 525 MiB used, 357 GiB / 358 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-24T11:33:53.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:53 smithi053 ceph-mon[47448]: from='client.? 172.21.15.53:0/2753941741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc68e143-c059-4fae-9d40-326fbd44dd8c"}]: dispatch 2024-05-24T11:33:53.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:53 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc68e143-c059-4fae-9d40-326fbd44dd8c"}]: dispatch 2024-05-24T11:33:53.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:53 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc68e143-c059-4fae-9d40-326fbd44dd8c"}]': finished 2024-05-24T11:33:53.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:53 smithi053 ceph-mon[47448]: osdmap e27: 5 total, 4 up, 5 in 2024-05-24T11:33:53.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:33:53.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:53 smithi137 ceph-mon[47907]: pgmap v80: 1 pgs: 1 peering; 577 KiB data, 525 MiB used, 357 GiB / 358 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-24T11:33:53.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:53 smithi137 ceph-mon[47907]: from='client.? 172.21.15.53:0/2753941741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc68e143-c059-4fae-9d40-326fbd44dd8c"}]: dispatch 2024-05-24T11:33:53.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:53 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc68e143-c059-4fae-9d40-326fbd44dd8c"}]: dispatch 2024-05-24T11:33:53.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:53 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc68e143-c059-4fae-9d40-326fbd44dd8c"}]': finished 2024-05-24T11:33:53.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:53 smithi137 ceph-mon[47907]: osdmap e27: 5 total, 4 up, 5 in 2024-05-24T11:33:53.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:33:53.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:53 smithi039 ceph-mon[42919]: pgmap v80: 1 pgs: 1 peering; 577 KiB data, 525 MiB used, 357 GiB / 358 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-24T11:33:53.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:53 smithi039 ceph-mon[42919]: from='client.? 172.21.15.53:0/2753941741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc68e143-c059-4fae-9d40-326fbd44dd8c"}]: dispatch 2024-05-24T11:33:53.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:53 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc68e143-c059-4fae-9d40-326fbd44dd8c"}]: dispatch 2024-05-24T11:33:53.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:53 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc68e143-c059-4fae-9d40-326fbd44dd8c"}]': finished 2024-05-24T11:33:53.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:53 smithi039 ceph-mon[42919]: osdmap e27: 5 total, 4 up, 5 in 2024-05-24T11:33:53.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:33:54.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:54 smithi053 ceph-mon[47448]: from='client.? 172.21.15.53:0/4092255426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:54.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:54 smithi137 ceph-mon[47907]: from='client.? 172.21.15.53:0/4092255426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:54.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:54 smithi039 ceph-mon[42919]: from='client.? 172.21.15.53:0/4092255426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:33:55.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:55 smithi053 ceph-mon[47448]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:33:55.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:55 smithi053 ceph-mon[47448]: osdmap e28: 5 total, 4 up, 5 in 2024-05-24T11:33:55.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:55 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:33:55.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:55 smithi137 ceph-mon[47907]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:33:55.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:55 smithi137 ceph-mon[47907]: osdmap e28: 5 total, 4 up, 5 in 2024-05-24T11:33:55.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:55 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:33:55.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:55 smithi039 ceph-mon[42919]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:33:55.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:55 smithi039 ceph-mon[42919]: osdmap e28: 5 total, 4 up, 5 in 2024-05-24T11:33:55.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:55 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:33:56.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:56 smithi053 ceph-mon[47448]: osdmap e29: 5 total, 4 up, 5 in 2024-05-24T11:33:56.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:33:56.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:56 smithi137 ceph-mon[47907]: osdmap e29: 5 total, 4 up, 5 in 2024-05-24T11:33:56.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:33:56.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:56 smithi039 ceph-mon[42919]: osdmap e29: 5 total, 4 up, 5 in 2024-05-24T11:33:56.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:33:57.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:57 smithi053 ceph-mon[47448]: pgmap v86: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:33:57.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:57 smithi137 ceph-mon[47907]: pgmap v86: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:33:57.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:57 smithi039 ceph-mon[42919]: pgmap v86: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:33:59.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:59 smithi053 ceph-mon[47448]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-05-24T11:33:59.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-24T11:33:59.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:33:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:59.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:59 smithi137 ceph-mon[47907]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-05-24T11:33:59.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-24T11:33:59.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:33:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:33:59.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:59 smithi039 ceph-mon[42919]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-05-24T11:33:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-24T11:33:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:33:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:00.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:00 smithi053 ceph-mon[47448]: Deploying daemon osd.4 on smithi053 2024-05-24T11:34:00.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:00 smithi137 ceph-mon[47907]: Deploying daemon osd.4 on smithi053 2024-05-24T11:34:00.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:00 smithi039 ceph-mon[42919]: Deploying daemon osd.4 on smithi053 2024-05-24T11:34:01.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:01 smithi053 ceph-mon[47448]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-24T11:34:01.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:01 smithi137 ceph-mon[47907]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-24T11:34:01.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:01 smithi039 ceph-mon[42919]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-24T11:34:03.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:03 smithi053 ceph-mon[47448]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-24T11:34:03.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:03 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:03.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:03 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:03.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:03 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:03.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:03 smithi137 ceph-mon[47907]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-24T11:34:03.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:03 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:03.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:03 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:03.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:03 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:04.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:03 smithi039 ceph-mon[42919]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-24T11:34:04.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:04.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:04.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:04.720 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.716+0000 7fe0527fc700 1 -- 172.21.15.53:0/3311536963 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe05c02ce10 con 0x7fe0480a11a0 2024-05-24T11:34:04.720 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 4 on host 'smithi053' 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0480a11a0 msgr2=0x7fe0480a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0480a11a0 0x7fe0480a3660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009a10 tx=0x7fe04c009190 comp rx=0 tx=0).stop 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 msgr2=0x7fe05c101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 0x7fe05c101440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe058005740 tx=0x7fe058009580 comp rx=0 tx=0).stop 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 shutdown_connections 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0480a11a0 0x7fe0480a3660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05c106d90 0x7fe05c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe05c101b30 0x7fe05c101440 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 --2- 172.21.15.53:0/3311536963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe05c104460 0x7fe05c0ffae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:04.723 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 >> 172.21.15.53:0/3311536963 conn(0x7fe05c0fb4a0 msgr2=0x7fe05c103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:34:04.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 shutdown_connections 2024-05-24T11:34:04.724 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:04.719+0000 7fe063829700 1 -- 172.21.15.53:0/3311536963 wait complete. 2024-05-24T11:34:05.124 DEBUG:teuthology.orchestra.run.smithi053:osd.4> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.4.service 2024-05-24T11:34:05.126 INFO:tasks.cephadm:Deploying osd.5 on smithi137 with /dev/vg_nvme/lv_3... 2024-05-24T11:34:05.126 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-24T11:34:05.165 INFO:journalctl@ceph.osd.4.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:34:05.360 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:34:05.414 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:05 smithi053 ceph-mon[47448]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-24T11:34:05.414 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.414 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.414 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:05.414 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:05.414 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.415 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.415 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.415 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:05.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:05 smithi137 ceph-mon[47907]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-24T11:34:05.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:05.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:05.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:05.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:05 smithi039 ceph-mon[42919]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-24T11:34:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:05.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:06.653 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-24T11:34:06.653 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-24T11:34:06.653 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-05-24T11:34:06.654 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-05-24T11:34:06.654 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0157283 s, 667 MB/s 2024-05-24T11:34:06.654 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-05-24T11:34:06.988 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:34:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-4[61198]: 2024-05-24T11:34:06.662+0000 7f95d50497c0 -1 osd.4 0 log_to_monitors true 2024-05-24T11:34:07.036 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-05-24T11:34:07.254 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:34:07.356 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:07 smithi137 ceph-mon[47907]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-24T11:34:07.356 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:07 smithi137 ceph-mon[47907]: from='osd.4 [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-24T11:34:07.356 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:07 smithi137 ceph-mon[47907]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-24T11:34:07.362 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:07 smithi053 ceph-mon[47448]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-24T11:34:07.363 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:07 smithi053 ceph-mon[47448]: from='osd.4 [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-24T11:34:07.363 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:07 smithi053 ceph-mon[47448]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-24T11:34:07.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:07 smithi039 ceph-mon[42919]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-24T11:34:07.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:07 smithi039 ceph-mon[42919]: from='osd.4 [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-24T11:34:07.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:07 smithi039 ceph-mon[42919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-24T11:34:08.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.156+0000 7f516c5dc700 1 -- 172.21.15.137:0/3384620295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51640ff8e0 msgr2=0x7f51640ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:08.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.156+0000 7f516c5dc700 1 --2- 172.21.15.137:0/3384620295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51640ff8e0 0x7f51640ffd00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5154009990 tx=0x7f5154009ca0 comp rx=0 tx=0).stop 2024-05-24T11:34:08.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.156+0000 7f516c5dc700 1 -- 172.21.15.137:0/3384620295 shutdown_connections 2024-05-24T11:34:08.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.156+0000 7f516c5dc700 1 --2- 172.21.15.137:0/3384620295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51640ff8e0 0x7f51640ffd00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:08.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.156+0000 7f516c5dc700 1 --2- 172.21.15.137:0/3384620295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5164100c00 0x7f5164101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:08.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.156+0000 7f516c5dc700 1 --2- 172.21.15.137:0/3384620295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5164100240 0x7f51641006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:08.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.156+0000 7f516c5dc700 1 -- 172.21.15.137:0/3384620295 >> 172.21.15.137:0/3384620295 conn(0x7f51640fb4a0 msgr2=0x7f51640fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:34:08.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.156+0000 7f516c5dc700 1 -- 172.21.15.137:0/3384620295 shutdown_connections 2024-05-24T11:34:08.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.156+0000 7f516c5dc700 1 -- 172.21.15.137:0/3384620295 wait complete. 2024-05-24T11:34:08.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516c5dc700 1 Processor -- start 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516c5dc700 1 -- start start 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516c5dc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640ff8e0 0x7f5164199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516c5dc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164100240 0x7f5164199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516c5dc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5164100c00 0x7f51641a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516c5dc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51641a0b70 con 0x7f51640ff8e0 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516c5dc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51641a0cb0 con 0x7f5164100c00 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516c5dc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51641a0e40 con 0x7f5164100240 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f5169b77700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164100240 0x7f5164199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f5169b77700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164100240 0x7f5164199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40740/0 (socket says 172.21.15.137:40740) 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f5169b77700 1 -- 172.21.15.137:0/2672559388 learned_addr learned my addr 172.21.15.137:0/2672559388 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516a378700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640ff8e0 0x7f5164199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f516ab79700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5164100c00 0x7f51641a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f5169b77700 1 -- 172.21.15.137:0/2672559388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5164100c00 msgr2=0x7f51641a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f5169b77700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5164100c00 0x7f51641a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f5169b77700 1 -- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640ff8e0 msgr2=0x7f5164199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f5169b77700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640ff8e0 0x7f5164199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f5169b77700 1 -- 172.21.15.137:0/2672559388 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51540095f0 con 0x7f5164100240 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.157+0000 7f5169b77700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164100240 0x7f5164199570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f516000d980 tx=0x7f516000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.158+0000 7f515b7fe700 1 -- 172.21.15.137:0/2672559388 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600098d0 con 0x7f5164100240 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.158+0000 7f515b7fe700 1 -- 172.21.15.137:0/2672559388 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5160010460 con 0x7f5164100240 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.158+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51641a1120 con 0x7f5164100240 2024-05-24T11:34:08.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.158+0000 7f515b7fe700 1 -- 172.21.15.137:0/2672559388 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000f620 con 0x7f5164100240 2024-05-24T11:34:08.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.158+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51641a1620 con 0x7f5164100240 2024-05-24T11:34:08.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.159+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516410a6c0 con 0x7f5164100240 2024-05-24T11:34:08.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.160+0000 7f515b7fe700 1 -- 172.21.15.137:0/2672559388 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5160009a30 con 0x7f5164100240 2024-05-24T11:34:08.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.160+0000 7f515b7fe700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51500a1120 0x7f51500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:08.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.160+0000 7f515b7fe700 1 -- 172.21.15.137:0/2672559388 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3805+0+0 (secure 0 0 0) 0x7f51600cb4c0 con 0x7f5164100240 2024-05-24T11:34:08.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.161+0000 7f516a378700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51500a1120 0x7f51500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:08.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.161+0000 7f516a378700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51500a1120 0x7f51500a35e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f5154005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:34:08.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.164+0000 7f515b7fe700 1 -- 172.21.15.137:0/2672559388 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5160095df0 con 0x7f5164100240 2024-05-24T11:34:08.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:08.367+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f51640613e0 con 0x7f51500a1120 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: osdmap e30: 5 total, 4 up, 5 in 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='osd.4 [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi053 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:08.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:08.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi053 to 8559M 2024-05-24T11:34:08.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:08.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:08.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:08 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-24T11:34:08.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: osdmap e30: 5 total, 4 up, 5 in 2024-05-24T11:34:08.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:08.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='osd.4 [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:34:08.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi053 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi053 to 8559M 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:08.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:08 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-24T11:34:08.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: osdmap e30: 5 total, 4 up, 5 in 2024-05-24T11:34:08.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='osd.4 [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi053 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi053 to 8559M 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:08.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:08 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:08.740 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:34:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-4[61198]: 2024-05-24T11:34:08.258+0000 7f95c9eac700 -1 osd.4 0 waiting for initial osdmap 2024-05-24T11:34:08.740 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:34:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-4[61198]: 2024-05-24T11:34:08.269+0000 7f95c64a2700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:34:09.364 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:09 smithi137 ceph-mon[47907]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:34:09.364 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:09 smithi137 ceph-mon[47907]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:34:09.364 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:09 smithi137 ceph-mon[47907]: osdmap e31: 5 total, 4 up, 5 in 2024-05-24T11:34:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:09 smithi137 ceph-mon[47907]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:34:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:34:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:34:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:09.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:09 smithi039 ceph-mon[42919]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:34:09.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:09 smithi039 ceph-mon[42919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:34:09.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:09 smithi039 ceph-mon[42919]: osdmap e31: 5 total, 4 up, 5 in 2024-05-24T11:34:09.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:09.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:09.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:09 smithi039 ceph-mon[42919]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:34:09.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:34:09.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:34:09.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:09 smithi053 ceph-mon[47448]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:34:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:09 smithi053 ceph-mon[47448]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:34:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:09 smithi053 ceph-mon[47448]: osdmap e31: 5 total, 4 up, 5 in 2024-05-24T11:34:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:09 smithi053 ceph-mon[47448]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:34:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:34:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:34:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:10.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:34:10.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:34:10.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: osd.4 [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] boot 2024-05-24T11:34:10.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: osdmap e32: 5 total, 5 up, 5 in 2024-05-24T11:34:10.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:10.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: from='client.? 172.21.15.137:0/1811733713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9f89542-51c6-4d02-a245-c6fb39f4d4e0"}]: dispatch 2024-05-24T11:34:10.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9f89542-51c6-4d02-a245-c6fb39f4d4e0"}]: dispatch 2024-05-24T11:34:10.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9f89542-51c6-4d02-a245-c6fb39f4d4e0"}]': finished 2024-05-24T11:34:10.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: osdmap e33: 6 total, 5 up, 6 in 2024-05-24T11:34:10.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:10.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:34:10.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:34:10.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: osd.4 [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] boot 2024-05-24T11:34:10.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: osdmap e32: 5 total, 5 up, 5 in 2024-05-24T11:34:10.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:10.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: from='client.? 172.21.15.137:0/1811733713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9f89542-51c6-4d02-a245-c6fb39f4d4e0"}]: dispatch 2024-05-24T11:34:10.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9f89542-51c6-4d02-a245-c6fb39f4d4e0"}]: dispatch 2024-05-24T11:34:10.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9f89542-51c6-4d02-a245-c6fb39f4d4e0"}]': finished 2024-05-24T11:34:10.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: osdmap e33: 6 total, 5 up, 6 in 2024-05-24T11:34:10.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:10.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:34:10.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:34:10.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: osd.4 [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] boot 2024-05-24T11:34:10.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: osdmap e32: 5 total, 5 up, 5 in 2024-05-24T11:34:10.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:34:10.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: from='client.? 172.21.15.137:0/1811733713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9f89542-51c6-4d02-a245-c6fb39f4d4e0"}]: dispatch 2024-05-24T11:34:10.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9f89542-51c6-4d02-a245-c6fb39f4d4e0"}]: dispatch 2024-05-24T11:34:10.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9f89542-51c6-4d02-a245-c6fb39f4d4e0"}]': finished 2024-05-24T11:34:10.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: osdmap e33: 6 total, 5 up, 6 in 2024-05-24T11:34:10.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:11.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:11 smithi039 ceph-mon[42919]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:34:11.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:11 smithi039 ceph-mon[42919]: from='client.? 172.21.15.137:0/1226657795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:34:11.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:11 smithi053 ceph-mon[47448]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:34:11.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:11 smithi053 ceph-mon[47448]: from='client.? 172.21.15.137:0/1226657795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:34:11.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:11 smithi137 ceph-mon[47907]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-24T11:34:11.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:11 smithi137 ceph-mon[47907]: from='client.? 172.21.15.137:0/1226657795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:34:13.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:13 smithi039 ceph-mon[42919]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:13.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:13 smithi053 ceph-mon[47448]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:13.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:13 smithi137 ceph-mon[47907]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:15.364 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:15 smithi137 ceph-mon[47907]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:15.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:15 smithi039 ceph-mon[42919]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:15.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:15 smithi053 ceph-mon[47448]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:16.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-24T11:34:16.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:16.617 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-24T11:34:16.617 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:16.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-24T11:34:16.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:17.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:17 smithi137 ceph-mon[47907]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:17.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:17 smithi137 ceph-mon[47907]: Deploying daemon osd.5 on smithi137 2024-05-24T11:34:17.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:17 smithi039 ceph-mon[42919]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:17.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:17 smithi039 ceph-mon[42919]: Deploying daemon osd.5 on smithi137 2024-05-24T11:34:17.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:17 smithi053 ceph-mon[47448]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:17.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:17 smithi053 ceph-mon[47448]: Deploying daemon osd.5 on smithi137 2024-05-24T11:34:19.561 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:19 smithi137 ceph-mon[47907]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:19.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:19 smithi039 ceph-mon[42919]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:19.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:19 smithi053 ceph-mon[47448]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:20.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:20 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:20.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:20 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:20.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:20 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:21.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:20 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:21.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:20 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:21.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:20 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:21.732 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:21 smithi137 ceph-mon[47907]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:21.733 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:21.733 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:21.733 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:21.733 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:21.733 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:21.877 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 5 on host 'smithi137' 2024-05-24T11:34:21.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.873+0000 7f515b7fe700 1 -- 172.21.15.137:0/2672559388 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f51640613e0 con 0x7f51500a1120 2024-05-24T11:34:21.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.875+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51500a1120 msgr2=0x7f51500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:21.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.875+0000 7f516c5dc700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51500a1120 0x7f51500a35e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f5154005fb0 comp rx=0 tx=0).stop 2024-05-24T11:34:21.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.875+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164100240 msgr2=0x7f5164199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:21.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.875+0000 7f516c5dc700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164100240 0x7f5164199570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f516000d980 tx=0x7f516000dc90 comp rx=0 tx=0).stop 2024-05-24T11:34:21.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.876+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 shutdown_connections 2024-05-24T11:34:21.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.876+0000 7f516c5dc700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51500a1120 0x7f51500a35e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:21.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.876+0000 7f516c5dc700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5164100c00 0x7f51641a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:21.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.876+0000 7f516c5dc700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640ff8e0 0x7f5164199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:21.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.876+0000 7f516c5dc700 1 --2- 172.21.15.137:0/2672559388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164100240 0x7f5164199570 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:21.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.876+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 >> 172.21.15.137:0/2672559388 conn(0x7f51640fb4a0 msgr2=0x7f51640fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:34:21.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.876+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 shutdown_connections 2024-05-24T11:34:21.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:34:21.876+0000 7f516c5dc700 1 -- 172.21.15.137:0/2672559388 wait complete. 2024-05-24T11:34:21.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:21 smithi053 ceph-mon[47448]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:21.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:21.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:21.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:21.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:21.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:21 smithi039 ceph-mon[42919]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:22.261 DEBUG:teuthology.orchestra.run.smithi137:osd.5> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.5.service 2024-05-24T11:34:22.263 INFO:tasks.cephadm:Deploying osd.6 on smithi039 with /dev/vg_nvme/lv_2... 2024-05-24T11:34:22.263 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-24T11:34:22.295 INFO:journalctl@ceph.osd.5.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:34:22.505 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:34:22.875 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:22 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:22.875 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:22 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:22.875 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:22 smithi137 ceph-mon[47907]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:22.875 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:22 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:22.984 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:22 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:22.984 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:22 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:22.984 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:22 smithi039 ceph-mon[42919]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:22.984 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:22 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:23.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:22 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:23.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:22 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:23.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:22 smithi053 ceph-mon[47448]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:23.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:22 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-24T11:34:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-24T11:34:23.837 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-24T11:34:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-24T11:34:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.014953 s, 701 MB/s 2024-05-24T11:34:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-24T11:34:23.838 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:34:23 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:34:23.578+0000 7fbc7f4ba7c0 -1 osd.5 0 log_to_monitors true 2024-05-24T11:34:24.120 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:23 smithi137 ceph-mon[47907]: from='osd.5 [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-24T11:34:24.120 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:23 smithi137 ceph-mon[47907]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-24T11:34:24.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:23 smithi039 ceph-mon[42919]: from='osd.5 [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-24T11:34:24.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:23 smithi039 ceph-mon[42919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-24T11:34:24.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_2 2024-05-24T11:34:24.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:23 smithi053 ceph-mon[47448]: from='osd.5 [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-24T11:34:24.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:23 smithi053 ceph-mon[47448]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-24T11:34:24.453 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:34:25.227 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-24T11:34:25.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: osdmap e34: 6 total, 5 up, 6 in 2024-05-24T11:34:25.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:25.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='osd.5 [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:34:25.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:25.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:34:25.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi137 2024-05-24T11:34:25.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:25.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:25.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi137 to 8556M 2024-05-24T11:34:25.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:25.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:25.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:24 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-24T11:34:25.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: osdmap e34: 6 total, 5 up, 6 in 2024-05-24T11:34:25.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='osd.5 [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:34:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:34:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi137 2024-05-24T11:34:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:25.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi137 to 8556M 2024-05-24T11:34:25.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:25.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:25.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:24 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-24T11:34:25.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: osdmap e34: 6 total, 5 up, 6 in 2024-05-24T11:34:25.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='osd.5 [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi137 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi137 to 8556M 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:25.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:24 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.404+0000 7fec98eb4700 1 -- 172.21.15.39:0/3109159527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec941043c0 msgr2=0x7fec941067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.404+0000 7fec98eb4700 1 --2- 172.21.15.39:0/3109159527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec941043c0 0x7fec941067b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec84009990 tx=0x7fec84009ca0 comp rx=0 tx=0).stop 2024-05-24T11:34:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.404+0000 7fec98eb4700 1 -- 172.21.15.39:0/3109159527 shutdown_connections 2024-05-24T11:34:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.404+0000 7fec98eb4700 1 --2- 172.21.15.39:0/3109159527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec94101a90 0x7fec94103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.404+0000 7fec98eb4700 1 --2- 172.21.15.39:0/3109159527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec94106cf0 0x7fec94109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.404+0000 7fec98eb4700 1 --2- 172.21.15.39:0/3109159527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec941043c0 0x7fec941067b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.404+0000 7fec98eb4700 1 -- 172.21.15.39:0/3109159527 >> 172.21.15.39:0/3109159527 conn(0x7fec940fb4a0 msgr2=0x7fec940fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:34:25.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.404+0000 7fec98eb4700 1 -- 172.21.15.39:0/3109159527 shutdown_connections 2024-05-24T11:34:25.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.404+0000 7fec98eb4700 1 -- 172.21.15.39:0/3109159527 wait complete. 2024-05-24T11:34:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.405+0000 7fec98eb4700 1 Processor -- start 2024-05-24T11:34:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.405+0000 7fec98eb4700 1 -- start start 2024-05-24T11:34:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.405+0000 7fec98eb4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec94101a90 0x7fec94196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.405+0000 7fec98eb4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec941043c0 0x7fec94197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.405+0000 7fec98eb4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec94106cf0 0x7fec9419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.405+0000 7fec98eb4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9419e9f0 con 0x7fec94106cf0 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.405+0000 7fec98eb4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9419eb30 con 0x7fec941043c0 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.405+0000 7fec98eb4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9419ecc0 con 0x7fec94101a90 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec92d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec94106cf0 0x7fec9419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec9259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec94101a90 0x7fec94196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec9259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec94101a90 0x7fec94196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:56884/0 (socket says 172.21.15.39:56884) 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec92d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec94106cf0 0x7fec9419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40002/0 (socket says 172.21.15.39:40002) 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec9259c700 1 -- 172.21.15.39:0/283834681 learned_addr learned my addr 172.21.15.39:0/283834681 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec91d9b700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec941043c0 0x7fec94197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec92d9d700 1 -- 172.21.15.39:0/283834681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec94101a90 msgr2=0x7fec94196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec92d9d700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec94101a90 0x7fec94196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec92d9d700 1 -- 172.21.15.39:0/283834681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec941043c0 msgr2=0x7fec94197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec92d9d700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec941043c0 0x7fec94197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec92d9d700 1 -- 172.21.15.39:0/283834681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840095f0 con 0x7fec94106cf0 2024-05-24T11:34:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec91d9b700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec941043c0 0x7fec94197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:34:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec92d9d700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec94106cf0 0x7fec9419e420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00be20 tx=0x7fec8c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:34:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec837fe700 1 -- 172.21.15.39:0/283834681 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c00cc10 con 0x7fec94106cf0 2024-05-24T11:34:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec837fe700 1 -- 172.21.15.39:0/283834681 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec8c00cd70 con 0x7fec94106cf0 2024-05-24T11:34:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9419efa0 con 0x7fec94106cf0 2024-05-24T11:34:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec837fe700 1 -- 172.21.15.39:0/283834681 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0125c0 con 0x7fec94106cf0 2024-05-24T11:34:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.406+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9419f3c0 con 0x7fec94106cf0 2024-05-24T11:34:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.407+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec940fd020 con 0x7fec94106cf0 2024-05-24T11:34:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.409+0000 7fec837fe700 1 -- 172.21.15.39:0/283834681 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fec8c014440 con 0x7fec94106cf0 2024-05-24T11:34:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.410+0000 7fec837fe700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec7c0a1120 0x7fec7c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.410+0000 7fec837fe700 1 -- 172.21.15.39:0/283834681 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4253+0+0 (secure 0 0 0) 0x7fec8c006e40 con 0x7fec94106cf0 2024-05-24T11:34:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.410+0000 7fec9259c700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec7c0a1120 0x7fec7c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.410+0000 7fec9259c700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec7c0a1120 0x7fec7c0a35e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec88005950 tx=0x7fec8800b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:34:25.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.414+0000 7fec837fe700 1 -- 172.21.15.39:0/283834681 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fec8c095dd0 con 0x7fec94106cf0 2024-05-24T11:34:25.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:25.628+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fec9402ceb0 con 0x7fec7c0a1120 2024-05-24T11:34:25.994 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:34:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:34:25.706+0000 7fbc7431d700 -1 osd.5 0 waiting for initial osdmap 2024-05-24T11:34:25.994 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:34:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:34:25.721+0000 7fbc6c90b700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:34:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:25 smithi137 ceph-mon[47907]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:34:25.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:25 smithi137 ceph-mon[47907]: osdmap e35: 6 total, 5 up, 6 in 2024-05-24T11:34:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:25 smithi137 ceph-mon[47907]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:34:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:34:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:34:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:25 smithi137 ceph-mon[47907]: from='osd.5 ' entity='osd.5' 2024-05-24T11:34:26.136 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:25 smithi039 ceph-mon[42919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:34:26.136 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:25 smithi039 ceph-mon[42919]: osdmap e35: 6 total, 5 up, 6 in 2024-05-24T11:34:26.136 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:26.136 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:26.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:25 smithi039 ceph-mon[42919]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:34:26.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:34:26.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:34:26.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:26.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:25 smithi039 ceph-mon[42919]: from='osd.5 ' entity='osd.5' 2024-05-24T11:34:26.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:25 smithi053 ceph-mon[47448]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:34:26.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:25 smithi053 ceph-mon[47448]: osdmap e35: 6 total, 5 up, 6 in 2024-05-24T11:34:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:25 smithi053 ceph-mon[47448]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:34:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:34:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:34:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:25 smithi053 ceph-mon[47448]: from='osd.5 ' entity='osd.5' 2024-05-24T11:34:27.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:26 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:34:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:26 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:34:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:26 smithi039 ceph-mon[42919]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:26 smithi039 ceph-mon[42919]: osd.5 [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] boot 2024-05-24T11:34:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:26 smithi039 ceph-mon[42919]: osdmap e36: 6 total, 6 up, 6 in 2024-05-24T11:34:27.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:26 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:27.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:26 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:34:27.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:26 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:34:27.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:26 smithi053 ceph-mon[47448]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:27.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:26 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:27.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:26 smithi053 ceph-mon[47448]: osd.5 [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] boot 2024-05-24T11:34:27.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:26 smithi053 ceph-mon[47448]: osdmap e36: 6 total, 6 up, 6 in 2024-05-24T11:34:27.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:26 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:27.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:26 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:34:27.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:26 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:34:27.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:26 smithi137 ceph-mon[47907]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-24T11:34:27.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:27.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:26 smithi137 ceph-mon[47907]: osd.5 [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] boot 2024-05-24T11:34:27.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:26 smithi137 ceph-mon[47907]: osdmap e36: 6 total, 6 up, 6 in 2024-05-24T11:34:27.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:26 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:34:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:27 smithi053 ceph-mon[47448]: osdmap e37: 6 total, 6 up, 6 in 2024-05-24T11:34:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:27 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3414456838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb26433d-c0d3-4cbe-ba66-2b1a27944123"}]: dispatch 2024-05-24T11:34:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:27 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3414456838' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb26433d-c0d3-4cbe-ba66-2b1a27944123"}]': finished 2024-05-24T11:34:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:27 smithi053 ceph-mon[47448]: osdmap e38: 7 total, 6 up, 7 in 2024-05-24T11:34:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:28.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:27 smithi137 ceph-mon[47907]: osdmap e37: 6 total, 6 up, 6 in 2024-05-24T11:34:28.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:27 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3414456838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb26433d-c0d3-4cbe-ba66-2b1a27944123"}]: dispatch 2024-05-24T11:34:28.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:27 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3414456838' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb26433d-c0d3-4cbe-ba66-2b1a27944123"}]': finished 2024-05-24T11:34:28.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:27 smithi137 ceph-mon[47907]: osdmap e38: 7 total, 6 up, 7 in 2024-05-24T11:34:28.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:28.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:27 smithi039 ceph-mon[42919]: osdmap e37: 6 total, 6 up, 6 in 2024-05-24T11:34:28.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:27 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3414456838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb26433d-c0d3-4cbe-ba66-2b1a27944123"}]: dispatch 2024-05-24T11:34:28.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:27 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3414456838' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb26433d-c0d3-4cbe-ba66-2b1a27944123"}]': finished 2024-05-24T11:34:28.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:27 smithi039 ceph-mon[42919]: osdmap e38: 7 total, 6 up, 7 in 2024-05-24T11:34:28.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:29.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:29 smithi053 ceph-mon[47448]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:29.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:29 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1871366625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:34:29.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:29 smithi137 ceph-mon[47907]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:29.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:29 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1871366625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:34:29.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:29 smithi039 ceph-mon[42919]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:29.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:29 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1871366625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:34:30.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:30 smithi053 ceph-mon[47448]: osdmap e39: 7 total, 6 up, 7 in 2024-05-24T11:34:30.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:30.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:30 smithi053 ceph-mon[47448]: osdmap e40: 7 total, 6 up, 7 in 2024-05-24T11:34:30.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:30.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:30 smithi137 ceph-mon[47907]: osdmap e39: 7 total, 6 up, 7 in 2024-05-24T11:34:30.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:30.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:30 smithi137 ceph-mon[47907]: osdmap e40: 7 total, 6 up, 7 in 2024-05-24T11:34:30.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:30.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:30 smithi039 ceph-mon[42919]: osdmap e39: 7 total, 6 up, 7 in 2024-05-24T11:34:30.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:30.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:30 smithi039 ceph-mon[42919]: osdmap e40: 7 total, 6 up, 7 in 2024-05-24T11:34:30.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:31.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:31 smithi053 ceph-mon[47448]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:31.498 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:31 smithi137 ceph-mon[47907]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:31.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:31 smithi039 ceph-mon[42919]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:33.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:33 smithi039 ceph-mon[42919]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-24T11:34:33.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:33 smithi053 ceph-mon[47448]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-24T11:34:33.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:33 smithi137 ceph-mon[47907]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-24T11:34:35.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:35 smithi039 ceph-mon[42919]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-24T11:34:35.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-24T11:34:35.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:35.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:35 smithi053 ceph-mon[47448]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-24T11:34:35.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-24T11:34:35.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:35.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:35 smithi137 ceph-mon[47907]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-24T11:34:35.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-24T11:34:35.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:36.477 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:36 smithi039 ceph-mon[42919]: Deploying daemon osd.6 on smithi039 2024-05-24T11:34:36.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:36 smithi053 ceph-mon[47448]: Deploying daemon osd.6 on smithi039 2024-05-24T11:34:36.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:36 smithi137 ceph-mon[47907]: Deploying daemon osd.6 on smithi039 2024-05-24T11:34:37.299 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:37 smithi039 ceph-mon[42919]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-24T11:34:37.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:37 smithi053 ceph-mon[47448]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-24T11:34:37.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:37 smithi137 ceph-mon[47907]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-24T11:34:39.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:39 smithi053 ceph-mon[47448]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-24T11:34:39.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:39.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:39.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:39.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:39 smithi137 ceph-mon[47907]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-24T11:34:39.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:39.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:39.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:39.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:39 smithi039 ceph-mon[42919]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-24T11:34:39.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:39.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:39.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:41.130 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 6 on host 'smithi039' 2024-05-24T11:34:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.126+0000 7fec837fe700 1 -- 172.21.15.39:0/283834681 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fec9402ceb0 con 0x7fec7c0a1120 2024-05-24T11:34:41.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec7c0a1120 msgr2=0x7fec7c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:41.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec7c0a1120 0x7fec7c0a35e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec88005950 tx=0x7fec8800b500 comp rx=0 tx=0).stop 2024-05-24T11:34:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec94106cf0 msgr2=0x7fec9419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec94106cf0 0x7fec9419e420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00be20 tx=0x7fec8c009ed0 comp rx=0 tx=0).stop 2024-05-24T11:34:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 shutdown_connections 2024-05-24T11:34:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec7c0a1120 0x7fec7c0a35e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec94101a90 0x7fec94196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec94106cf0 0x7fec9419e420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 --2- 172.21.15.39:0/283834681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec941043c0 0x7fec94197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:41.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 >> 172.21.15.39:0/283834681 conn(0x7fec940fb4a0 msgr2=0x7fec94102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:34:41.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.128+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 shutdown_connections 2024-05-24T11:34:41.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:34:41.129+0000 7fec98eb4700 1 -- 172.21.15.39:0/283834681 wait complete. 2024-05-24T11:34:41.587 DEBUG:teuthology.orchestra.run.smithi039:osd.6> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.6.service 2024-05-24T11:34:41.589 INFO:tasks.cephadm:Deploying osd.7 on smithi053 with /dev/vg_nvme/lv_2... 2024-05-24T11:34:41.590 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-24T11:34:41.624 INFO:journalctl@ceph.osd.6.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:34:41.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:41 smithi053 ceph-mon[47448]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-24T11:34:41.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:41.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:41.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:41.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:41 smithi137 ceph-mon[47907]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-24T11:34:41.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:41.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:41.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:41 smithi039 ceph-mon[42919]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-24T11:34:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:34:43.077 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:34:42 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:34:42.818+0000 7faa7654b7c0 -1 osd.6 0 log_to_monitors true 2024-05-24T11:34:43.372 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-24T11:34:43.372 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-24T11:34:43.372 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-24T11:34:43.372 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-24T11:34:43.372 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0159311 s, 658 MB/s 2024-05-24T11:34:43.372 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 2024-05-24T11:34:43.373 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-24T11:34:43.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:43 smithi053 ceph-mon[47448]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-24T11:34:43.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:43 smithi053 ceph-mon[47448]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-24T11:34:43.732 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-05-24T11:34:43.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:43 smithi137 ceph-mon[47907]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-24T11:34:43.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:43 smithi137 ceph-mon[47907]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-24T11:34:43.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:43 smithi039 ceph-mon[42919]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-24T11:34:43.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:43 smithi039 ceph-mon[42919]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-24T11:34:43.987 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:34:44.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-24T11:34:44.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: osdmap e41: 7 total, 6 up, 7 in 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi039 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-24T11:34:44.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: osdmap e41: 7 total, 6 up, 7 in 2024-05-24T11:34:44.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:44.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:34:44.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi039 2024-05-24T11:34:44.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-24T11:34:44.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: osdmap e41: 7 total, 6 up, 7 in 2024-05-24T11:34:44.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi039 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:45.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.029+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/1018078004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 msgr2=0x7fe3e0102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:45.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.029+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/1018078004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 0x7fe3e0102640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40099c0 tx=0x7fe3d4009cd0 comp rx=0 tx=0).stop 2024-05-24T11:34:45.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.029+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/1018078004 shutdown_connections 2024-05-24T11:34:45.033 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.029+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/1018078004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3e0102b80 0x7fe3e0109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:45.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.029+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/1018078004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3e0100fe0 0x7fe3e0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:45.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.029+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/1018078004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 0x7fe3e0102640 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:45.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.029+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/1018078004 >> 172.21.15.53:0/1018078004 conn(0x7fe3e00fc580 msgr2=0x7fe3e00fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:34:45.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.029+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/1018078004 shutdown_connections 2024-05-24T11:34:45.034 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.029+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/1018078004 wait complete. 2024-05-24T11:34:45.035 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3e5c7f700 1 Processor -- start 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3e5c7f700 1 -- start start 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3e5c7f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3e0100fe0 0x7fe3e0196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3e5c7f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 0x7fe3e0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3e5c7f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3e0102b80 0x7fe3e019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3e5c7f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019e950 con 0x7fe3e0100fe0 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3e5c7f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019ea90 con 0x7fe3e01021e0 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3e5c7f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019ec20 con 0x7fe3e0102b80 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3deffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 0x7fe3e0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3df7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3e0100fe0 0x7fe3e0196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3df7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3e0100fe0 0x7fe3e0196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.53:48756/0 (socket says 172.21.15.53:48756) 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3deffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 0x7fe3e0197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33762/0 (socket says 172.21.15.53:33762) 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3df7fe700 1 -- 172.21.15.53:0/3551125815 learned_addr learned my addr 172.21.15.53:0/3551125815 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.030+0000 7fe3dffff700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3e0102b80 0x7fe3e019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3deffd700 1 -- 172.21.15.53:0/3551125815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3e0102b80 msgr2=0x7fe3e019e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3deffd700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3e0102b80 0x7fe3e019e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3deffd700 1 -- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3e0100fe0 msgr2=0x7fe3e0196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3deffd700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3e0100fe0 0x7fe3e0196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3deffd700 1 -- 172.21.15.53:0/3551125815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d40095f0 con 0x7fe3e01021e0 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3df7fe700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3e0100fe0 0x7fe3e0196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3deffd700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 0x7fe3e0197350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009fd0 tx=0x7fe3d400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3dcff9700 1 -- 172.21.15.53:0/3551125815 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d401d070 con 0x7fe3e01021e0 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3dcff9700 1 -- 172.21.15.53:0/3551125815 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3d400bd30 con 0x7fe3e01021e0 2024-05-24T11:34:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e019eea0 con 0x7fe3e01021e0 2024-05-24T11:34:45.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3dcff9700 1 -- 172.21.15.53:0/3551125815 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40173f0 con 0x7fe3e01021e0 2024-05-24T11:34:45.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.031+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e019f250 con 0x7fe3e01021e0 2024-05-24T11:34:45.038 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.032+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e01084e0 con 0x7fe3e01021e0 2024-05-24T11:34:45.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.033+0000 7fe3dcff9700 1 -- 172.21.15.53:0/3551125815 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe3d4022470 con 0x7fe3e01021e0 2024-05-24T11:34:45.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.034+0000 7fe3dcff9700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c80a1120 0x7fe3c80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:34:45.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.034+0000 7fe3dcff9700 1 -- 172.21.15.53:0/3551125815 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4685+0+0 (secure 0 0 0) 0x7fe3d40cd420 con 0x7fe3e01021e0 2024-05-24T11:34:45.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.034+0000 7fe3df7fe700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c80a1120 0x7fe3c80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:34:45.039 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.035+0000 7fe3df7fe700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c80a1120 0x7fe3c80a35e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8003ef0 tx=0x7fe3d800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:34:45.042 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.038+0000 7fe3dcff9700 1 -- 172.21.15.53:0/3551125815 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3d40979d0 con 0x7fe3e01021e0 2024-05-24T11:34:45.264 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:45.260+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3e002ce10 con 0x7fe3c80a1120 2024-05-24T11:34:45.577 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:34:45 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:34:45.278+0000 7faa6b3ae700 -1 osd.6 0 waiting for initial osdmap 2024-05-24T11:34:45.577 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:34:45 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:34:45.289+0000 7faa6399c700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:34:45.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi039 to 5704M 2024-05-24T11:34:45.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:45.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:34:45.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: osdmap e42: 7 total, 6 up, 7 in 2024-05-24T11:34:45.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:45.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:45.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:34:45.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:34:45.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:45.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:45 smithi039 ceph-mon[42919]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' 2024-05-24T11:34:45.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi039 to 5704M 2024-05-24T11:34:45.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:45.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:34:45.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: osdmap e42: 7 total, 6 up, 7 in 2024-05-24T11:34:45.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:45.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:45.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:34:45.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:34:45.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:45.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:45 smithi053 ceph-mon[47448]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' 2024-05-24T11:34:45.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi039 to 5704M 2024-05-24T11:34:45.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:45.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:34:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: osdmap e42: 7 total, 6 up, 7 in 2024-05-24T11:34:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:34:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:34:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:45 smithi137 ceph-mon[47907]: from='osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915]' entity='osd.6' 2024-05-24T11:34:46.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:46 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:34:46.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:46 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:34:46.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:46 smithi039 ceph-mon[42919]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:34:46.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:46 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:46.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:46 smithi039 ceph-mon[42919]: osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] boot 2024-05-24T11:34:46.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:46 smithi039 ceph-mon[42919]: osdmap e43: 7 total, 7 up, 7 in 2024-05-24T11:34:46.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:46 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:46 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:34:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:46 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:34:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:46 smithi053 ceph-mon[47448]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:34:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:46 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:46 smithi053 ceph-mon[47448]: osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] boot 2024-05-24T11:34:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:46 smithi053 ceph-mon[47448]: osdmap e43: 7 total, 7 up, 7 in 2024-05-24T11:34:46.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:46 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:46.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:46 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:34:46.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:46 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:34:46.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:46 smithi137 ceph-mon[47907]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:34:46.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:46 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:46.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:46 smithi137 ceph-mon[47907]: osd.6 [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] boot 2024-05-24T11:34:46.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:46 smithi137 ceph-mon[47907]: osdmap e43: 7 total, 7 up, 7 in 2024-05-24T11:34:46.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:46 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:34:47.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:47 smithi039 ceph-mon[42919]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:47.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:47 smithi039 ceph-mon[42919]: from='client.? 172.21.15.53:0/306817889' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3690ac-bcb8-4bee-93b1-d2c729919262"}]: dispatch 2024-05-24T11:34:47.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:47 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3690ac-bcb8-4bee-93b1-d2c729919262"}]: dispatch 2024-05-24T11:34:47.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:47 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab3690ac-bcb8-4bee-93b1-d2c729919262"}]': finished 2024-05-24T11:34:47.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:47 smithi039 ceph-mon[42919]: osdmap e44: 8 total, 7 up, 8 in 2024-05-24T11:34:47.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:34:47.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:47 smithi053 ceph-mon[47448]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:47.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:47 smithi053 ceph-mon[47448]: from='client.? 172.21.15.53:0/306817889' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3690ac-bcb8-4bee-93b1-d2c729919262"}]: dispatch 2024-05-24T11:34:47.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:47 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3690ac-bcb8-4bee-93b1-d2c729919262"}]: dispatch 2024-05-24T11:34:47.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:47 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab3690ac-bcb8-4bee-93b1-d2c729919262"}]': finished 2024-05-24T11:34:47.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:47 smithi053 ceph-mon[47448]: osdmap e44: 8 total, 7 up, 8 in 2024-05-24T11:34:47.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:34:47.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:47 smithi137 ceph-mon[47907]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-24T11:34:47.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:47 smithi137 ceph-mon[47907]: from='client.? 172.21.15.53:0/306817889' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3690ac-bcb8-4bee-93b1-d2c729919262"}]: dispatch 2024-05-24T11:34:47.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:47 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3690ac-bcb8-4bee-93b1-d2c729919262"}]: dispatch 2024-05-24T11:34:47.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:47 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab3690ac-bcb8-4bee-93b1-d2c729919262"}]': finished 2024-05-24T11:34:47.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:47 smithi137 ceph-mon[47907]: osdmap e44: 8 total, 7 up, 8 in 2024-05-24T11:34:47.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:34:48.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:48 smithi039 ceph-mon[42919]: from='client.? 172.21.15.53:0/2262295203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:34:48.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:48 smithi053 ceph-mon[47448]: from='client.? 172.21.15.53:0/2262295203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:34:48.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:48 smithi137 ceph-mon[47907]: from='client.? 172.21.15.53:0/2262295203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:34:49.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:49 smithi039 ceph-mon[42919]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:49.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:49 smithi053 ceph-mon[47448]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:49.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:49 smithi137 ceph-mon[47907]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:51.803 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:51 smithi053 ceph-mon[47448]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:51.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:51 smithi039 ceph-mon[42919]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:51.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:51 smithi137 ceph-mon[47907]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:53.802 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:53 smithi053 ceph-mon[47448]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:53.802 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-24T11:34:53.803 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:53.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:53 smithi039 ceph-mon[42919]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:53.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-24T11:34:53.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:53.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:53 smithi137 ceph-mon[47907]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:53.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-24T11:34:53.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:54.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:54 smithi039 ceph-mon[42919]: Deploying daemon osd.7 on smithi053 2024-05-24T11:34:54.899 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:54 smithi053 ceph-mon[47448]: Deploying daemon osd.7 on smithi053 2024-05-24T11:34:54.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:54 smithi137 ceph-mon[47907]: Deploying daemon osd.7 on smithi053 2024-05-24T11:34:55.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:55 smithi039 ceph-mon[42919]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:55.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:55 smithi053 ceph-mon[47448]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:55.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:55 smithi137 ceph-mon[47907]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:57.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:57 smithi053 ceph-mon[47448]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:57.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:57 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:57.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:57 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:57.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:57 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:57.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:57 smithi039 ceph-mon[42919]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:57.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:57 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:57.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:57 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:57.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:57 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:57.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:57 smithi137 ceph-mon[47907]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:57.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:57 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:57.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:57 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:57.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:57 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:59.117 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.113+0000 7fe3dcff9700 1 -- 172.21.15.53:0/3551125815 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe3e002ce10 con 0x7fe3c80a1120 2024-05-24T11:34:59.118 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 7 on host 'smithi053' 2024-05-24T11:34:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c80a1120 msgr2=0x7fe3c80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c80a1120 0x7fe3c80a35e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8003ef0 tx=0x7fe3d800b040 comp rx=0 tx=0).stop 2024-05-24T11:34:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 msgr2=0x7fe3e0197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:34:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 0x7fe3e0197350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009fd0 tx=0x7fe3d400fa00 comp rx=0 tx=0).stop 2024-05-24T11:34:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 shutdown_connections 2024-05-24T11:34:59.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c80a1120 0x7fe3c80a35e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:59.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3e0102b80 0x7fe3e019e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:59.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3e0100fe0 0x7fe3e0196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:59.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 --2- 172.21.15.53:0/3551125815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3e01021e0 0x7fe3e0197350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:34:59.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 >> 172.21.15.53:0/3551125815 conn(0x7fe3e00fc580 msgr2=0x7fe3e0105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:34:59.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 shutdown_connections 2024-05-24T11:34:59.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:34:59.116+0000 7fe3e5c7f700 1 -- 172.21.15.53:0/3551125815 wait complete. 2024-05-24T11:34:59.511 DEBUG:teuthology.orchestra.run.smithi053:osd.7> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.7.service 2024-05-24T11:34:59.513 INFO:tasks.cephadm:Deploying osd.8 on smithi137 with /dev/vg_nvme/lv_2... 2024-05-24T11:34:59.514 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-24T11:34:59.552 INFO:journalctl@ceph.osd.7.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:34:59.659 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:59 smithi053 ceph-mon[47448]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:59.659 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.659 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.660 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:59.660 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:59.660 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.660 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.660 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.661 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:34:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:59 smithi137 ceph-mon[47907]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:34:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:34:59.748 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:34:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:59 smithi039 ceph-mon[42919]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:34:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:34:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:34:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:34:59.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:34:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:00.986 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-24T11:35:00.987 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-24T11:35:00.987 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-05-24T11:35:00.987 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-05-24T11:35:00.987 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0156915 s, 668 MB/s 2024-05-24T11:35:00.987 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-05-24T11:35:01.239 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:35:00 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:35:00.897+0000 7fec86aba7c0 -1 osd.7 0 log_to_monitors true 2024-05-24T11:35:01.370 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-05-24T11:35:01.590 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:35:01.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:01 smithi053 ceph-mon[47448]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:35:01.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:01 smithi053 ceph-mon[47448]: from='osd.7 [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-24T11:35:01.681 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:01 smithi053 ceph-mon[47448]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-24T11:35:01.705 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:01 smithi137 ceph-mon[47907]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:35:01.705 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:01 smithi137 ceph-mon[47907]: from='osd.7 [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-24T11:35:01.706 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:01 smithi137 ceph-mon[47907]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-24T11:35:01.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:01 smithi039 ceph-mon[42919]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:35:01.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:01 smithi039 ceph-mon[42919]: from='osd.7 [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-24T11:35:01.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:01 smithi039 ceph-mon[42919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.522+0000 7f7a65cb9700 1 -- 172.21.15.137:0/131891210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 msgr2=0x7f7a60103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.522+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/131891210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 0x7f7a60103e80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009990 tx=0x7f7a50009ca0 comp rx=0 tx=0).stop 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.522+0000 7f7a65cb9700 1 -- 172.21.15.137:0/131891210 shutdown_connections 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.522+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/131891210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 0x7f7a60103e80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.522+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/131891210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a60106cf0 0x7f7a60109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.522+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/131891210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a601043c0 0x7f7a601067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.522+0000 7f7a65cb9700 1 -- 172.21.15.137:0/131891210 >> 172.21.15.137:0/131891210 conn(0x7f7a600fb4a0 msgr2=0x7f7a600fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.522+0000 7f7a65cb9700 1 -- 172.21.15.137:0/131891210 shutdown_connections 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.522+0000 7f7a65cb9700 1 -- 172.21.15.137:0/131891210 wait complete. 2024-05-24T11:35:02.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.523+0000 7f7a65cb9700 1 Processor -- start 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.523+0000 7f7a65cb9700 1 -- start start 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a65cb9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 0x7f7a60196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a65cb9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a601043c0 0x7f7a60197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a65cb9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a60106cf0 0x7f7a6019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a65cb9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6019e950 con 0x7f7a60106cf0 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a65cb9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6019ea90 con 0x7f7a601043c0 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a65cb9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6019ec20 con 0x7f7a60101a90 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 0x7f7a60196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 0x7f7a60196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44020/0 (socket says 172.21.15.137:44020) 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5f7fe700 1 -- 172.21.15.137:0/3014795608 learned_addr learned my addr 172.21.15.137:0/3014795608 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5effd700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a601043c0 0x7f7a60197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5ffff700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a60106cf0 0x7f7a6019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:02.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5f7fe700 1 -- 172.21.15.137:0/3014795608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a601043c0 msgr2=0x7f7a60197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5f7fe700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a601043c0 0x7f7a60197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5f7fe700 1 -- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a60106cf0 msgr2=0x7f7a6019e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5f7fe700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a60106cf0 0x7f7a6019e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5f7fe700 1 -- 172.21.15.137:0/3014795608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500095f0 con 0x7f7a60101a90 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5f7fe700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 0x7f7a60196e10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a50000c00 tx=0x7f7a50003a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5cff9700 1 -- 172.21.15.137:0/3014795608 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5001d070 con 0x7f7a60101a90 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5cff9700 1 -- 172.21.15.137:0/3014795608 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a5000bd90 con 0x7f7a60101a90 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6019eea0 con 0x7f7a60101a90 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a5cff9700 1 -- 172.21.15.137:0/3014795608 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50022c40 con 0x7f7a60101a90 2024-05-24T11:35:02.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.524+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6019f200 con 0x7f7a60101a90 2024-05-24T11:35:02.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.525+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a600fd020 con 0x7f7a60101a90 2024-05-24T11:35:02.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.526+0000 7f7a5cff9700 1 -- 172.21.15.137:0/3014795608 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7a50022470 con 0x7f7a60101a90 2024-05-24T11:35:02.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.527+0000 7f7a5cff9700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7a480a1120 0x7f7a480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:02.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.527+0000 7f7a5cff9700 1 -- 172.21.15.137:0/3014795608 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f7a500cd1e0 con 0x7f7a60101a90 2024-05-24T11:35:02.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.527+0000 7f7a5effd700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7a480a1120 0x7f7a480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:02.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.528+0000 7f7a5effd700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7a480a1120 0x7f7a480a35e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005950 tx=0x7f7a5400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:35:02.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.532+0000 7f7a5cff9700 1 -- 172.21.15.137:0/3014795608 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7a50097680 con 0x7f7a60101a90 2024-05-24T11:35:02.738 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:35:02 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:35:02.421+0000 7fec7b91d700 -1 osd.7 0 waiting for initial osdmap 2024-05-24T11:35:02.738 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:35:02 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:35:02.435+0000 7fec77f13700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: osdmap e45: 8 total, 7 up, 8 in 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='osd.7 [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi053 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:02.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:02.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:02 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:02.734+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a6002cf00 con 0x7f7a480a1120 2024-05-24T11:35:02.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: osdmap e45: 8 total, 7 up, 8 in 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='osd.7 [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi053 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:02.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:02 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-24T11:35:02.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: osdmap e45: 8 total, 7 up, 8 in 2024-05-24T11:35:02.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:02.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='osd.7 [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:02.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:02.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi053 2024-05-24T11:35:02.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:02.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:02.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:02.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:02.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:02.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:02.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:02 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:03.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi053 to 5706M 2024-05-24T11:35:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:35:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:35:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: osdmap e46: 8 total, 7 up, 8 in 2024-05-24T11:35:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:03 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:03.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi053 to 5706M 2024-05-24T11:35:03.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:35:03.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:35:03.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: osdmap e46: 8 total, 7 up, 8 in 2024-05-24T11:35:03.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:03.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:03.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:03.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:03.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:03.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:03 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:03.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi053 to 5706M 2024-05-24T11:35:03.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-05-24T11:35:03.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:35:03.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: osdmap e46: 8 total, 7 up, 8 in 2024-05-24T11:35:03.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:03.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:03.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:03.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:03.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:03.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:03 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:04.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:04 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:35:04.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:04 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:35:04.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:04 smithi053 ceph-mon[47448]: osd.7 [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] boot 2024-05-24T11:35:04.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:04 smithi053 ceph-mon[47448]: osdmap e47: 8 total, 8 up, 8 in 2024-05-24T11:35:04.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:04 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:04.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:04 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:35:04.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:04 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:35:04.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:04 smithi137 ceph-mon[47907]: osd.7 [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] boot 2024-05-24T11:35:04.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:04 smithi137 ceph-mon[47907]: osdmap e47: 8 total, 8 up, 8 in 2024-05-24T11:35:04.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:04 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:04.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:04 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:35:04.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:04 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:35:04.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:04 smithi039 ceph-mon[42919]: osd.7 [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] boot 2024-05-24T11:35:04.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:04 smithi039 ceph-mon[42919]: osdmap e47: 8 total, 8 up, 8 in 2024-05-24T11:35:04.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:04 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:35:05.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:05 smithi053 ceph-mon[47448]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:05.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:05 smithi053 ceph-mon[47448]: osdmap e48: 8 total, 8 up, 8 in 2024-05-24T11:35:05.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:05 smithi053 ceph-mon[47448]: from='client.? 172.21.15.137:0/1093196125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ef14bbb-dbe7-497f-b170-89772a0b6ec0"}]: dispatch 2024-05-24T11:35:05.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:05 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ef14bbb-dbe7-497f-b170-89772a0b6ec0"}]: dispatch 2024-05-24T11:35:05.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:05 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ef14bbb-dbe7-497f-b170-89772a0b6ec0"}]': finished 2024-05-24T11:35:05.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:05 smithi053 ceph-mon[47448]: osdmap e49: 9 total, 8 up, 9 in 2024-05-24T11:35:05.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:05.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:05 smithi053 ceph-mon[47448]: from='client.? 172.21.15.137:0/1766269161' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:35:05.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:05 smithi137 ceph-mon[47907]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:05.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:05 smithi137 ceph-mon[47907]: osdmap e48: 8 total, 8 up, 8 in 2024-05-24T11:35:05.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:05 smithi137 ceph-mon[47907]: from='client.? 172.21.15.137:0/1093196125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ef14bbb-dbe7-497f-b170-89772a0b6ec0"}]: dispatch 2024-05-24T11:35:05.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:05 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ef14bbb-dbe7-497f-b170-89772a0b6ec0"}]: dispatch 2024-05-24T11:35:05.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:05 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ef14bbb-dbe7-497f-b170-89772a0b6ec0"}]': finished 2024-05-24T11:35:05.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:05 smithi137 ceph-mon[47907]: osdmap e49: 9 total, 8 up, 9 in 2024-05-24T11:35:05.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:05.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:05 smithi137 ceph-mon[47907]: from='client.? 172.21.15.137:0/1766269161' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:35:05.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:05 smithi039 ceph-mon[42919]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:05.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:05 smithi039 ceph-mon[42919]: osdmap e48: 8 total, 8 up, 8 in 2024-05-24T11:35:05.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:05 smithi039 ceph-mon[42919]: from='client.? 172.21.15.137:0/1093196125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ef14bbb-dbe7-497f-b170-89772a0b6ec0"}]: dispatch 2024-05-24T11:35:05.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:05 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ef14bbb-dbe7-497f-b170-89772a0b6ec0"}]: dispatch 2024-05-24T11:35:05.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:05 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ef14bbb-dbe7-497f-b170-89772a0b6ec0"}]': finished 2024-05-24T11:35:05.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:05 smithi039 ceph-mon[42919]: osdmap e49: 9 total, 8 up, 9 in 2024-05-24T11:35:05.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:05.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:05 smithi039 ceph-mon[42919]: from='client.? 172.21.15.137:0/1766269161' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:35:06.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:06 smithi039 ceph-mon[42919]: osdmap e50: 9 total, 8 up, 9 in 2024-05-24T11:35:06.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:06 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:06.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:06 smithi053 ceph-mon[47448]: osdmap e50: 9 total, 8 up, 9 in 2024-05-24T11:35:06.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:06 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:06.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:06 smithi137 ceph-mon[47907]: osdmap e50: 9 total, 8 up, 9 in 2024-05-24T11:35:06.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:06 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:07.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:07 smithi039 ceph-mon[42919]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:07.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:07 smithi039 ceph-mon[42919]: osdmap e51: 9 total, 8 up, 9 in 2024-05-24T11:35:07.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:07 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:07.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:07 smithi053 ceph-mon[47448]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:07.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:07 smithi053 ceph-mon[47448]: osdmap e51: 9 total, 8 up, 9 in 2024-05-24T11:35:07.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:07 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:07.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:07 smithi137 ceph-mon[47907]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:07.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:07 smithi137 ceph-mon[47907]: osdmap e51: 9 total, 8 up, 9 in 2024-05-24T11:35:07.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:07 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:09.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:09 smithi039 ceph-mon[42919]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:09.888 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:09 smithi137 ceph-mon[47907]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:09.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:09 smithi053 ceph-mon[47448]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:11.680 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:11 smithi137 ceph-mon[47907]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:11.680 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:11 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-24T11:35:11.680 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:11 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:11.680 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:11 smithi137 ceph-mon[47907]: Deploying daemon osd.8 on smithi137 2024-05-24T11:35:11.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:11 smithi039 ceph-mon[42919]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:11.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:11 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-24T11:35:11.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:11 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:11.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:11 smithi039 ceph-mon[42919]: Deploying daemon osd.8 on smithi137 2024-05-24T11:35:11.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:11 smithi053 ceph-mon[47448]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:11.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:11 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-24T11:35:11.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:11 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:11.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:11 smithi053 ceph-mon[47448]: Deploying daemon osd.8 on smithi137 2024-05-24T11:35:13.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:13 smithi137 ceph-mon[47907]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:13.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:13 smithi039 ceph-mon[42919]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:13.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:13 smithi053 ceph-mon[47448]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:15.241 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:15 smithi137 ceph-mon[47907]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:15.241 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:15 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:15.241 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:15 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:15.241 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:15 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:15.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:15 smithi053 ceph-mon[47448]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:15.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:15 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:15.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:15 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:15.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:15 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:15.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:15 smithi039 ceph-mon[42919]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:15.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:15 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:15.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:15 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:15.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:15 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:16.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.339+0000 7f7a5cff9700 1 -- 172.21.15.137:0/3014795608 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7a6002cf00 con 0x7f7a480a1120 2024-05-24T11:35:16.343 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 8 on host 'smithi137' 2024-05-24T11:35:16.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7a480a1120 msgr2=0x7f7a480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:16.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7a480a1120 0x7f7a480a35e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a54005950 tx=0x7f7a5400b410 comp rx=0 tx=0).stop 2024-05-24T11:35:16.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 msgr2=0x7f7a60196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:16.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 0x7f7a60196e10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a50000c00 tx=0x7f7a50003a50 comp rx=0 tx=0).stop 2024-05-24T11:35:16.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 shutdown_connections 2024-05-24T11:35:16.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7a480a1120 0x7f7a480a35e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:16.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a60101a90 0x7f7a60196e10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:16.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a60106cf0 0x7f7a6019e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:16.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 --2- 172.21.15.137:0/3014795608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a601043c0 0x7f7a60197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:16.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.341+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 >> 172.21.15.137:0/3014795608 conn(0x7f7a600fb4a0 msgr2=0x7f7a60103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:35:16.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.342+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 shutdown_connections 2024-05-24T11:35:16.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:16.342+0000 7f7a65cb9700 1 -- 172.21.15.137:0/3014795608 wait complete. 2024-05-24T11:35:16.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:16.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:16.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.712 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.713 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:16 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:16.714 DEBUG:teuthology.orchestra.run.smithi137:osd.8> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.8.service 2024-05-24T11:35:16.716 INFO:tasks.cephadm:Deploying osd.9 on smithi039 with /dev/vg_nvme/lv_1... 2024-05-24T11:35:16.716 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-24T11:35:16.759 INFO:journalctl@ceph.osd.8.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:35:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:16 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:16.972 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:35:16.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:16.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:16.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:16.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:16 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:17.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:17 smithi137 ceph-mon[47907]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:17.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:17 smithi039 ceph-mon[42919]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:17.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:17 smithi053 ceph-mon[47448]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:18.294 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-24T11:35:18.294 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-24T11:35:18.294 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-24T11:35:18.294 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-24T11:35:18.294 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112988 s, 928 MB/s 2024-05-24T11:35:18.294 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-24T11:35:18.470 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:35:18 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:35:18.148+0000 7f20f2f367c0 -1 osd.8 0 log_to_monitors true 2024-05-24T11:35:18.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:18 smithi039 ceph-mon[42919]: from='osd.8 [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-24T11:35:18.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:18 smithi039 ceph-mon[42919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-24T11:35:18.680 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_1 2024-05-24T11:35:18.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:18 smithi137 ceph-mon[47907]: from='osd.8 [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-24T11:35:18.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:18 smithi137 ceph-mon[47907]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-24T11:35:18.917 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:35:18.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:18 smithi053 ceph-mon[47448]: from='osd.8 [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-24T11:35:18.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:18 smithi053 ceph-mon[47448]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-24T11:35:19.769 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:19.769 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-24T11:35:19.769 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: osdmap e52: 9 total, 8 up, 9 in 2024-05-24T11:35:19.769 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:19.769 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='osd.8 [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:35:19.769 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:35:19.769 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi137 2024-05-24T11:35:19.769 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.770 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.770 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:19.770 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:19.770 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:19.770 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi137 to 5704M 2024-05-24T11:35:19.770 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.770 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:19.770 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:19.770 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:19 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3011173057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c0fff00 msgr2=0x7f1b6c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3011173057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c0fff00 0x7f1b6c100320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009990 tx=0x7f1b5c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:35:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3011173057 shutdown_connections 2024-05-24T11:35:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3011173057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b6c101a20 0x7f1b6c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3011173057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c0fff00 0x7f1b6c100320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3011173057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b6c101060 0x7f1b6c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3011173057 >> 172.21.15.39:0/3011173057 conn(0x7f1b6c0fb4a0 msgr2=0x7f1b6c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:35:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3011173057 shutdown_connections 2024-05-24T11:35:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3011173057 wait complete. 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.954+0000 7f1b71cf1700 1 Processor -- start 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b71cf1700 1 -- start start 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b71cf1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b6c0fff00 0x7f1b6c196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b71cf1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c101060 0x7f1b6c1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b71cf1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b6c101a20 0x7f1b6c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b71cf1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c19e980 con 0x7f1b6c101060 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b71cf1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c19eac0 con 0x7f1b6c101a20 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b71cf1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c19ec50 con 0x7f1b6c0fff00 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c101060 0x7f1b6c1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c101060 0x7f1b6c1973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45030/0 (socket says 172.21.15.39:45030) 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6affd700 1 -- 172.21.15.39:0/3024252931 learned_addr learned my addr 172.21.15.39:0/3024252931 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:35:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6bfff700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b6c101a20 0x7f1b6c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6b7fe700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b6c0fff00 0x7f1b6c196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6affd700 1 -- 172.21.15.39:0/3024252931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b6c0fff00 msgr2=0x7f1b6c196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6affd700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b6c0fff00 0x7f1b6c196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6affd700 1 -- 172.21.15.39:0/3024252931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b6c101a20 msgr2=0x7f1b6c19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6affd700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b6c101a20 0x7f1b6c19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6affd700 1 -- 172.21.15.39:0/3024252931 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c0095f0 con 0x7f1b6c101060 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6bfff700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b6c101a20 0x7f1b6c19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.955+0000 7f1b6affd700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c101060 0x7f1b6c1973d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000d980 tx=0x7f1b6000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.956+0000 7f1b68ff9700 1 -- 172.21.15.39:0/3024252931 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60009b40 con 0x7f1b6c101060 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.956+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c19ef30 con 0x7f1b6c101060 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.956+0000 7f1b68ff9700 1 -- 172.21.15.39:0/3024252931 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b6000b6e0 con 0x7f1b6c101060 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.956+0000 7f1b68ff9700 1 -- 172.21.15.39:0/3024252931 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000f620 con 0x7f1b6c101060 2024-05-24T11:35:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.956+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c19f3a0 con 0x7f1b6c101060 2024-05-24T11:35:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.957+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c04ec60 con 0x7f1b6c101060 2024-05-24T11:35:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.958+0000 7f1b68ff9700 1 -- 172.21.15.39:0/3024252931 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1b60010460 con 0x7f1b6c101060 2024-05-24T11:35:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.959+0000 7f1b68ff9700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b540a1120 0x7f1b540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.959+0000 7f1b6b7fe700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b540a1120 0x7f1b540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.959+0000 7f1b68ff9700 1 -- 172.21.15.39:0/3024252931 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5549+0+0 (secure 0 0 0) 0x7f1b600cbcc0 con 0x7f1b6c101060 2024-05-24T11:35:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.959+0000 7f1b6b7fe700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b540a1120 0x7f1b540a35e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00b5c0 tx=0x7f1b5c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:35:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:19.963+0000 7f1b68ff9700 1 -- 172.21.15.39:0/3024252931 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1b60095f20 con 0x7f1b6c101060 2024-05-24T11:35:19.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:19.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-24T11:35:19.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: osdmap e52: 9 total, 8 up, 9 in 2024-05-24T11:35:19.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='osd.8 [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:35:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:35:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi137 2024-05-24T11:35:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi137 to 5704M 2024-05-24T11:35:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:19 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.994 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:35:19 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:35:19.578+0000 7f20e7d99700 -1 osd.8 0 waiting for initial osdmap 2024-05-24T11:35:19.994 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:35:19 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:35:19.593+0000 7f20e438f700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:35:19.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: osdmap e52: 9 total, 8 up, 9 in 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='osd.8 [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi137 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi137 to 5704M 2024-05-24T11:35:19.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:19.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:19.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:19.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:19 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:20.176+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b6c069060 con 0x7f1b540a1120 2024-05-24T11:35:20.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:20 smithi039 ceph-mon[42919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:35:20.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:20 smithi039 ceph-mon[42919]: osdmap e53: 9 total, 8 up, 9 in 2024-05-24T11:35:20.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:20 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:20.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:20 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:20.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:20 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:20.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:20 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:20.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:20 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:20.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:20 smithi053 ceph-mon[47448]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:35:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:20 smithi053 ceph-mon[47448]: osdmap e53: 9 total, 8 up, 9 in 2024-05-24T11:35:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:20.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:20 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:20.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:20 smithi137 ceph-mon[47907]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:35:20.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:20 smithi137 ceph-mon[47907]: osdmap e53: 9 total, 8 up, 9 in 2024-05-24T11:35:20.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:20 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:20.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:20 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:20.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:20 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:20.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:20 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:20.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:20 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:21.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:21 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:35:21.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:21 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:35:21.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:21 smithi053 ceph-mon[47448]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:21.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:21 smithi053 ceph-mon[47448]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:21.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:21 smithi053 ceph-mon[47448]: osd.8 [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] boot 2024-05-24T11:35:21.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:21 smithi053 ceph-mon[47448]: osdmap e54: 9 total, 9 up, 9 in 2024-05-24T11:35:21.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:21.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:21 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:35:21.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:21 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:35:21.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:21 smithi137 ceph-mon[47907]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:21.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:21 smithi137 ceph-mon[47907]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:21.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:21 smithi137 ceph-mon[47907]: osd.8 [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] boot 2024-05-24T11:35:21.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:21 smithi137 ceph-mon[47907]: osdmap e54: 9 total, 9 up, 9 in 2024-05-24T11:35:21.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:21 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:35:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:21 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:35:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:21 smithi039 ceph-mon[42919]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-24T11:35:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:21 smithi039 ceph-mon[42919]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:21 smithi039 ceph-mon[42919]: osd.8 [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] boot 2024-05-24T11:35:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:21 smithi039 ceph-mon[42919]: osdmap e54: 9 total, 9 up, 9 in 2024-05-24T11:35:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:35:22.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:22 smithi053 ceph-mon[47448]: osdmap e55: 9 total, 9 up, 9 in 2024-05-24T11:35:22.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:22 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2388495652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32dc829d-7871-4c18-890f-82f263600944"}]: dispatch 2024-05-24T11:35:22.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:22 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2388495652' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32dc829d-7871-4c18-890f-82f263600944"}]': finished 2024-05-24T11:35:22.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:22 smithi053 ceph-mon[47448]: osdmap e56: 10 total, 9 up, 10 in 2024-05-24T11:35:22.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:22 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:22.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:22 smithi137 ceph-mon[47907]: osdmap e55: 9 total, 9 up, 9 in 2024-05-24T11:35:22.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:22 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2388495652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32dc829d-7871-4c18-890f-82f263600944"}]: dispatch 2024-05-24T11:35:22.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:22 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2388495652' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32dc829d-7871-4c18-890f-82f263600944"}]': finished 2024-05-24T11:35:22.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:22 smithi137 ceph-mon[47907]: osdmap e56: 10 total, 9 up, 10 in 2024-05-24T11:35:22.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:22 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:23.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:22 smithi039 ceph-mon[42919]: osdmap e55: 9 total, 9 up, 9 in 2024-05-24T11:35:23.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:22 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2388495652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32dc829d-7871-4c18-890f-82f263600944"}]: dispatch 2024-05-24T11:35:23.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:22 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2388495652' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32dc829d-7871-4c18-890f-82f263600944"}]': finished 2024-05-24T11:35:23.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:22 smithi039 ceph-mon[42919]: osdmap e56: 10 total, 9 up, 10 in 2024-05-24T11:35:23.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:22 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:24.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:23 smithi053 ceph-mon[47448]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:24.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:23 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1605647260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:35:24.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:23 smithi053 ceph-mon[47448]: osdmap e57: 10 total, 9 up, 10 in 2024-05-24T11:35:24.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:24.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:23 smithi137 ceph-mon[47907]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:23 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1605647260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:35:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:23 smithi137 ceph-mon[47907]: osdmap e57: 10 total, 9 up, 10 in 2024-05-24T11:35:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:24.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:23 smithi039 ceph-mon[42919]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:24.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:23 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1605647260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:35:24.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:23 smithi039 ceph-mon[42919]: osdmap e57: 10 total, 9 up, 10 in 2024-05-24T11:35:24.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:25.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:25 smithi039 ceph-mon[42919]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-24T11:35:25.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:25 smithi039 ceph-mon[42919]: osdmap e58: 10 total, 9 up, 10 in 2024-05-24T11:35:25.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:25.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:25 smithi053 ceph-mon[47448]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-24T11:35:25.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:25 smithi053 ceph-mon[47448]: osdmap e58: 10 total, 9 up, 10 in 2024-05-24T11:35:25.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:25.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:25 smithi137 ceph-mon[47907]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-24T11:35:25.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:25 smithi137 ceph-mon[47907]: osdmap e58: 10 total, 9 up, 10 in 2024-05-24T11:35:25.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:27.397 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:27 smithi039 ceph-mon[42919]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-24T11:35:27.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:27 smithi053 ceph-mon[47448]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-24T11:35:27.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:27 smithi137 ceph-mon[47907]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-24T11:35:29.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:29 smithi039 ceph-mon[42919]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-24T11:35:29.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:29 smithi053 ceph-mon[47448]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-24T11:35:29.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:29 smithi137 ceph-mon[47907]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-24T11:35:30.378 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-24T11:35:30.378 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:30.378 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:30 smithi039 ceph-mon[42919]: Deploying daemon osd.9 on smithi039 2024-05-24T11:35:30.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-24T11:35:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:30 smithi053 ceph-mon[47448]: Deploying daemon osd.9 on smithi039 2024-05-24T11:35:30.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-24T11:35:30.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:30.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:30 smithi137 ceph-mon[47907]: Deploying daemon osd.9 on smithi039 2024-05-24T11:35:31.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:31 smithi039 ceph-mon[42919]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-24T11:35:31.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:31 smithi053 ceph-mon[47448]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-24T11:35:31.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:31 smithi137 ceph-mon[47907]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-24T11:35:33.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:33 smithi039 ceph-mon[42919]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-24T11:35:33.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:33 smithi053 ceph-mon[47448]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-24T11:35:33.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:33 smithi137 ceph-mon[47907]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-24T11:35:34.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:34.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:34.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:34.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:34.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:34.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:34.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:34.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:34.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:35.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:35 smithi039 ceph-mon[42919]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:35.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:35.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:35.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:35.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:35.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:35.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:35 smithi053 ceph-mon[47448]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:35.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:35.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:35.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:35.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:35.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:35.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:35 smithi137 ceph-mon[47907]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:35.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:35.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:35.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:35.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:35.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:35.788 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 9 on host 'smithi039' 2024-05-24T11:35:35.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.784+0000 7f1b68ff9700 1 -- 172.21.15.39:0/3024252931 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1b6c069060 con 0x7f1b540a1120 2024-05-24T11:35:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b540a1120 msgr2=0x7f1b540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b540a1120 0x7f1b540a35e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00b5c0 tx=0x7f1b5c005fb0 comp rx=0 tx=0).stop 2024-05-24T11:35:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c101060 msgr2=0x7f1b6c1973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c101060 0x7f1b6c1973d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000d980 tx=0x7f1b6000dd40 comp rx=0 tx=0).stop 2024-05-24T11:35:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 shutdown_connections 2024-05-24T11:35:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b540a1120 0x7f1b540a35e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b6c0fff00 0x7f1b6c196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b6c101a20 0x7f1b6c19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 --2- 172.21.15.39:0/3024252931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c101060 0x7f1b6c1973d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:35.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 >> 172.21.15.39:0/3024252931 conn(0x7f1b6c0fb4a0 msgr2=0x7f1b6c068950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:35:35.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 shutdown_connections 2024-05-24T11:35:35.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:35:35.787+0000 7f1b71cf1700 1 -- 172.21.15.39:0/3024252931 wait complete. 2024-05-24T11:35:36.195 DEBUG:teuthology.orchestra.run.smithi039:osd.9> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.9.service 2024-05-24T11:35:36.197 INFO:tasks.cephadm:Deploying osd.10 on smithi053 with /dev/vg_nvme/lv_1... 2024-05-24T11:35:36.197 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-24T11:35:36.231 INFO:journalctl@ceph.osd.9.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:35:36.463 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:35:36.880 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:36 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:36.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:36 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:36.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:36 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:37.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:36 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:37.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:36 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:37.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:36 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:37.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:36 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:37.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:36 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:37.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:36 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:37.577 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:35:37 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:35:37.330+0000 7f9fd90f97c0 -1 osd.9 0 log_to_monitors true 2024-05-24T11:35:37.806 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-24T11:35:37.806 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-24T11:35:37.807 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2024-05-24T11:35:37.807 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2024-05-24T11:35:37.807 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00942173 s, 1.1 GB/s 2024-05-24T11:35:37.807 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2024-05-24T11:35:38.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:37 smithi039 ceph-mon[42919]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:38.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:37 smithi039 ceph-mon[42919]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-24T11:35:38.085 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:37 smithi053 ceph-mon[47448]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:38.085 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:37 smithi053 ceph-mon[47448]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-24T11:35:38.226 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-05-24T11:35:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:37 smithi137 ceph-mon[47907]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:37 smithi137 ceph-mon[47907]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-24T11:35:38.481 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:35:38.880 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-24T11:35:38.880 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: osdmap e59: 10 total, 9 up, 10 in 2024-05-24T11:35:38.880 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:38.880 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:35:38.880 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: osdmap e60: 10 total, 9 up, 10 in 2024-05-24T11:35:38.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:39.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-24T11:35:39.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: osdmap e59: 10 total, 9 up, 10 in 2024-05-24T11:35:39.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:39.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:35:39.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:35:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: osdmap e60: 10 total, 9 up, 10 in 2024-05-24T11:35:39.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: osdmap e59: 10 total, 9 up, 10 in 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:39.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:39.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:39.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-24T11:35:39.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: osdmap e60: 10 total, 9 up, 10 in 2024-05-24T11:35:39.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:39.475 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.470+0000 7fc1919cc700 1 -- 172.21.15.53:0/168652306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 msgr2=0x7fc18c101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:39.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.470+0000 7fc1919cc700 1 --2- 172.21.15.53:0/168652306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 0x7fc18c101400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0099c0 tx=0x7fc17c009cd0 comp rx=0 tx=0).stop 2024-05-24T11:35:39.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.470+0000 7fc1919cc700 1 -- 172.21.15.53:0/168652306 shutdown_connections 2024-05-24T11:35:39.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.470+0000 7fc1919cc700 1 --2- 172.21.15.53:0/168652306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c102b80 0x7fc18c109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:39.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.470+0000 7fc1919cc700 1 --2- 172.21.15.53:0/168652306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 0x7fc18c101400 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:39.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.470+0000 7fc1919cc700 1 --2- 172.21.15.53:0/168652306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc18c1021e0 0x7fc18c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:39.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.470+0000 7fc1919cc700 1 -- 172.21.15.53:0/168652306 >> 172.21.15.53:0/168652306 conn(0x7fc18c0fc580 msgr2=0x7fc18c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:35:39.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.470+0000 7fc1919cc700 1 -- 172.21.15.53:0/168652306 shutdown_connections 2024-05-24T11:35:39.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.470+0000 7fc1919cc700 1 -- 172.21.15.53:0/168652306 wait complete. 2024-05-24T11:35:39.476 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.471+0000 7fc1919cc700 1 Processor -- start 2024-05-24T11:35:39.477 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.471+0000 7fc1919cc700 1 -- start start 2024-05-24T11:35:39.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc1919cc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 0x7fc18c196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:39.478 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc1919cc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c1021e0 0x7fc18c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc1919cc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc18c102b80 0x7fc18c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc1919cc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c19e9c0 con 0x7fc18c1021e0 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc1919cc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c19eb00 con 0x7fc18c100fe0 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc1919cc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c19ec90 con 0x7fc18c102b80 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 0x7fc18c196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 0x7fc18c196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44276/0 (socket says 172.21.15.53:44276) 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18affd700 1 -- 172.21.15.53:0/2702522314 learned_addr learned my addr 172.21.15.53:0/2702522314 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc1827fc700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c1021e0 0x7fc18c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18b7fe700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc18c102b80 0x7fc18c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18affd700 1 -- 172.21.15.53:0/2702522314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc18c102b80 msgr2=0x7fc18c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18affd700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc18c102b80 0x7fc18c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18affd700 1 -- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c1021e0 msgr2=0x7fc18c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:39.479 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18affd700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c1021e0 0x7fc18c1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:39.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18affd700 1 -- 172.21.15.53:0/2702522314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c0095f0 con 0x7fc18c100fe0 2024-05-24T11:35:39.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc18affd700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 0x7fc18c196e80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009f50 tx=0x7fc17c005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:35:39.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.472+0000 7fc1909ca700 1 -- 172.21.15.53:0/2702522314 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c01d070 con 0x7fc18c100fe0 2024-05-24T11:35:39.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.473+0000 7fc1909ca700 1 -- 172.21.15.53:0/2702522314 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc17c00bda0 con 0x7fc18c100fe0 2024-05-24T11:35:39.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.473+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c19ef10 con 0x7fc18c100fe0 2024-05-24T11:35:39.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.473+0000 7fc1909ca700 1 -- 172.21.15.53:0/2702522314 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c022ca0 con 0x7fc18c100fe0 2024-05-24T11:35:39.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.473+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c19f2c0 con 0x7fc18c100fe0 2024-05-24T11:35:39.480 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.474+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18c1084b0 con 0x7fc18c100fe0 2024-05-24T11:35:39.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.475+0000 7fc1909ca700 1 -- 172.21.15.53:0/2702522314 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc17c022470 con 0x7fc18c100fe0 2024-05-24T11:35:39.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.476+0000 7fc1909ca700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc1780a1120 0x7fc1780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:39.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.476+0000 7fc1827fc700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc1780a1120 0x7fc1780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:39.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.476+0000 7fc1909ca700 1 -- 172.21.15.53:0/2702522314 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5981+0+0 (secure 0 0 0) 0x7fc17c0cd730 con 0x7fc18c100fe0 2024-05-24T11:35:39.481 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.476+0000 7fc1827fc700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc1780a1120 0x7fc1780a35e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc174009a10 tx=0x7fc174009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:35:39.484 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.480+0000 7fc1909ca700 1 -- 172.21.15.53:0/2702522314 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc17c0977d0 con 0x7fc18c100fe0 2024-05-24T11:35:39.706 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:39.701+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc18c02ce40 con 0x7fc1780a1120 2024-05-24T11:35:39.827 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:35:39 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:35:39.553+0000 7f9fcdf5c700 -1 osd.9 0 waiting for initial osdmap 2024-05-24T11:35:39.827 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:35:39 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:35:39.572+0000 7f9fc654a700 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:35:39.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:39 smithi039 ceph-mon[42919]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:39 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi039 2024-05-24T11:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:39 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi039 to 4278M 2024-05-24T11:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:39 smithi039 ceph-mon[42919]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' 2024-05-24T11:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:40.087 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:39 smithi053 ceph-mon[47448]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:40.087 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:39 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi039 2024-05-24T11:35:40.087 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:39 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi039 to 4278M 2024-05-24T11:35:40.087 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:40.087 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:39 smithi053 ceph-mon[47448]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' 2024-05-24T11:35:40.087 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:40.088 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:40.088 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:40.088 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:40.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:39 smithi137 ceph-mon[47907]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:40.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:39 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi039 2024-05-24T11:35:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:39 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi039 to 4278M 2024-05-24T11:35:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:39 smithi137 ceph-mon[47907]: from='osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773]' entity='osd.9' 2024-05-24T11:35:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:40 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:35:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:40 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:35:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:40 smithi053 ceph-mon[47448]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:40 smithi053 ceph-mon[47448]: osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] boot 2024-05-24T11:35:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:40 smithi053 ceph-mon[47448]: osdmap e61: 10 total, 10 up, 10 in 2024-05-24T11:35:40.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:41.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:40 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:35:41.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:40 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:35:41.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:40 smithi039 ceph-mon[42919]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:41.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:40 smithi039 ceph-mon[42919]: osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] boot 2024-05-24T11:35:41.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:40 smithi039 ceph-mon[42919]: osdmap e61: 10 total, 10 up, 10 in 2024-05-24T11:35:41.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:40 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:35:41.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:40 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:35:41.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:40 smithi137 ceph-mon[47907]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:41.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:40 smithi137 ceph-mon[47907]: osd.9 [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] boot 2024-05-24T11:35:41.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:40 smithi137 ceph-mon[47907]: osdmap e61: 10 total, 10 up, 10 in 2024-05-24T11:35:41.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:35:42.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:41 smithi039 ceph-mon[42919]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:42.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:41 smithi039 ceph-mon[42919]: from='client.? 172.21.15.53:0/3709309065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b20b6f-c19f-4c58-aafa-feb7cc027de3"}]: dispatch 2024-05-24T11:35:42.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:41 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b20b6f-c19f-4c58-aafa-feb7cc027de3"}]: dispatch 2024-05-24T11:35:42.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:41 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1b20b6f-c19f-4c58-aafa-feb7cc027de3"}]': finished 2024-05-24T11:35:42.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:41 smithi039 ceph-mon[42919]: osdmap e62: 11 total, 10 up, 11 in 2024-05-24T11:35:42.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:42.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:41 smithi053 ceph-mon[47448]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:42.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:41 smithi053 ceph-mon[47448]: from='client.? 172.21.15.53:0/3709309065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b20b6f-c19f-4c58-aafa-feb7cc027de3"}]: dispatch 2024-05-24T11:35:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:41 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b20b6f-c19f-4c58-aafa-feb7cc027de3"}]: dispatch 2024-05-24T11:35:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:41 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1b20b6f-c19f-4c58-aafa-feb7cc027de3"}]': finished 2024-05-24T11:35:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:41 smithi053 ceph-mon[47448]: osdmap e62: 11 total, 10 up, 11 in 2024-05-24T11:35:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:41 smithi137 ceph-mon[47907]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-24T11:35:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:41 smithi137 ceph-mon[47907]: from='client.? 172.21.15.53:0/3709309065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b20b6f-c19f-4c58-aafa-feb7cc027de3"}]: dispatch 2024-05-24T11:35:42.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:41 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1b20b6f-c19f-4c58-aafa-feb7cc027de3"}]: dispatch 2024-05-24T11:35:42.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:41 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1b20b6f-c19f-4c58-aafa-feb7cc027de3"}]': finished 2024-05-24T11:35:42.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:41 smithi137 ceph-mon[47907]: osdmap e62: 11 total, 10 up, 11 in 2024-05-24T11:35:42.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:43.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:42 smithi039 ceph-mon[42919]: from='client.? 172.21.15.53:0/2300851685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:35:43.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:42 smithi053 ceph-mon[47448]: from='client.? 172.21.15.53:0/2300851685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:35:43.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:42 smithi137 ceph-mon[47907]: from='client.? 172.21.15.53:0/2300851685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:35:44.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:43 smithi039 ceph-mon[42919]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:44.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:43 smithi053 ceph-mon[47448]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:44.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:43 smithi137 ceph-mon[47907]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:45.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:45 smithi053 ceph-mon[47448]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:46.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:45 smithi039 ceph-mon[42919]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:46.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:45 smithi137 ceph-mon[47907]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:46.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:46 smithi053 ceph-mon[47448]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:47.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:46 smithi137 ceph-mon[47907]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:47.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:46 smithi039 ceph-mon[42919]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:48.229 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-24T11:35:48.230 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:48.230 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:47 smithi053 ceph-mon[47448]: Deploying daemon osd.10 on smithi053 2024-05-24T11:35:48.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-24T11:35:48.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:48.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:47 smithi137 ceph-mon[47907]: Deploying daemon osd.10 on smithi053 2024-05-24T11:35:48.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-24T11:35:48.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:48.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:47 smithi039 ceph-mon[42919]: Deploying daemon osd.10 on smithi053 2024-05-24T11:35:49.170 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:48 smithi053 ceph-mon[47448]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:49.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:48 smithi137 ceph-mon[47907]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:49.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:48 smithi039 ceph-mon[42919]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:51.210 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:50 smithi053 ceph-mon[47448]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:51.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:50 smithi137 ceph-mon[47907]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:51.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:50 smithi039 ceph-mon[42919]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:52 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:52 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:52 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:52.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:52 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:52.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:52 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:52.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:52 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:52.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:52 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:52.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:52 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:52.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:52 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:53.559 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.553+0000 7fc1909ca700 1 -- 172.21.15.53:0/2702522314 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc18c02ce40 con 0x7fc1780a1120 2024-05-24T11:35:53.559 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 10 on host 'smithi053' 2024-05-24T11:35:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc1780a1120 msgr2=0x7fc1780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc1780a1120 0x7fc1780a35e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc174009a10 tx=0x7fc174009190 comp rx=0 tx=0).stop 2024-05-24T11:35:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 msgr2=0x7fc18c196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 0x7fc18c196e80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009f50 tx=0x7fc17c005b00 comp rx=0 tx=0).stop 2024-05-24T11:35:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 shutdown_connections 2024-05-24T11:35:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc1780a1120 0x7fc1780a35e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:53.561 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc18c102b80 0x7fc18c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc18c100fe0 0x7fc18c196e80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 --2- 172.21.15.53:0/2702522314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc18c1021e0 0x7fc18c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.556+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 >> 172.21.15.53:0/2702522314 conn(0x7fc18c0fc580 msgr2=0x7fc18c1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:35:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.557+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 shutdown_connections 2024-05-24T11:35:53.562 INFO:teuthology.orchestra.run.smithi053.stderr:2024-05-24T11:35:53.557+0000 7fc1919cc700 1 -- 172.21.15.53:0/2702522314 wait complete. 2024-05-24T11:35:53.932 DEBUG:teuthology.orchestra.run.smithi053:osd.10> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.10.service 2024-05-24T11:35:53.934 INFO:tasks.cephadm:Deploying osd.11 on smithi137 with /dev/vg_nvme/lv_1... 2024-05-24T11:35:53.935 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-24T11:35:53.967 INFO:journalctl@ceph.osd.10.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:35:54.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:53 smithi053 ceph-mon[47448]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:54.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:54.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:54.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.021 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:53 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:54.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:53 smithi039 ceph-mon[42919]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:54.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:54.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:54.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:53 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:54.168 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:35:54.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:53 smithi137 ceph-mon[47907]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:54.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:54.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:54.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:54.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:53 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:35:55.437 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-24T11:35:55.438 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-24T11:35:55.438 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-05-24T11:35:55.438 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-05-24T11:35:55.438 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0155035 s, 676 MB/s 2024-05-24T11:35:55.438 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-05-24T11:35:55.739 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:35:55 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:35:55.346+0000 7f461a3ad7c0 -1 osd.10 0 log_to_monitors true 2024-05-24T11:35:55.827 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-05-24T11:35:56.046 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:35:56.146 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:55 smithi137 ceph-mon[47907]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:56.147 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:55 smithi137 ceph-mon[47907]: from='osd.10 [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-24T11:35:56.147 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:55 smithi137 ceph-mon[47907]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-24T11:35:56.203 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:55 smithi053 ceph-mon[47448]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:56.203 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:55 smithi053 ceph-mon[47448]: from='osd.10 [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-24T11:35:56.204 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:55 smithi053 ceph-mon[47448]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-24T11:35:56.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:55 smithi039 ceph-mon[42919]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:56.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:55 smithi039 ceph-mon[42919]: from='osd.10 [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-24T11:35:56.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:55 smithi039 ceph-mon[42919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-24T11:35:56.838 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-24T11:35:56.838 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: osdmap e63: 11 total, 10 up, 11 in 2024-05-24T11:35:56.838 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:56.838 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='osd.10 [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:56.838 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:56.838 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:56.838 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi053 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi053 to 4279M 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: osdmap e64: 11 total, 10 up, 11 in 2024-05-24T11:35:56.839 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:56 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:57.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.021+0000 7fe77a572700 1 -- 172.21.15.137:0/286493224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 msgr2=0x7fe774108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.021+0000 7fe77a572700 1 --2- 172.21.15.137:0/286493224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 0x7fe774108340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00b0a0 tx=0x7fe76c00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.021+0000 7fe77a572700 1 -- 172.21.15.137:0/286493224 shutdown_connections 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.021+0000 7fe77a572700 1 --2- 172.21.15.137:0/286493224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 0x7fe774108340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.021+0000 7fe77a572700 1 --2- 172.21.15.137:0/286493224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7740fff00 0x7fe774100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.021+0000 7fe77a572700 1 --2- 172.21.15.137:0/286493224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe774101060 0x7fe7741014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.021+0000 7fe77a572700 1 -- 172.21.15.137:0/286493224 >> 172.21.15.137:0/286493224 conn(0x7fe7740fb4a0 msgr2=0x7fe7740fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.021+0000 7fe77a572700 1 -- 172.21.15.137:0/286493224 shutdown_connections 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.021+0000 7fe77a572700 1 -- 172.21.15.137:0/286493224 wait complete. 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe77a572700 1 Processor -- start 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe77a572700 1 -- start start 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe77a572700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7740fff00 0x7fe7740694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:57.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe77a572700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe774101060 0x7fe77419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe77a572700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 0x7fe77419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe77a572700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77419e7c0 con 0x7fe774101060 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe77a572700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77419e930 con 0x7fe7740fff00 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe77a572700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77419eaf0 con 0x7fe774101a20 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe778b0f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 0x7fe77419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe778b0f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 0x7fe77419e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37130/0 (socket says 172.21.15.137:37130) 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe778b0f700 1 -- 172.21.15.137:0/2681647266 learned_addr learned my addr 172.21.15.137:0/2681647266 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe773fff700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7740fff00 0x7fe7740694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.022+0000 7fe7737fe700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe774101060 0x7fe77419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe778b0f700 1 -- 172.21.15.137:0/2681647266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7740fff00 msgr2=0x7fe7740694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe778b0f700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7740fff00 0x7fe7740694c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe778b0f700 1 -- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe774101060 msgr2=0x7fe77419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe778b0f700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe774101060 0x7fe77419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:35:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe778b0f700 1 -- 172.21.15.137:0/2681647266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c009d00 con 0x7fe774101a20 2024-05-24T11:35:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe7737fe700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe774101060 0x7fe77419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:35:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe778b0f700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 0x7fe77419e020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00bd60 tx=0x7fe76c00bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:35:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe7717fa700 1 -- 172.21.15.137:0/2681647266 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00e040 con 0x7fe774101a20 2024-05-24T11:35:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77419ed70 con 0x7fe774101a20 2024-05-24T11:35:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe7717fa700 1 -- 172.21.15.137:0/2681647266 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe76c004e60 con 0x7fe774101a20 2024-05-24T11:35:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe7717fa700 1 -- 172.21.15.137:0/2681647266 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c0052a0 con 0x7fe774101a20 2024-05-24T11:35:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.023+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77419f190 con 0x7fe774101a20 2024-05-24T11:35:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.024+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe774107320 con 0x7fe774101a20 2024-05-24T11:35:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.025+0000 7fe7717fa700 1 -- 172.21.15.137:0/2681647266 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe76c01d470 con 0x7fe774101a20 2024-05-24T11:35:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.025+0000 7fe7717fa700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe75c0a1120 0x7fe75c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:35:57.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.025+0000 7fe7717fa700 1 -- 172.21.15.137:0/2681647266 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe76c0cd6e0 con 0x7fe774101a20 2024-05-24T11:35:57.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.026+0000 7fe773fff700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe75c0a1120 0x7fe75c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:35:57.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.026+0000 7fe773fff700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe75c0a1120 0x7fe75c0a35e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe768005950 tx=0x7fe76800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:35:57.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.030+0000 7fe7717fa700 1 -- 172.21.15.137:0/2681647266 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe76c097670 con 0x7fe774101a20 2024-05-24T11:35:57.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:35:57.229+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe77402cf00 con 0x7fe75c0a1120 2024-05-24T11:35:57.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-24T11:35:57.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: osdmap e63: 11 total, 10 up, 11 in 2024-05-24T11:35:57.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='osd.10 [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi053 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi053 to 4279M 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:57.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:57.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:35:57.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: osdmap e64: 11 total, 10 up, 11 in 2024-05-24T11:35:57.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:56 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:57.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-24T11:35:57.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: osdmap e63: 11 total, 10 up, 11 in 2024-05-24T11:35:57.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='osd.10 [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi053 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi053 to 4279M 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:57.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:35:57.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:35:57.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-05-24T11:35:57.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: osdmap e64: 11 total, 10 up, 11 in 2024-05-24T11:35:57.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:56 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:57.989 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:35:57 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:35:57.642+0000 7f460f210700 -1 osd.10 0 waiting for initial osdmap 2024-05-24T11:35:57.989 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:35:57 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:35:57.658+0000 7f46077fe700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:35:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:57 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:57 smithi053 ceph-mon[47448]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:57 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:57 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:57 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:57 smithi053 ceph-mon[47448]: from='osd.10 ' entity='osd.10' 2024-05-24T11:35:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:57 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:58.205 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:57 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:58.206 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:57 smithi137 ceph-mon[47907]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:58.206 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:57 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:58.206 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:57 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:58.206 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:57 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:58.206 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:57 smithi137 ceph-mon[47907]: from='osd.10 ' entity='osd.10' 2024-05-24T11:35:58.206 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:57 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:58.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:57 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:58.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:57 smithi039 ceph-mon[42919]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:35:58.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:57 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-24T11:35:58.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:57 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-24T11:35:58.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:57 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:35:58.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:57 smithi039 ceph-mon[42919]: from='osd.10 ' entity='osd.10' 2024-05-24T11:35:58.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:57 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:59.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:58 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:35:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:58 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:35:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:58 smithi053 ceph-mon[47448]: osd.10 [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] boot 2024-05-24T11:35:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:58 smithi053 ceph-mon[47448]: osdmap e65: 11 total, 11 up, 11 in 2024-05-24T11:35:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:58 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:58 smithi053 ceph-mon[47448]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:59.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:58 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:35:59.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:58 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:35:59.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:58 smithi137 ceph-mon[47907]: osd.10 [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] boot 2024-05-24T11:35:59.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:58 smithi137 ceph-mon[47907]: osdmap e65: 11 total, 11 up, 11 in 2024-05-24T11:35:59.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:58 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:59.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:58 smithi137 ceph-mon[47907]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:35:59.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:58 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:35:59.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:58 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:35:59.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:58 smithi039 ceph-mon[42919]: osd.10 [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] boot 2024-05-24T11:35:59.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:58 smithi039 ceph-mon[42919]: osdmap e65: 11 total, 11 up, 11 in 2024-05-24T11:35:59.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:58 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:35:59.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:58 smithi039 ceph-mon[42919]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-05-24T11:36:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:59 smithi053 ceph-mon[47448]: osdmap e66: 11 total, 11 up, 11 in 2024-05-24T11:36:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:59 smithi053 ceph-mon[47448]: from='client.? 172.21.15.137:0/1667356957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78bc93a0-953b-478a-9d7e-25a4005fd6a7"}]: dispatch 2024-05-24T11:36:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:59 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78bc93a0-953b-478a-9d7e-25a4005fd6a7"}]: dispatch 2024-05-24T11:36:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:59 smithi053 ceph-mon[47448]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78bc93a0-953b-478a-9d7e-25a4005fd6a7"}]': finished 2024-05-24T11:36:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:59 smithi053 ceph-mon[47448]: osdmap e67: 12 total, 11 up, 12 in 2024-05-24T11:36:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:59 smithi053 ceph-mon[47448]: osdmap e68: 12 total, 11 up, 12 in 2024-05-24T11:36:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:59 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:35:59 smithi053 ceph-mon[47448]: from='client.? 172.21.15.137:0/1775690137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:36:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:59 smithi137 ceph-mon[47907]: osdmap e66: 11 total, 11 up, 11 in 2024-05-24T11:36:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:59 smithi137 ceph-mon[47907]: from='client.? 172.21.15.137:0/1667356957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78bc93a0-953b-478a-9d7e-25a4005fd6a7"}]: dispatch 2024-05-24T11:36:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:59 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78bc93a0-953b-478a-9d7e-25a4005fd6a7"}]: dispatch 2024-05-24T11:36:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:59 smithi137 ceph-mon[47907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78bc93a0-953b-478a-9d7e-25a4005fd6a7"}]': finished 2024-05-24T11:36:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:59 smithi137 ceph-mon[47907]: osdmap e67: 12 total, 11 up, 12 in 2024-05-24T11:36:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:59 smithi137 ceph-mon[47907]: osdmap e68: 12 total, 11 up, 12 in 2024-05-24T11:36:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:59 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:35:59 smithi137 ceph-mon[47907]: from='client.? 172.21.15.137:0/1775690137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:36:00.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:59 smithi039 ceph-mon[42919]: osdmap e66: 11 total, 11 up, 11 in 2024-05-24T11:36:00.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:59 smithi039 ceph-mon[42919]: from='client.? 172.21.15.137:0/1667356957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78bc93a0-953b-478a-9d7e-25a4005fd6a7"}]: dispatch 2024-05-24T11:36:00.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:59 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78bc93a0-953b-478a-9d7e-25a4005fd6a7"}]: dispatch 2024-05-24T11:36:00.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:59 smithi039 ceph-mon[42919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78bc93a0-953b-478a-9d7e-25a4005fd6a7"}]': finished 2024-05-24T11:36:00.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:59 smithi039 ceph-mon[42919]: osdmap e67: 12 total, 11 up, 12 in 2024-05-24T11:36:00.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:00.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:59 smithi039 ceph-mon[42919]: osdmap e68: 12 total, 11 up, 12 in 2024-05-24T11:36:00.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:59 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:00.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:35:59 smithi039 ceph-mon[42919]: from='client.? 172.21.15.137:0/1775690137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-24T11:36:01.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:01 smithi039 ceph-mon[42919]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-05-24T11:36:01.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:01 smithi039 ceph-mon[42919]: osdmap e69: 12 total, 11 up, 12 in 2024-05-24T11:36:01.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:01 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:01.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:01 smithi053 ceph-mon[47448]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-05-24T11:36:01.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:01 smithi053 ceph-mon[47448]: osdmap e69: 12 total, 11 up, 12 in 2024-05-24T11:36:01.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:01 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:01.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:01 smithi137 ceph-mon[47907]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-05-24T11:36:01.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:01 smithi137 ceph-mon[47907]: osdmap e69: 12 total, 11 up, 12 in 2024-05-24T11:36:01.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:01 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:03.896 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:03 smithi137 ceph-mon[47907]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-05-24T11:36:03.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:03 smithi053 ceph-mon[47448]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-05-24T11:36:04.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:03 smithi039 ceph-mon[42919]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 339 MiB used, 983 GiB / 983 GiB avail 2024-05-24T11:36:05.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:05 smithi137 ceph-mon[47907]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-24T11:36:05.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-24T11:36:05.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:05 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:05.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:05 smithi053 ceph-mon[47448]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-24T11:36:05.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-24T11:36:05.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:05 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:06.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:05 smithi039 ceph-mon[42919]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-24T11:36:06.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-24T11:36:06.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:05 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:06.890 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:06 smithi137 ceph-mon[47907]: Deploying daemon osd.11 on smithi137 2024-05-24T11:36:06.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:06 smithi053 ceph-mon[47448]: Deploying daemon osd.11 on smithi137 2024-05-24T11:36:07.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:06 smithi039 ceph-mon[42919]: Deploying daemon osd.11 on smithi137 2024-05-24T11:36:07.871 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:07 smithi137 ceph-mon[47907]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-24T11:36:07.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:07 smithi053 ceph-mon[47448]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-24T11:36:08.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:07 smithi039 ceph-mon[42919]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-24T11:36:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:09 smithi137 ceph-mon[47907]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-24T11:36:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:09.365 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:09 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:09.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:09 smithi053 ceph-mon[47448]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-24T11:36:09.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:09.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:09.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:09 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:09.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:09 smithi039 ceph-mon[42919]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-24T11:36:09.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:09.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:09.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:09 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:10.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.331+0000 7fe7717fa700 1 -- 172.21.15.137:0/2681647266 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fe77402cf00 con 0x7fe75c0a1120 2024-05-24T11:36:10.336 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 11 on host 'smithi137' 2024-05-24T11:36:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe75c0a1120 msgr2=0x7fe75c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:10.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe75c0a1120 0x7fe75c0a35e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe768005950 tx=0x7fe76800b410 comp rx=0 tx=0).stop 2024-05-24T11:36:10.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 msgr2=0x7fe77419e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:10.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 0x7fe77419e020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00bd60 tx=0x7fe76c00bd90 comp rx=0 tx=0).stop 2024-05-24T11:36:10.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 shutdown_connections 2024-05-24T11:36:10.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe75c0a1120 0x7fe75c0a35e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:10.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe774101a20 0x7fe77419e020 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:10.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7740fff00 0x7fe7740694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:10.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 --2- 172.21.15.137:0/2681647266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe774101060 0x7fe77419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:10.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 >> 172.21.15.137:0/2681647266 conn(0x7fe7740fb4a0 msgr2=0x7fe774104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:10.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 shutdown_connections 2024-05-24T11:36:10.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:10.334+0000 7fe77a572700 1 -- 172.21.15.137:0/2681647266 wait complete. 2024-05-24T11:36:10.699 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.699 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.699 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:10.699 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:10.699 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.699 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.699 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.699 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:10 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:10.701 DEBUG:teuthology.orchestra.run.smithi137:osd.11> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.11.service 2024-05-24T11:36:10.703 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-24T11:36:10.703 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-24T11:36:10.735 INFO:journalctl@ceph.osd.11.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:36:10.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:10.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:10.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:10 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:10.946 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:36:10.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:10.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:10 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:11.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:11 smithi039 ceph-mon[42919]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-24T11:36:11.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.977+0000 7feedb254700 1 -- 172.21.15.39:0/1788358682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4101ad0 msgr2=0x7feed4103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:11.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.977+0000 7feedb254700 1 --2- 172.21.15.39:0/1788358682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4101ad0 0x7feed4103ec0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feec8009990 tx=0x7feec8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:11.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.977+0000 7feedb254700 1 -- 172.21.15.39:0/1788358682 shutdown_connections 2024-05-24T11:36:11.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.977+0000 7feedb254700 1 --2- 172.21.15.39:0/1788358682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed4106d30 0x7feed4109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:11.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.977+0000 7feedb254700 1 --2- 172.21.15.39:0/1788358682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4101ad0 0x7feed4103ec0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:11.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.977+0000 7feedb254700 1 --2- 172.21.15.39:0/1788358682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4104400 0x7feed41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:11.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.977+0000 7feedb254700 1 -- 172.21.15.39:0/1788358682 >> 172.21.15.39:0/1788358682 conn(0x7feed40fb4a0 msgr2=0x7feed40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:11.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.977+0000 7feedb254700 1 -- 172.21.15.39:0/1788358682 shutdown_connections 2024-05-24T11:36:11.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.977+0000 7feedb254700 1 -- 172.21.15.39:0/1788358682 wait complete. 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feedb254700 1 Processor -- start 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feedb254700 1 -- start start 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feedb254700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4101ad0 0x7feed41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feedb254700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed4104400 0x7feed4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feedb254700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4106d30 0x7feed41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feedb254700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed41a0c00 con 0x7feed4106d30 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feedb254700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed41a0d40 con 0x7feed4101ad0 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feedb254700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed41a0ed0 con 0x7feed4104400 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed97f1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4106d30 0x7feed41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed3fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed4104400 0x7feed4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed97f1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4106d30 0x7feed41a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34868/0 (socket says 172.21.15.39:34868) 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed97f1700 1 -- 172.21.15.39:0/3439590911 learned_addr learned my addr 172.21.15.39:0/3439590911 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:11.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed8ff0700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4101ad0 0x7feed41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed97f1700 1 -- 172.21.15.39:0/3439590911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed4104400 msgr2=0x7feed4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed97f1700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed4104400 0x7feed4199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed97f1700 1 -- 172.21.15.39:0/3439590911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4101ad0 msgr2=0x7feed41990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed97f1700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4101ad0 0x7feed41990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed97f1700 1 -- 172.21.15.39:0/3439590911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec80095f0 con 0x7feed4106d30 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.978+0000 7feed97f1700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4106d30 0x7feed41a0630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feecc007ee0 tx=0x7feecc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.979+0000 7feed1ffb700 1 -- 172.21.15.39:0/3439590911 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc004db0 con 0x7feed4106d30 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.979+0000 7feed1ffb700 1 -- 172.21.15.39:0/3439590911 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feecc00f040 con 0x7feed4106d30 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.979+0000 7feed1ffb700 1 -- 172.21.15.39:0/3439590911 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc005800 con 0x7feed4106d30 2024-05-24T11:36:11.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.979+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed41a11b0 con 0x7feed4106d30 2024-05-24T11:36:11.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.979+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed41a1660 con 0x7feed4106d30 2024-05-24T11:36:11.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.980+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed404ec60 con 0x7feed4106d30 2024-05-24T11:36:11.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.981+0000 7feed1ffb700 1 -- 172.21.15.39:0/3439590911 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7feecc00db20 con 0x7feed4106d30 2024-05-24T11:36:11.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.981+0000 7feed1ffb700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7feec40a11a0 0x7feec40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:11.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.982+0000 7feed8ff0700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7feec40a11a0 0x7feec40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:11.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.982+0000 7feed1ffb700 1 -- 172.21.15.39:0/3439590911 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6809+0+0 (secure 0 0 0) 0x7feecc0cbc50 con 0x7feed4106d30 2024-05-24T11:36:11.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.982+0000 7feed8ff0700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7feec40a11a0 0x7feec40a3660 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feec8003780 tx=0x7feec8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:11.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:11 smithi053 ceph-mon[47448]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-24T11:36:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:11.986+0000 7feed1ffb700 1 -- 172.21.15.39:0/3439590911 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feecc0d1020 con 0x7feed4106d30 2024-05-24T11:36:11.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:11 smithi137 ceph-mon[47907]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-24T11:36:12.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.189+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7feed410c730 con 0x7feed4106d30 2024-05-24T11:36:12.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.189+0000 7feed1ffb700 1 -- 172.21.15.39:0/3439590911 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) v1 ==== 74+0+133 (secure 0 0 0) 0x7feecc0959c0 con 0x7feed4106d30 2024-05-24T11:36:12.194 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7feec40a11a0 msgr2=0x7feec40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7feec40a11a0 0x7feec40a3660 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feec8003780 tx=0x7feec8005fb0 comp rx=0 tx=0).stop 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4106d30 msgr2=0x7feed41a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4106d30 0x7feed41a0630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feecc007ee0 tx=0x7feecc00d3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 shutdown_connections 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7feec40a11a0 0x7feec40a3660 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4106d30 0x7feed41a0630 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feed4101ad0 0x7feed41990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 --2- 172.21.15.39:0/3439590911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed4104400 0x7feed4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.191+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 >> 172.21.15.39:0/3439590911 conn(0x7feed40fb4a0 msgr2=0x7feed40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:12.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.192+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 shutdown_connections 2024-05-24T11:36:12.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:12.192+0000 7feedb254700 1 -- 172.21.15.39:0/3439590911 wait complete. 2024-05-24T11:36:12.594 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":69,"num_osds":12,"num_up_osds":11,"osd_up_since":1716550557,"num_in_osds":12,"osd_in_since":1716550558,"num_remapped_pgs":0} 2024-05-24T11:36:12.656 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:36:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:36:12.362+0000 7fa5a9dfe7c0 -1 osd.11 0 log_to_monitors true 2024-05-24T11:36:12.656 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:12 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3439590911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-24T11:36:12.656 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:12 smithi137 ceph-mon[47907]: from='osd.11 [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-24T11:36:12.657 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:12 smithi137 ceph-mon[47907]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-24T11:36:12.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:12 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3439590911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-24T11:36:12.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:12 smithi039 ceph-mon[42919]: from='osd.11 [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-24T11:36:12.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:12 smithi039 ceph-mon[42919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-24T11:36:12.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:12 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3439590911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-24T11:36:12.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:12 smithi053 ceph-mon[47448]: from='osd.11 [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-24T11:36:12.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:12 smithi053 ceph-mon[47448]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-24T11:36:13.595 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-24T11:36:13.833 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: osdmap e70: 12 total, 11 up, 12 in 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='osd.11 [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: Detected new or changed devices on smithi137 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi137 to 4278M 2024-05-24T11:36:13.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:13.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:13.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:13 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: osdmap e70: 12 total, 11 up, 12 in 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='osd.11 [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: Detected new or changed devices on smithi137 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi137 to 4278M 2024-05-24T11:36:13.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:13.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:13.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:13 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.994 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:36:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:36:13.590+0000 7fa59ec61700 -1 osd.11 0 waiting for initial osdmap 2024-05-24T11:36:13.995 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:36:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:36:13.611+0000 7fa59b257700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: osdmap e70: 12 total, 11 up, 12 in 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='osd.11 [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: Detected new or changed devices on smithi137 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:36:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi137 to 4278M 2024-05-24T11:36:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:13 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:14.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.817+0000 7fd12fe47700 1 -- 172.21.15.39:0/2887684213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 msgr2=0x7fd1281014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:14.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.817+0000 7fd12fe47700 1 --2- 172.21.15.39:0/2887684213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 0x7fd1281014e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd124009990 tx=0x7fd124009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:14.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.817+0000 7fd12fe47700 1 -- 172.21.15.39:0/2887684213 shutdown_connections 2024-05-24T11:36:14.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.817+0000 7fd12fe47700 1 --2- 172.21.15.39:0/2887684213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd128101a20 0x7fd128108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:14.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.817+0000 7fd12fe47700 1 --2- 172.21.15.39:0/2887684213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1280fff00 0x7fd128100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:14.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.817+0000 7fd12fe47700 1 --2- 172.21.15.39:0/2887684213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 0x7fd1281014e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:14.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.817+0000 7fd12fe47700 1 -- 172.21.15.39:0/2887684213 >> 172.21.15.39:0/2887684213 conn(0x7fd1280fb4c0 msgr2=0x7fd1280fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:14.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.817+0000 7fd12fe47700 1 -- 172.21.15.39:0/2887684213 shutdown_connections 2024-05-24T11:36:14.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.817+0000 7fd12fe47700 1 -- 172.21.15.39:0/2887684213 wait complete. 2024-05-24T11:36:14.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12fe47700 1 Processor -- start 2024-05-24T11:36:14.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12fe47700 1 -- start start 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12fe47700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1280fff00 0x7fd128069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12fe47700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 0x7fd12819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12fe47700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd128101a20 0x7fd12819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12fe47700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12819e840 con 0x7fd128101060 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12fe47700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12819e9b0 con 0x7fd1280fff00 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12fe47700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12819eb70 con 0x7fd128101a20 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12d3e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 0x7fd12819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12d3e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 0x7fd12819bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34882/0 (socket says 172.21.15.39:34882) 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12d3e2700 1 -- 172.21.15.39:0/4068536613 learned_addr learned my addr 172.21.15.39:0/4068536613 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12dbe3700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1280fff00 0x7fd128069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.818+0000 7fd12e3e4700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd128101a20 0x7fd12819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:14.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd12d3e2700 1 -- 172.21.15.39:0/4068536613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd128101a20 msgr2=0x7fd12819e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd12d3e2700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd128101a20 0x7fd12819e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd12d3e2700 1 -- 172.21.15.39:0/4068536613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1280fff00 msgr2=0x7fd128069d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd12d3e2700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1280fff00 0x7fd128069d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd12d3e2700 1 -- 172.21.15.39:0/4068536613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1240095f0 con 0x7fd128101060 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd12dbe3700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1280fff00 0x7fd128069d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd12d3e2700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 0x7fd12819bab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd124000c00 tx=0x7fd124003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd11effd700 1 -- 172.21.15.39:0/4068536613 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12401d070 con 0x7fd128101060 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd11effd700 1 -- 172.21.15.39:0/4068536613 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd12400bd10 con 0x7fd128101060 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12819edf0 con 0x7fd128101060 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd11effd700 1 -- 172.21.15.39:0/4068536613 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124022e60 con 0x7fd128101060 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.819+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12819f200 con 0x7fd128101060 2024-05-24T11:36:14.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.820+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1281072b0 con 0x7fd128101060 2024-05-24T11:36:14.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.821+0000 7fd11effd700 1 -- 172.21.15.39:0/4068536613 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd124022470 con 0x7fd128101060 2024-05-24T11:36:14.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.822+0000 7fd11effd700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd1140a11a0 0x7fd1140a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:14.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.822+0000 7fd12dbe3700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd1140a11a0 0x7fd1140a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:14.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.822+0000 7fd11effd700 1 -- 172.21.15.39:0/4068536613 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd1240cd9b0 con 0x7fd128101060 2024-05-24T11:36:14.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.822+0000 7fd12dbe3700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd1140a11a0 0x7fd1140a3660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd118005fd0 tx=0x7fd118005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:14.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:14.826+0000 7fd11effd700 1 -- 172.21.15.39:0/4068536613 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd12405dd60 con 0x7fd128101060 2024-05-24T11:36:14.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:14 smithi053 ceph-mon[47448]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:36:14.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:14 smithi053 ceph-mon[47448]: osdmap e71: 12 total, 11 up, 12 in 2024-05-24T11:36:14.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:14 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:14.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:14 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:14.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:14 smithi137 ceph-mon[47907]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:36:14.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:14 smithi137 ceph-mon[47907]: osdmap e71: 12 total, 11 up, 12 in 2024-05-24T11:36:14.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:14 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:14.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:14 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:15.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.026+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd12802cda0 con 0x7fd128101060 2024-05-24T11:36:15.032 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:15.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.027+0000 7fd11effd700 1 -- 172.21.15.39:0/4068536613 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) v1 ==== 74+0+133 (secure 0 0 0) 0x7fd1240976f0 con 0x7fd128101060 2024-05-24T11:36:15.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd1140a11a0 msgr2=0x7fd1140a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:15.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd1140a11a0 0x7fd1140a3660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd118005fd0 tx=0x7fd118005e30 comp rx=0 tx=0).stop 2024-05-24T11:36:15.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 msgr2=0x7fd12819bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:15.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 0x7fd12819bab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd124000c00 tx=0x7fd124003b00 comp rx=0 tx=0).stop 2024-05-24T11:36:15.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 shutdown_connections 2024-05-24T11:36:15.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd1140a11a0 0x7fd1140a3660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:15.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd128101a20 0x7fd12819e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:15.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1280fff00 0x7fd128069d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:15.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 --2- 172.21.15.39:0/4068536613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd128101060 0x7fd12819bab0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:15.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 >> 172.21.15.39:0/4068536613 conn(0x7fd1280fb4c0 msgr2=0x7fd1281043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:15.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 shutdown_connections 2024-05-24T11:36:15.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:15.029+0000 7fd12fe47700 1 -- 172.21.15.39:0/4068536613 wait complete. 2024-05-24T11:36:15.054 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:14 smithi039 ceph-mon[42919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-05-24T11:36:15.054 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:14 smithi039 ceph-mon[42919]: osdmap e71: 12 total, 11 up, 12 in 2024-05-24T11:36:15.054 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:14 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:15.054 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:14 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:15.406 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1716550574,"num_in_osds":12,"osd_in_since":1716550558,"num_remapped_pgs":0} 2024-05-24T11:36:15.407 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-24T11:36:15.636 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:36:15.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:15 smithi039 ceph-mon[42919]: purged_snaps scrub starts 2024-05-24T11:36:15.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:15 smithi039 ceph-mon[42919]: purged_snaps scrub ok 2024-05-24T11:36:15.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:15 smithi039 ceph-mon[42919]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-24T11:36:15.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:15 smithi039 ceph-mon[42919]: osd.11 [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] boot 2024-05-24T11:36:15.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:15 smithi039 ceph-mon[42919]: osdmap e72: 12 total, 12 up, 12 in 2024-05-24T11:36:15.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:15 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:15.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:15 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4068536613' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-24T11:36:15.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:15 smithi053 ceph-mon[47448]: purged_snaps scrub starts 2024-05-24T11:36:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:15 smithi053 ceph-mon[47448]: purged_snaps scrub ok 2024-05-24T11:36:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:15 smithi053 ceph-mon[47448]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-24T11:36:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:15 smithi053 ceph-mon[47448]: osd.11 [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] boot 2024-05-24T11:36:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:15 smithi053 ceph-mon[47448]: osdmap e72: 12 total, 12 up, 12 in 2024-05-24T11:36:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:15 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:15 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4068536613' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-24T11:36:15.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:15 smithi137 ceph-mon[47907]: purged_snaps scrub starts 2024-05-24T11:36:15.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:15 smithi137 ceph-mon[47907]: purged_snaps scrub ok 2024-05-24T11:36:15.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:15 smithi137 ceph-mon[47907]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-24T11:36:15.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:15 smithi137 ceph-mon[47907]: osd.11 [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] boot 2024-05-24T11:36:15.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:15 smithi137 ceph-mon[47907]: osdmap e72: 12 total, 12 up, 12 in 2024-05-24T11:36:15.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:15 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:36:15.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:15 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4068536613' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-24T11:36:16.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.786+0000 7f54a97ae700 1 -- 172.21.15.39:0/2858030643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 msgr2=0x7f54a4105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:16.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.786+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2858030643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 0x7f54a4105be0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5498009990 tx=0x7f5498009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:16.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.786+0000 7f54a97ae700 1 -- 172.21.15.39:0/2858030643 shutdown_connections 2024-05-24T11:36:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.786+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2858030643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41062c0 0x7f54a410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.786+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2858030643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a4069960 0x7f54a4069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.786+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2858030643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 0x7f54a4105be0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.786+0000 7f54a97ae700 1 -- 172.21.15.39:0/2858030643 >> 172.21.15.39:0/2858030643 conn(0x7f54a4076f40 msgr2=0x7f54a4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.786+0000 7f54a97ae700 1 -- 172.21.15.39:0/2858030643 shutdown_connections 2024-05-24T11:36:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.786+0000 7f54a97ae700 1 -- 172.21.15.39:0/2858030643 wait complete. 2024-05-24T11:36:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a97ae700 1 Processor -- start 2024-05-24T11:36:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a97ae700 1 -- start start 2024-05-24T11:36:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a97ae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a4069960 0x7f54a41972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a97ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 0x7f54a4197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a97ae700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a41062c0 0x7f54a419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a97ae700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a419c1c0 con 0x7f54a406a310 2024-05-24T11:36:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a97ae700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a419c330 con 0x7f54a41062c0 2024-05-24T11:36:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a97ae700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a419c4f0 con 0x7f54a4069960 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 0x7f54a4197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 0x7f54a4197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34892/0 (socket says 172.21.15.39:34892) 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a27fc700 1 -- 172.21.15.39:0/2402877644 learned_addr learned my addr 172.21.15.39:0/2402877644 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a37fe700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a41062c0 0x7f54a419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a2ffd700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a4069960 0x7f54a41972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a27fc700 1 -- 172.21.15.39:0/2402877644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a4069960 msgr2=0x7f54a41972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a27fc700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a4069960 0x7f54a41972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a27fc700 1 -- 172.21.15.39:0/2402877644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a41062c0 msgr2=0x7f54a419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a27fc700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a41062c0 0x7f54a419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a27fc700 1 -- 172.21.15.39:0/2402877644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54980095f0 con 0x7f54a406a310 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a2ffd700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a4069960 0x7f54a41972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.787+0000 7f54a37fe700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a41062c0 0x7f54a419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.788+0000 7f54a27fc700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 0x7f54a4197800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54980098e0 tx=0x7f54980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.788+0000 7f548bfff700 1 -- 172.21.15.39:0/2402877644 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549801d070 con 0x7f54a406a310 2024-05-24T11:36:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.788+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a419c770 con 0x7f54a406a310 2024-05-24T11:36:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.788+0000 7f548bfff700 1 -- 172.21.15.39:0/2402877644 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f549800fc00 con 0x7f54a406a310 2024-05-24T11:36:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.788+0000 7f548bfff700 1 -- 172.21.15.39:0/2402877644 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498022ca0 con 0x7f54a406a310 2024-05-24T11:36:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.788+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a41a4030 con 0x7f54a406a310 2024-05-24T11:36:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.789+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a41095e0 con 0x7f54a406a310 2024-05-24T11:36:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.790+0000 7f548bfff700 1 -- 172.21.15.39:0/2402877644 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5498022470 con 0x7f54a406a310 2024-05-24T11:36:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.791+0000 7f548bfff700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f548c0a1120 0x7f548c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.791+0000 7f54a2ffd700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f548c0a1120 0x7f548c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.791+0000 7f548bfff700 1 -- 172.21.15.39:0/2402877644 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f54980cdfc0 con 0x7f54a406a310 2024-05-24T11:36:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.791+0000 7f54a2ffd700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f548c0a1120 0x7f548c0a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5494005950 tx=0x7f54940058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.794+0000 7f548bfff700 1 -- 172.21.15.39:0/2402877644 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5498097c80 con 0x7f54a406a310 2024-05-24T11:36:16.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:16 smithi053 ceph-mon[47448]: osdmap e73: 12 total, 12 up, 12 in 2024-05-24T11:36:16.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:16 smithi137 ceph-mon[47907]: osdmap e73: 12 total, 12 up, 12 in 2024-05-24T11:36:17.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.998+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54a404ec60 con 0x7f54a406a310 2024-05-24T11:36:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:16.999+0000 7f548bfff700 1 -- 172.21.15.39:0/2402877644 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18364 (secure 0 0 0) 0x7f549809b2a0 con 0x7f54a406a310 2024-05-24T11:36:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:17.003 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":73,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:15.592615+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:36:17.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f548c0a1120 msgr2=0x7f548c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:17.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f548c0a1120 0x7f548c0a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5494005950 tx=0x7f54940058e0 comp rx=0 tx=0).stop 2024-05-24T11:36:17.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 msgr2=0x7f54a4197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:17.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 0x7f54a4197800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54980098e0 tx=0x7f54980057f0 comp rx=0 tx=0).stop 2024-05-24T11:36:17.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 shutdown_connections 2024-05-24T11:36:17.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f548c0a1120 0x7f548c0a35e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:17.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a4069960 0x7f54a41972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:17.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a41062c0 0x7f54a419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:17.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 --2- 172.21.15.39:0/2402877644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a406a310 0x7f54a4197800 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:17.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 >> 172.21.15.39:0/2402877644 conn(0x7f54a4076f40 msgr2=0x7f54a4107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:17.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 shutdown_connections 2024-05-24T11:36:17.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:17.001+0000 7f54a97ae700 1 -- 172.21.15.39:0/2402877644 wait complete. 2024-05-24T11:36:17.033 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:16 smithi039 ceph-mon[42919]: osdmap e73: 12 total, 12 up, 12 in 2024-05-24T11:36:17.424 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:36:17.424 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-24T11:36:17.653 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:36:17.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:17 smithi053 ceph-mon[47448]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:17.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:17 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2402877644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:17.994 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:17 smithi137 ceph-mon[47907]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:17.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:17 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2402877644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:18.070 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:17 smithi039 ceph-mon[42919]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:18.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:17 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2402877644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:18.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.779+0000 7f4256786700 1 -- 172.21.15.39:0/1623537863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 msgr2=0x7f4250100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:18.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.779+0000 7f4256786700 1 --2- 172.21.15.39:0/1623537863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 0x7f4250100ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f42440099c0 tx=0x7f4244009cd0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.779+0000 7f4256786700 1 -- 172.21.15.39:0/1623537863 shutdown_connections 2024-05-24T11:36:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.779+0000 7f4256786700 1 --2- 172.21.15.39:0/1623537863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4250104ed0 0x7f42501052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.779+0000 7f4256786700 1 --2- 172.21.15.39:0/1623537863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4250101410 0x7f425010cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.779+0000 7f4256786700 1 --2- 172.21.15.39:0/1623537863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 0x7f4250100ed0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.779+0000 7f4256786700 1 -- 172.21.15.39:0/1623537863 >> 172.21.15.39:0/1623537863 conn(0x7f4250075260 msgr2=0x7f4250075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.779+0000 7f4256786700 1 -- 172.21.15.39:0/1623537863 shutdown_connections 2024-05-24T11:36:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.779+0000 7f4256786700 1 -- 172.21.15.39:0/1623537863 wait complete. 2024-05-24T11:36:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f4256786700 1 Processor -- start 2024-05-24T11:36:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f4256786700 1 -- start start 2024-05-24T11:36:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f4256786700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 0x7f4250101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f4256786700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4250101410 0x7f4250102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f4256786700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4250104ed0 0x7f4250102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f4256786700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42501a5670 con 0x7f4250100a50 2024-05-24T11:36:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f4256786700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42501a57e0 con 0x7f4250104ed0 2024-05-24T11:36:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f4256786700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42501a59a0 con 0x7f4250101410 2024-05-24T11:36:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f424ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 0x7f4250101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f424ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 0x7f4250101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40254/0 (socket says 172.21.15.39:40254) 2024-05-24T11:36:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.780+0000 7f424ffff700 1 -- 172.21.15.39:0/3771608731 learned_addr learned my addr 172.21.15.39:0/3771608731 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424f7fe700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4250101410 0x7f4250102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424ffff700 1 -- 172.21.15.39:0/3771608731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4250101410 msgr2=0x7f4250102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424ffff700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4250101410 0x7f4250102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424ffff700 1 -- 172.21.15.39:0/3771608731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4250104ed0 msgr2=0x7f4250102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f4254d23700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4250104ed0 0x7f4250102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424ffff700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4250104ed0 0x7f4250102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424ffff700 1 -- 172.21.15.39:0/3771608731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440095f0 con 0x7f4250100a50 2024-05-24T11:36:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424f7fe700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4250101410 0x7f4250102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:36:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424ffff700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 0x7f4250101bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f424000c970 tx=0x7f424000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424d7fa700 1 -- 172.21.15.39:0/3771608731 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240007aa0 con 0x7f4250100a50 2024-05-24T11:36:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424d7fa700 1 -- 172.21.15.39:0/3771608731 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4240004cc0 con 0x7f4250100a50 2024-05-24T11:36:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42501a5c80 con 0x7f4250100a50 2024-05-24T11:36:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f424d7fa700 1 -- 172.21.15.39:0/3771608731 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400054a0 con 0x7f4250100a50 2024-05-24T11:36:18.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.781+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42501a6130 con 0x7f4250100a50 2024-05-24T11:36:18.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.782+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42501094d0 con 0x7f4250100a50 2024-05-24T11:36:18.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.784+0000 7f424d7fa700 1 -- 172.21.15.39:0/3771608731 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4240004750 con 0x7f4250100a50 2024-05-24T11:36:18.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.785+0000 7f424d7fa700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42380a1120 0x7f42380a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:18.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.785+0000 7f424f7fe700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42380a1120 0x7f42380a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:18.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.785+0000 7f424d7fa700 1 -- 172.21.15.39:0/3771608731 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4240013070 con 0x7f4250100a50 2024-05-24T11:36:18.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.785+0000 7f424f7fe700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42380a1120 0x7f42380a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4244006010 tx=0x7f4244003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:18.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.788+0000 7f424d7fa700 1 -- 172.21.15.39:0/3771608731 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4240096250 con 0x7f4250100a50 2024-05-24T11:36:18.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.990+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4250103150 con 0x7f4250100a50 2024-05-24T11:36:18.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.990+0000 7f424d7fa700 1 -- 172.21.15.39:0/3771608731 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4240099870 con 0x7f4250100a50 2024-05-24T11:36:18.994 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:36:18.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.992+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42380a1120 msgr2=0x7f42380a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:18.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.992+0000 7f4256786700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42380a1120 0x7f42380a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4244006010 tx=0x7f4244003680 comp rx=0 tx=0).stop 2024-05-24T11:36:18.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.992+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 msgr2=0x7f4250101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:18.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.992+0000 7f4256786700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 0x7f4250101bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f424000c970 tx=0x7f424000cd30 comp rx=0 tx=0).stop 2024-05-24T11:36:18.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.993+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 shutdown_connections 2024-05-24T11:36:18.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.993+0000 7f4256786700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42380a1120 0x7f42380a35e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.993+0000 7f4256786700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4250104ed0 0x7f4250102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.993+0000 7f4256786700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4250100a50 0x7f4250101bd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.993+0000 7f4256786700 1 --2- 172.21.15.39:0/3771608731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4250101410 0x7f4250102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:18.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.993+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 >> 172.21.15.39:0/3771608731 conn(0x7f4250075260 msgr2=0x7f42500ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:18.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.993+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 shutdown_connections 2024-05-24T11:36:18.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:18.993+0000 7f4256786700 1 -- 172.21.15.39:0/3771608731 wait complete. 2024-05-24T11:36:19.463 INFO:tasks.cephadm:Adding mds.a on smithi039 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.d on smithi039 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.g on smithi039 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.j on smithi039 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.b on smithi053 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.e on smithi053 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.h on smithi053 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.k on smithi053 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.c on smithi137 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.f on smithi137 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.i on smithi137 2024-05-24T11:36:19.464 INFO:tasks.cephadm:Adding mds.l on smithi137 2024-05-24T11:36:19.465 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l' 2024-05-24T11:36:19.701 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:36:19.801 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:19 smithi137 ceph-mon[47907]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:19.802 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:19 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3771608731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:36:19.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:19 smithi039 ceph-mon[42919]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:19.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:19 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3771608731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:36:19.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:19 smithi053 ceph-mon[47448]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:19 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3771608731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:36:20.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.593+0000 7fed912e5700 1 -- 172.21.15.137:0/532900932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101a30 msgr2=0x7fed8c108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:20.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.593+0000 7fed912e5700 1 --2- 172.21.15.137:0/532900932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101a30 0x7fed8c108350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed8400b0a0 tx=0x7fed8400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.593+0000 7fed912e5700 1 -- 172.21.15.137:0/532900932 shutdown_connections 2024-05-24T11:36:20.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.593+0000 7fed912e5700 1 --2- 172.21.15.137:0/532900932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101a30 0x7fed8c108350 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.593+0000 7fed912e5700 1 --2- 172.21.15.137:0/532900932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed8c0fff10 0x7fed8c100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.593+0000 7fed912e5700 1 --2- 172.21.15.137:0/532900932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8c101070 0x7fed8c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.593+0000 7fed912e5700 1 -- 172.21.15.137:0/532900932 >> 172.21.15.137:0/532900932 conn(0x7fed8c0fb490 msgr2=0x7fed8c0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:20.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.593+0000 7fed912e5700 1 -- 172.21.15.137:0/532900932 shutdown_connections 2024-05-24T11:36:20.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.593+0000 7fed912e5700 1 -- 172.21.15.137:0/532900932 wait complete. 2024-05-24T11:36:20.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed912e5700 1 Processor -- start 2024-05-24T11:36:20.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed912e5700 1 -- start start 2024-05-24T11:36:20.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed912e5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8c0fff10 0x7fed8c06a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:20.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed912e5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101070 0x7fed8c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:20.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed912e5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed8c101a30 0x7fed8c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:20.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed912e5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c19e720 con 0x7fed8c101a30 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed912e5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c19e890 con 0x7fed8c0fff10 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed912e5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c19ea50 con 0x7fed8c101070 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed8a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101070 0x7fed8c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed8a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101070 0x7fed8c19b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44624/0 (socket says 172.21.15.137:44624) 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.594+0000 7fed8a7fc700 1 -- 172.21.15.137:0/342323551 learned_addr learned my addr 172.21.15.137:0/342323551 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed8b7fe700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed8c101a30 0x7fed8c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed8affd700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8c0fff10 0x7fed8c06a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed8a7fc700 1 -- 172.21.15.137:0/342323551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8c0fff10 msgr2=0x7fed8c06a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed8a7fc700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8c0fff10 0x7fed8c06a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed8a7fc700 1 -- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed8c101a30 msgr2=0x7fed8c19df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed8a7fc700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed8c101a30 0x7fed8c19df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed8a7fc700 1 -- 172.21.15.137:0/342323551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed84009d00 con 0x7fed8c101070 2024-05-24T11:36:20.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed8b7fe700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed8c101a30 0x7fed8c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:36:20.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed8a7fc700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101070 0x7fed8c19b940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00c920 tx=0x7fed7c00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:20.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed83fff700 1 -- 172.21.15.137:0/342323551 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c007b00 con 0x7fed8c101070 2024-05-24T11:36:20.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed83fff700 1 -- 172.21.15.137:0/342323551 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed7c004cc0 con 0x7fed8c101070 2024-05-24T11:36:20.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c19ed30 con 0x7fed8c101070 2024-05-24T11:36:20.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed83fff700 1 -- 172.21.15.137:0/342323551 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c005590 con 0x7fed8c101070 2024-05-24T11:36:20.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.595+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c19f1d0 con 0x7fed8c101070 2024-05-24T11:36:20.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.596+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c1072c0 con 0x7fed8c101070 2024-05-24T11:36:20.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.597+0000 7fed83fff700 1 -- 172.21.15.137:0/342323551 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fed7c004750 con 0x7fed8c101070 2024-05-24T11:36:20.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.598+0000 7fed83fff700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fed780a1120 0x7fed780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:20.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.598+0000 7fed83fff700 1 -- 172.21.15.137:0/342323551 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fed7c0cc400 con 0x7fed8c101070 2024-05-24T11:36:20.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.598+0000 7fed8affd700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fed780a1120 0x7fed780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:20.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.599+0000 7fed8affd700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fed780a1120 0x7fed780a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fed7400ba60 tx=0x7fed74005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:20.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.602+0000 7fed83fff700 1 -- 172.21.15.137:0/342323551 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fed7c096030 con 0x7fed8c101070 2024-05-24T11:36:20.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.805+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fed8c02cf40 con 0x7fed780a1120 2024-05-24T11:36:20.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.815+0000 7fed83fff700 1 -- 172.21.15.137:0/342323551 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fed8c02cf40 con 0x7fed780a1120 2024-05-24T11:36:20.865 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mds.all update... 2024-05-24T11:36:20.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fed780a1120 msgr2=0x7fed780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:20.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fed780a1120 0x7fed780a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fed7400ba60 tx=0x7fed74005d50 comp rx=0 tx=0).stop 2024-05-24T11:36:20.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101070 msgr2=0x7fed8c19b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:20.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101070 0x7fed8c19b940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00c920 tx=0x7fed7c00cce0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 shutdown_connections 2024-05-24T11:36:20.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fed780a1120 0x7fed780a35e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed8c101a30 0x7fed8c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8c0fff10 0x7fed8c06a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 --2- 172.21.15.137:0/342323551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed8c101070 0x7fed8c19b940 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:20.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 >> 172.21.15.137:0/342323551 conn(0x7fed8c0fb490 msgr2=0x7fed8c104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:20.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 shutdown_connections 2024-05-24T11:36:20.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-05-24T11:36:20.818+0000 7fed912e5700 1 -- 172.21.15.137:0/342323551 wait complete. 2024-05-24T11:36:21.190 DEBUG:teuthology.orchestra.run.smithi039:mds.a> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.a.service 2024-05-24T11:36:21.193 DEBUG:teuthology.orchestra.run.smithi039:mds.d> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.d.service 2024-05-24T11:36:21.224 INFO:journalctl@ceph.mds.a.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:36:21.226 DEBUG:teuthology.orchestra.run.smithi039:mds.g> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.g.service 2024-05-24T11:36:21.253 INFO:journalctl@ceph.mds.d.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:36:21.254 DEBUG:teuthology.orchestra.run.smithi039:mds.j> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.j.service 2024-05-24T11:36:21.281 INFO:journalctl@ceph.mds.g.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:36:21.283 DEBUG:teuthology.orchestra.run.smithi053:mds.b> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.b.service 2024-05-24T11:36:21.286 DEBUG:teuthology.orchestra.run.smithi053:mds.e> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.e.service 2024-05-24T11:36:21.309 INFO:journalctl@ceph.mds.j.smithi039.stdout:-- Logs begin at Fri 2024-05-24 11:14:18 UTC. -- 2024-05-24T11:36:21.327 INFO:journalctl@ceph.mds.b.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:36:21.328 DEBUG:teuthology.orchestra.run.smithi053:mds.h> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.h.service 2024-05-24T11:36:21.361 INFO:journalctl@ceph.mds.e.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:36:21.362 DEBUG:teuthology.orchestra.run.smithi053:mds.k> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.k.service 2024-05-24T11:36:21.393 INFO:journalctl@ceph.mds.h.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:36:21.395 DEBUG:teuthology.orchestra.run.smithi137:mds.c> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.c.service 2024-05-24T11:36:21.397 DEBUG:teuthology.orchestra.run.smithi137:mds.f> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.f.service 2024-05-24T11:36:21.426 INFO:journalctl@ceph.mds.k.smithi053.stdout:-- Logs begin at Fri 2024-05-24 11:12:53 UTC. -- 2024-05-24T11:36:21.426 INFO:journalctl@ceph.mds.c.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:36:21.427 DEBUG:teuthology.orchestra.run.smithi137:mds.i> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.i.service 2024-05-24T11:36:21.453 INFO:journalctl@ceph.mds.f.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:36:21.454 DEBUG:teuthology.orchestra.run.smithi137:mds.l> sudo journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.l.service 2024-05-24T11:36:21.482 INFO:journalctl@ceph.mds.i.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:36:21.483 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-24T11:36:21.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:36:21.510 INFO:journalctl@ceph.mds.l.smithi137.stdout:-- Logs begin at Fri 2024-05-24 11:13:10 UTC. -- 2024-05-24T11:36:21.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 -- 172.21.15.39:0/3346012867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 msgr2=0x7f7f081092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:21.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 --2- 172.21.15.39:0/3346012867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 0x7f7f081092a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000b0a0 tx=0x7f7f0000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 -- 172.21.15.39:0/3346012867 shutdown_connections 2024-05-24T11:36:21.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 --2- 172.21.15.39:0/3346012867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 0x7f7f081092a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 --2- 172.21.15.39:0/3346012867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f08100e70 0x7f7f08101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 --2- 172.21.15.39:0/3346012867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f08101fc0 0x7f7f08102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 -- 172.21.15.39:0/3346012867 >> 172.21.15.39:0/3346012867 conn(0x7f7f080fc400 msgr2=0x7f7f080fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 -- 172.21.15.39:0/3346012867 shutdown_connections 2024-05-24T11:36:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 -- 172.21.15.39:0/3346012867 wait complete. 2024-05-24T11:36:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 Processor -- start 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.629+0000 7f7f107b5700 1 -- start start 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f107b5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f08100e70 0x7f7f08196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f107b5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f08101fc0 0x7f7f08197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f107b5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 0x7f7f0819e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f107b5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819e930 con 0x7f7f08102980 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f107b5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819ea70 con 0x7f7f08100e70 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f107b5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819ec00 con 0x7f7f08101fc0 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0ed52700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 0x7f7f0819e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0dd50700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f08101fc0 0x7f7f08197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0ed52700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 0x7f7f0819e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40278/0 (socket says 172.21.15.39:40278) 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0dd50700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f08101fc0 0x7f7f08197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:56256/0 (socket says 172.21.15.39:56256) 2024-05-24T11:36:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0ed52700 1 -- 172.21.15.39:0/1846379291 learned_addr learned my addr 172.21.15.39:0/1846379291 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0e551700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f08100e70 0x7f7f08196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0ed52700 1 -- 172.21.15.39:0/1846379291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f08101fc0 msgr2=0x7f7f08197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0ed52700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f08101fc0 0x7f7f08197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0ed52700 1 -- 172.21.15.39:0/1846379291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f08100e70 msgr2=0x7f7f08196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0ed52700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f08100e70 0x7f7f08196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0ed52700 1 -- 172.21.15.39:0/1846379291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f00009d00 con 0x7f7f08102980 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0e551700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f08100e70 0x7f7f08196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f0ed52700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 0x7f7f0819e360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009840 tx=0x7f7f0000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7efb7fe700 1 -- 172.21.15.39:0/1846379291 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000e040 con 0x7f7f08102980 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7efb7fe700 1 -- 172.21.15.39:0/1846379291 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f00004c50 con 0x7f7f08102980 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0819ee80 con 0x7f7f08102980 2024-05-24T11:36:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.630+0000 7f7efb7fe700 1 -- 172.21.15.39:0/1846379291 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f000052a0 con 0x7f7f08102980 2024-05-24T11:36:21.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.631+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0819f2b0 con 0x7f7f08102980 2024-05-24T11:36:21.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.631+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f081082b0 con 0x7f7f08102980 2024-05-24T11:36:21.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.633+0000 7f7efb7fe700 1 -- 172.21.15.39:0/1846379291 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7f0001d470 con 0x7f7f08102980 2024-05-24T11:36:21.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.633+0000 7f7efb7fe700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7ef40a11a0 0x7f7ef40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:21.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.633+0000 7f7f0e551700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7ef40a11a0 0x7f7ef40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:21.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.633+0000 7f7efb7fe700 1 -- 172.21.15.39:0/1846379291 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7f000ce310 con 0x7f7f08102980 2024-05-24T11:36:21.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.634+0000 7f7f0e551700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7ef40a11a0 0x7f7ef40a3660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005fd0 tx=0x7f7f04005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:21.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.637+0000 7f7efb7fe700 1 -- 172.21.15.39:0/1846379291 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7f000980f0 con 0x7f7f08102980 2024-05-24T11:36:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.849+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f0810c520 con 0x7f7f08102980 2024-05-24T11:36:21.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.850+0000 7f7efb7fe700 1 -- 172.21.15.39:0/1846379291 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18364 (secure 0 0 0) 0x7f7f0009b710 con 0x7f7f08102980 2024-05-24T11:36:21.855 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:21.856 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":73,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:15.592615+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:36:21.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7ef40a11a0 msgr2=0x7f7ef40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:21.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7ef40a11a0 0x7f7ef40a3660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005fd0 tx=0x7f7f04005dc0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 msgr2=0x7f7f0819e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:21.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 0x7f7f0819e360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009840 tx=0x7f7f0000ba50 comp rx=0 tx=0).stop 2024-05-24T11:36:21.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 shutdown_connections 2024-05-24T11:36:21.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7ef40a11a0 0x7f7ef40a3660 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08102980 0x7f7f0819e360 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f08100e70 0x7f7f08196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 --2- 172.21.15.39:0/1846379291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f08101fc0 0x7f7f08197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:21.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 >> 172.21.15.39:0/1846379291 conn(0x7f7f080fc400 msgr2=0x7f7f081053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:21.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 shutdown_connections 2024-05-24T11:36:21.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.852+0000 7f7f107b5700 1 -- 172.21.15.39:0/1846379291 wait complete. 2024-05-24T11:36:21.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:36:21.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-24T11:36:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.997+0000 7f7be08ea700 1 -- 172.21.15.39:0/659807690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc100fb0 msgr2=0x7f7bdc101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.997+0000 7f7be08ea700 1 --2- 172.21.15.39:0/659807690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc100fb0 0x7f7bdc101430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009990 tx=0x7f7bcc009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.997+0000 7f7be08ea700 1 -- 172.21.15.39:0/659807690 shutdown_connections 2024-05-24T11:36:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.997+0000 7f7be08ea700 1 --2- 172.21.15.39:0/659807690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bdc0ffe60 0x7f7bdc100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.997+0000 7f7be08ea700 1 --2- 172.21.15.39:0/659807690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bdc101970 0x7f7bdc108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.997+0000 7f7be08ea700 1 --2- 172.21.15.39:0/659807690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc100fb0 0x7f7bdc101430 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.997+0000 7f7be08ea700 1 -- 172.21.15.39:0/659807690 >> 172.21.15.39:0/659807690 conn(0x7f7bdc0fb3f0 msgr2=0x7f7bdc0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.997+0000 7f7be08ea700 1 -- 172.21.15.39:0/659807690 shutdown_connections 2024-05-24T11:36:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.997+0000 7f7be08ea700 1 -- 172.21.15.39:0/659807690 wait complete. 2024-05-24T11:36:22.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7be08ea700 1 Processor -- start 2024-05-24T11:36:22.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7be08ea700 1 -- start start 2024-05-24T11:36:22.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7be08ea700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bdc0ffe60 0x7f7bdc068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7be08ea700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bdc100fb0 0x7f7bdc19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7be08ea700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc101970 0x7f7bdc19e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7be08ea700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc19e910 con 0x7f7bdc101970 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7be08ea700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc19ea80 con 0x7f7bdc0ffe60 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7be08ea700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc19ec40 con 0x7f7bdc100fb0 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bdad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc101970 0x7f7bdc19e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bdad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc101970 0x7f7bdc19e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40304/0 (socket says 172.21.15.39:40304) 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bdad9d700 1 -- 172.21.15.39:0/834861554 learned_addr learned my addr 172.21.15.39:0/834861554 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bda59c700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bdc0ffe60 0x7f7bdc068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bd9d9b700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bdc100fb0 0x7f7bdc19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bdad9d700 1 -- 172.21.15.39:0/834861554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bdc100fb0 msgr2=0x7f7bdc19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bdad9d700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bdc100fb0 0x7f7bdc19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bdad9d700 1 -- 172.21.15.39:0/834861554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bdc0ffe60 msgr2=0x7f7bdc068b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bdad9d700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bdc0ffe60 0x7f7bdc068b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.998+0000 7f7bdad9d700 1 -- 172.21.15.39:0/834861554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0095f0 con 0x7f7bdc101970 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.999+0000 7f7bdad9d700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc101970 0x7f7bdc19e170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400ada0 tx=0x7f7bd400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.999+0000 7f7bcb7fe700 1 -- 172.21.15.39:0/834861554 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400f070 con 0x7f7bdc101970 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.999+0000 7f7bcb7fe700 1 -- 172.21.15.39:0/834861554 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bd4007b30 con 0x7f7bdc101970 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.999+0000 7f7bcb7fe700 1 -- 172.21.15.39:0/834861554 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4005280 con 0x7f7bdc101970 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.999+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc19ef20 con 0x7f7bdc101970 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:21.999+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc19f350 con 0x7f7bdc101970 2024-05-24T11:36:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.000+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc107340 con 0x7f7bdc101970 2024-05-24T11:36:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.001+0000 7f7bcb7fe700 1 -- 172.21.15.39:0/834861554 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7bd400a090 con 0x7f7bdc101970 2024-05-24T11:36:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.001+0000 7f7bcb7fe700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7bc40a1120 0x7f7bc40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.002+0000 7f7bda59c700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7bc40a1120 0x7f7bc40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.002+0000 7f7bcb7fe700 1 -- 172.21.15.39:0/834861554 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7bd40cbfd0 con 0x7f7bdc101970 2024-05-24T11:36:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.002+0000 7f7bda59c700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7bc40a1120 0x7f7bc40a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005950 tx=0x7f7bd000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:22.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.005+0000 7f7bcb7fe700 1 -- 172.21.15.39:0/834861554 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7bd4095bf0 con 0x7f7bdc101970 2024-05-24T11:36:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l;count:12 2024-05-24T11:36:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:21 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:22.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.210+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bdc10b560 con 0x7f7bdc101970 2024-05-24T11:36:22.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.210+0000 7f7bcb7fe700 1 -- 172.21.15.39:0/834861554 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bd4099210 con 0x7f7bdc101970 2024-05-24T11:36:22.215 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7bc40a1120 msgr2=0x7f7bc40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7bc40a1120 0x7f7bc40a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005950 tx=0x7f7bd000b410 comp rx=0 tx=0).stop 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc101970 msgr2=0x7f7bdc19e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc101970 0x7f7bdc19e170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400ada0 tx=0x7f7bd400c330 comp rx=0 tx=0).stop 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 shutdown_connections 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7bc40a1120 0x7f7bc40a35e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bdc101970 0x7f7bdc19e170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bdc0ffe60 0x7f7bdc068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 --2- 172.21.15.39:0/834861554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bdc100fb0 0x7f7bdc19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 >> 172.21.15.39:0/834861554 conn(0x7f7bdc0fb3f0 msgr2=0x7f7bdc104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 shutdown_connections 2024-05-24T11:36:22.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.212+0000 7f7be08ea700 1 -- 172.21.15.39:0/834861554 wait complete. 2024-05-24T11:36:22.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:22.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l;count:12 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:21 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:22.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l;count:12 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:21 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.350+0000 7f4fd07a4700 1 -- 172.21.15.39:0/3175185155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8101fc0 msgr2=0x7f4fc8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.350+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/3175185155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8101fc0 0x7f4fc8102440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009990 tx=0x7f4fbc009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.350+0000 7f4fd07a4700 1 -- 172.21.15.39:0/3175185155 shutdown_connections 2024-05-24T11:36:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.350+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/3175185155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc8100e70 0x7f4fc8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.350+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/3175185155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fc8102980 0x7f4fc81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.350+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/3175185155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8101fc0 0x7f4fc8102440 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.350+0000 7f4fd07a4700 1 -- 172.21.15.39:0/3175185155 >> 172.21.15.39:0/3175185155 conn(0x7f4fc80fc400 msgr2=0x7f4fc80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.350+0000 7f4fd07a4700 1 -- 172.21.15.39:0/3175185155 shutdown_connections 2024-05-24T11:36:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.350+0000 7f4fd07a4700 1 -- 172.21.15.39:0/3175185155 wait complete. 2024-05-24T11:36:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fd07a4700 1 Processor -- start 2024-05-24T11:36:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fd07a4700 1 -- start start 2024-05-24T11:36:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fd07a4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8100e70 0x7f4fc8196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fd07a4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc8101fc0 0x7f4fc81972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fd07a4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fc8102980 0x7f4fc819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fd07a4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc819e880 con 0x7f4fc8100e70 2024-05-24T11:36:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fd07a4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc819e9f0 con 0x7f4fc8102980 2024-05-24T11:36:22.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fd07a4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc819ebb0 con 0x7f4fc8101fc0 2024-05-24T11:36:22.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fce540700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8100e70 0x7f4fc8196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:22.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fce540700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8100e70 0x7f4fc8196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40330/0 (socket says 172.21.15.39:40330) 2024-05-24T11:36:22.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.351+0000 7f4fce540700 1 -- 172.21.15.39:0/1634241452 learned_addr learned my addr 172.21.15.39:0/1634241452 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:22.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fcdd3f700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc8101fc0 0x7f4fc81972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fced41700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fc8102980 0x7f4fc819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fce540700 1 -- 172.21.15.39:0/1634241452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc8101fc0 msgr2=0x7f4fc81972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fce540700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc8101fc0 0x7f4fc81972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fce540700 1 -- 172.21.15.39:0/1634241452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fc8102980 msgr2=0x7f4fc819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fce540700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fc8102980 0x7f4fc819e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fce540700 1 -- 172.21.15.39:0/1634241452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0095f0 con 0x7f4fc8100e70 2024-05-24T11:36:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fced41700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fc8102980 0x7f4fc819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:22.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fce540700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8100e70 0x7f4fc8196d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400b530 tx=0x7f4fc400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:22.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fbb7fe700 1 -- 172.21.15.39:0/1634241452 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4011830 con 0x7f4fc8100e70 2024-05-24T11:36:22.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fbb7fe700 1 -- 172.21.15.39:0/1634241452 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fc4011e20 con 0x7f4fc8100e70 2024-05-24T11:36:22.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc819ee90 con 0x7f4fc8100e70 2024-05-24T11:36:22.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fbb7fe700 1 -- 172.21.15.39:0/1634241452 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400f2e0 con 0x7f4fc8100e70 2024-05-24T11:36:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.352+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc819f330 con 0x7f4fc8100e70 2024-05-24T11:36:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.353+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc8108380 con 0x7f4fc8100e70 2024-05-24T11:36:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.354+0000 7f4fbb7fe700 1 -- 172.21.15.39:0/1634241452 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4fc4011990 con 0x7f4fc8100e70 2024-05-24T11:36:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.355+0000 7f4fbb7fe700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4fb40a1120 0x7f4fb40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.355+0000 7f4fcdd3f700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4fb40a1120 0x7f4fb40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.355+0000 7f4fbb7fe700 1 -- 172.21.15.39:0/1634241452 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4fc40cc370 con 0x7f4fc8100e70 2024-05-24T11:36:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.355+0000 7f4fcdd3f700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4fb40a1120 0x7f4fb40a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c010 tx=0x7f4fbc003720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.358+0000 7f4fbb7fe700 1 -- 172.21.15.39:0/1634241452 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4fc4095fa0 con 0x7f4fc8100e70 2024-05-24T11:36:22.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.613+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fc8067380 con 0x7f4fc8100e70 2024-05-24T11:36:22.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.614+0000 7f4fbb7fe700 1 -- 172.21.15.39:0/1634241452 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f4fc40995c0 con 0x7f4fc8100e70 2024-05-24T11:36:22.618 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:22.618 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-24T11:36:22.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4fb40a1120 msgr2=0x7f4fb40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4fb40a1120 0x7f4fb40a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c010 tx=0x7f4fbc003720 comp rx=0 tx=0).stop 2024-05-24T11:36:22.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8100e70 msgr2=0x7f4fc8196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:22.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8100e70 0x7f4fc8196d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400b530 tx=0x7f4fc400b840 comp rx=0 tx=0).stop 2024-05-24T11:36:22.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 shutdown_connections 2024-05-24T11:36:22.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f4fb40a1120 0x7f4fb40a35e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fc8102980 0x7f4fc819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fc8100e70 0x7f4fc8196d90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 --2- 172.21.15.39:0/1634241452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc8101fc0 0x7f4fc81972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:22.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 >> 172.21.15.39:0/1634241452 conn(0x7f4fc80fc400 msgr2=0x7f4fc81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:22.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 shutdown_connections 2024-05-24T11:36:22.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:22.616+0000 7f4fd07a4700 1 -- 172.21.15.39:0/1634241452 wait complete. 2024-05-24T11:36:22.621 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 1 2024-05-24T11:36:23.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:22 smithi039 ceph-mon[42919]: Deploying daemon mds.l on smithi137 2024-05-24T11:36:23.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:22 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1846379291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:23.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:22 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/834861554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:36:23.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:22 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1634241452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:23.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:22 smithi053 ceph-mon[47448]: Deploying daemon mds.l on smithi137 2024-05-24T11:36:23.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:22 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1846379291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:23.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:22 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/834861554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:36:23.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:22 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1634241452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:23.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:22 smithi137 ceph-mon[47907]: Deploying daemon mds.l on smithi137 2024-05-24T11:36:23.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:22 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1846379291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:23.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:22 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/834861554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:36:23.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:22 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1634241452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:23.245 INFO:journalctl@ceph.mds.l.smithi137.stdout:May 24 11:36:22 smithi137 bash[84527]: a3624f5e4de2f30cff1d38a6495cfb4afc0a7fabd7a25fd32f8de9ceeec515e9 2024-05-24T11:36:23.245 INFO:journalctl@ceph.mds.l.smithi137.stdout:May 24 11:36:22 smithi137 systemd[1]: Started Ceph mds.l for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6832/451866213,v1:172.21.15.137:6833/451866213] up:boot 2024-05-24T11:36:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: fsmap 1 up:standby 2024-05-24T11:36:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-24T11:36:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:23.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:23.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:23.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:23.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:23 smithi053 ceph-mon[47448]: Deploying daemon mds.k on smithi053 2024-05-24T11:36:24.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6832/451866213,v1:172.21.15.137:6833/451866213] up:boot 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: fsmap 1 up:standby 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:23 smithi039 ceph-mon[42919]: Deploying daemon mds.k on smithi053 2024-05-24T11:36:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6832/451866213,v1:172.21.15.137:6833/451866213] up:boot 2024-05-24T11:36:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: fsmap 1 up:standby 2024-05-24T11:36:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-24T11:36:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:24.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:24.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:24.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:23 smithi137 ceph-mon[47907]: Deploying daemon mds.k on smithi053 2024-05-24T11:36:24.391 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 24 11:36:24 smithi053 systemd[1]: Starting Ceph mds.k for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:24.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:24.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.769+0000 7f7fec87b700 1 -- 172.21.15.39:0/1146783934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8100f80 msgr2=0x7f7fe81013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:24.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.769+0000 7f7fec87b700 1 --2- 172.21.15.39:0/1146783934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8100f80 0x7f7fe81013e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0099c0 tx=0x7f7fdc009cd0 comp rx=0 tx=0).stop 2024-05-24T11:36:24.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.769+0000 7f7fec87b700 1 -- 172.21.15.39:0/1146783934 shutdown_connections 2024-05-24T11:36:24.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.769+0000 7f7fec87b700 1 --2- 172.21.15.39:0/1146783934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe80ffd90 0x7f7fe81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:24.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.769+0000 7f7fec87b700 1 --2- 172.21.15.39:0/1146783934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe8101920 0x7f7fe81081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:24.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.769+0000 7f7fec87b700 1 --2- 172.21.15.39:0/1146783934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8100f80 0x7f7fe81013e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.769+0000 7f7fec87b700 1 -- 172.21.15.39:0/1146783934 >> 172.21.15.39:0/1146783934 conn(0x7f7fe80fb300 msgr2=0x7f7fe80fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.769+0000 7f7fec87b700 1 -- 172.21.15.39:0/1146783934 shutdown_connections 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.769+0000 7f7fec87b700 1 -- 172.21.15.39:0/1146783934 wait complete. 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fec87b700 1 Processor -- start 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fec87b700 1 -- start start 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fec87b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe80ffd90 0x7f7fe8196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fec87b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe8100f80 0x7f7fe8197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fec87b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8101920 0x7f7fe819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fec87b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe819e8c0 con 0x7f7fe8101920 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fec87b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe819ea00 con 0x7f7fe80ffd90 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fec87b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe819eb90 con 0x7f7fe8100f80 2024-05-24T11:36:24.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fe6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8101920 0x7f7fe819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fe6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8101920 0x7f7fe819e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40336/0 (socket says 172.21.15.39:40336) 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.770+0000 7f7fe6d9d700 1 -- 172.21.15.39:0/916861868 learned_addr learned my addr 172.21.15.39:0/916861868 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fe659c700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe80ffd90 0x7f7fe8196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fe5d9b700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe8100f80 0x7f7fe8197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fe6d9d700 1 -- 172.21.15.39:0/916861868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe8100f80 msgr2=0x7f7fe8197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fe6d9d700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe8100f80 0x7f7fe8197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fe6d9d700 1 -- 172.21.15.39:0/916861868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe80ffd90 msgr2=0x7f7fe8196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fe6d9d700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe80ffd90 0x7f7fe8196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fe6d9d700 1 -- 172.21.15.39:0/916861868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0095f0 con 0x7f7fe8101920 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fe6d9d700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8101920 0x7f7fe819e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0007ee0 tx=0x7f7fe000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fd77fe700 1 -- 172.21.15.39:0/916861868 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0017070 con 0x7f7fe8101920 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fd77fe700 1 -- 172.21.15.39:0/916861868 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fe000f040 con 0x7f7fe8101920 2024-05-24T11:36:24.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe819ee70 con 0x7f7fe8101920 2024-05-24T11:36:24.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fd77fe700 1 -- 172.21.15.39:0/916861868 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0005570 con 0x7f7fe8101920 2024-05-24T11:36:24.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.771+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe819f2f0 con 0x7f7fe8101920 2024-05-24T11:36:24.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.772+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe8067380 con 0x7f7fe8101920 2024-05-24T11:36:24.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.774+0000 7f7fd77fe700 1 -- 172.21.15.39:0/916861868 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7fe000db20 con 0x7f7fe8101920 2024-05-24T11:36:24.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.774+0000 7f7fd77fe700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7fd00a1120 0x7f7fd00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:24.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.775+0000 7f7fe659c700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7fd00a1120 0x7f7fd00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:24.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.775+0000 7f7fd77fe700 1 -- 172.21.15.39:0/916861868 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7fe00cc240 con 0x7f7fe8101920 2024-05-24T11:36:24.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.775+0000 7f7fe659c700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7fd00a1120 0x7f7fd00a35e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8100de0 tx=0x7f7fd800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:24.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:24.778+0000 7f7fd77fe700 1 -- 172.21.15.39:0/916861868 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7fe0095dd0 con 0x7f7fe8101920 2024-05-24T11:36:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.038+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fe810b6f0 con 0x7f7fe8101920 2024-05-24T11:36:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.038+0000 7f7fd77fe700 1 -- 172.21.15.39:0/916861868 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7f7fe00993f0 con 0x7f7fe8101920 2024-05-24T11:36:25.043 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:25.043 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-24T11:36:25.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7fd00a1120 msgr2=0x7f7fd00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7fd00a1120 0x7f7fd00a35e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8100de0 tx=0x7f7fd800b410 comp rx=0 tx=0).stop 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8101920 msgr2=0x7f7fe819e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8101920 0x7f7fe819e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0007ee0 tx=0x7f7fe000d3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 shutdown_connections 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7fd00a1120 0x7f7fd00a35e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fe8101920 0x7f7fe819e2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fe80ffd90 0x7f7fe8196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 --2- 172.21.15.39:0/916861868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe8100f80 0x7f7fe8197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 >> 172.21.15.39:0/916861868 conn(0x7f7fe80fb300 msgr2=0x7f7fe8069240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 shutdown_connections 2024-05-24T11:36:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:25.040+0000 7f7fec87b700 1 -- 172.21.15.39:0/916861868 wait complete. 2024-05-24T11:36:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2024-05-24T11:36:25.056 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:24 smithi039 ceph-mon[42919]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:25.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:24 smithi053 ceph-mon[47448]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:25.239 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 24 11:36:24 smithi053 bash[83689]: 3db246f966229bb36f2cb5e6f7e3e7e8fe23a8173bcf2b630f7a44b9291562f8 2024-05-24T11:36:25.239 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 24 11:36:24 smithi053 systemd[1]: Started Ceph mds.k for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:25.239 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 24 11:36:24 smithi053 ceph-mds[83726]: -- 172.21.15.53:0/2109935301 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5563c2c341a0 con 0x5563c2c0a800 2024-05-24T11:36:25.239 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 24 11:36:24 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-k[83703]: starting mds.k at 2024-05-24T11:36:25.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:24 smithi137 ceph-mon[47907]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: Deploying daemon mds.j on smithi039 2024-05-24T11:36:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/916861868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.53:6832/2983171503,v1:172.21.15.53:6833/2983171503] up:boot 2024-05-24T11:36:26.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: fsmap 2 up:standby 2024-05-24T11:36:26.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:25 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: Deploying daemon mds.j on smithi039 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/916861868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.53:6832/2983171503,v1:172.21.15.53:6833/2983171503] up:boot 2024-05-24T11:36:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: fsmap 2 up:standby 2024-05-24T11:36:26.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:25 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: Deploying daemon mds.j on smithi039 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/916861868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.53:6832/2983171503,v1:172.21.15.53:6833/2983171503] up:boot 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: fsmap 2 up:standby 2024-05-24T11:36:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:25 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-24T11:36:26.424 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 24 11:36:26 smithi039 systemd[1]: Starting Ceph mds.j for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:26.827 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 24 11:36:26 smithi039 ceph-mds[93327]: -- 172.21.15.39:0/1762873672 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5593123861a0 con 0x55931235cc00 2024-05-24T11:36:26.827 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 24 11:36:26 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-j[93304]: starting mds.j at 2024-05-24T11:36:26.828 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 24 11:36:26 smithi039 bash[93289]: 015c100436f6c8e1edffe8f6fe542afaef6ab670815241a1af5481d89ff456e8 2024-05-24T11:36:26.828 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 24 11:36:26 smithi039 systemd[1]: Started Ceph mds.j for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:27.058 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:27.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.190+0000 7fbc2ea00700 1 -- 172.21.15.39:0/948590610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 msgr2=0x7fbc28101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:27.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.190+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/948590610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 0x7fbc28101290 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbc180098f0 tx=0x7fbc18009c00 comp rx=0 tx=0).stop 2024-05-24T11:36:27.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.191+0000 7fbc2ea00700 1 -- 172.21.15.39:0/948590610 shutdown_connections 2024-05-24T11:36:27.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.191+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/948590610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc28102980 0x7fbc281092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:27.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.191+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/948590610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 0x7fbc28101290 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:27.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.191+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/948590610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc28101fc0 0x7fbc28102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:27.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.191+0000 7fbc2ea00700 1 -- 172.21.15.39:0/948590610 >> 172.21.15.39:0/948590610 conn(0x7fbc280fc400 msgr2=0x7fbc280fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:27.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.191+0000 7fbc2ea00700 1 -- 172.21.15.39:0/948590610 shutdown_connections 2024-05-24T11:36:27.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.191+0000 7fbc2ea00700 1 -- 172.21.15.39:0/948590610 wait complete. 2024-05-24T11:36:27.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.191+0000 7fbc2ea00700 1 Processor -- start 2024-05-24T11:36:27.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.191+0000 7fbc2ea00700 1 -- start start 2024-05-24T11:36:27.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc2ea00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 0x7fbc28196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:27.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc2ea00700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc28101fc0 0x7fbc281972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:27.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc2ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc28102980 0x7fbc2819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:27.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc2ea00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2819e8d0 con 0x7fbc28100e70 2024-05-24T11:36:27.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc2ea00700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2819ea10 con 0x7fbc28101fc0 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc2ea00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2819eba0 con 0x7fbc28102980 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc27fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 0x7fbc28196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc27fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 0x7fbc28196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40352/0 (socket says 172.21.15.39:40352) 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc27fff700 1 -- 172.21.15.39:0/4223243878 learned_addr learned my addr 172.21.15.39:0/4223243878 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc277fe700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc28101fc0 0x7fbc281972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc2cf9d700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc28102980 0x7fbc2819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc27fff700 1 -- 172.21.15.39:0/4223243878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc28102980 msgr2=0x7fbc2819e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc27fff700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc28102980 0x7fbc2819e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc27fff700 1 -- 172.21.15.39:0/4223243878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc28101fc0 msgr2=0x7fbc281972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc27fff700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc28101fc0 0x7fbc281972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:27.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc27fff700 1 -- 172.21.15.39:0/4223243878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc18009520 con 0x7fbc28100e70 2024-05-24T11:36:27.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc277fe700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc28101fc0 0x7fbc281972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:27.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.192+0000 7fbc27fff700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 0x7fbc28196da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009810 tx=0x7fbc1800b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:27.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.193+0000 7fbc257fa700 1 -- 172.21.15.39:0/4223243878 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1801c070 con 0x7fbc28100e70 2024-05-24T11:36:27.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.193+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2819ee20 con 0x7fbc28100e70 2024-05-24T11:36:27.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.193+0000 7fbc257fa700 1 -- 172.21.15.39:0/4223243878 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc18005040 con 0x7fbc28100e70 2024-05-24T11:36:27.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.193+0000 7fbc257fa700 1 -- 172.21.15.39:0/4223243878 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc180039f0 con 0x7fbc28100e70 2024-05-24T11:36:27.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.193+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2819f1e0 con 0x7fbc28100e70 2024-05-24T11:36:27.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.194+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc281083d0 con 0x7fbc28100e70 2024-05-24T11:36:27.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.195+0000 7fbc257fa700 1 -- 172.21.15.39:0/4223243878 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbc18020030 con 0x7fbc28100e70 2024-05-24T11:36:27.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.196+0000 7fbc257fa700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbc140a1120 0x7fbc140a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:27.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.196+0000 7fbc277fe700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbc140a1120 0x7fbc140a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:27.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.196+0000 7fbc257fa700 1 -- 172.21.15.39:0/4223243878 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbc180cdc20 con 0x7fbc28100e70 2024-05-24T11:36:27.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.196+0000 7fbc277fe700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbc140a1120 0x7fbc140a35e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc10005fd0 tx=0x7fbc10009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:27.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.200+0000 7fbc257fa700 1 -- 172.21.15.39:0/4223243878 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbc180978e0 con 0x7fbc28100e70 2024-05-24T11:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.460+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc28067380 con 0x7fbc28100e70 2024-05-24T11:36:27.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.461+0000 7fbc257fa700 1 -- 172.21.15.39:0/4223243878 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7fbc1809af00 con 0x7fbc28100e70 2024-05-24T11:36:27.465 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:27.465 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-24T11:36:27.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbc140a1120 msgr2=0x7fbc140a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:27.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbc140a1120 0x7fbc140a35e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc10005fd0 tx=0x7fbc10009310 comp rx=0 tx=0).stop 2024-05-24T11:36:27.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 msgr2=0x7fbc28196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:27.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 0x7fbc28196da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009810 tx=0x7fbc1800b500 comp rx=0 tx=0).stop 2024-05-24T11:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 shutdown_connections 2024-05-24T11:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbc140a1120 0x7fbc140a35e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc28102980 0x7fbc2819e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc28100e70 0x7fbc28196da0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 --2- 172.21.15.39:0/4223243878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc28101fc0 0x7fbc281972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 >> 172.21.15.39:0/4223243878 conn(0x7fbc280fc400 msgr2=0x7fbc28105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 shutdown_connections 2024-05-24T11:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:27.463+0000 7fbc2ea00700 1 -- 172.21.15.39:0/4223243878 wait complete. 2024-05-24T11:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 4 2024-05-24T11:36:27.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: Deploying daemon mds.d on smithi039 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.39:6834/1458882734,v1:172.21.15.39:6835/1458882734] up:boot 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: fsmap 3 up:standby 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-24T11:36:27.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:27 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4223243878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: Deploying daemon mds.d on smithi039 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.39:6834/1458882734,v1:172.21.15.39:6835/1458882734] up:boot 2024-05-24T11:36:27.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: fsmap 3 up:standby 2024-05-24T11:36:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-24T11:36:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:27 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4223243878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:27.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:27.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:27.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:27.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:27.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:27.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:27.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:27.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: Deploying daemon mds.d on smithi039 2024-05-24T11:36:27.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.39:6834/1458882734,v1:172.21.15.39:6835/1458882734] up:boot 2024-05-24T11:36:27.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: fsmap 3 up:standby 2024-05-24T11:36:27.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-24T11:36:27.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:27 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4223243878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:28.264 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 24 11:36:27 smithi039 systemd[1]: Starting Ceph mds.d for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:28.827 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 24 11:36:28 smithi039 ceph-mds[93712]: -- 172.21.15.39:0/535750284 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5592c0c581a0 con 0x5592c0c2e800 2024-05-24T11:36:28.828 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 24 11:36:28 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-d[93689]: starting mds.d at 2024-05-24T11:36:28.828 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 24 11:36:28 smithi039 bash[93674]: dc554467db18e4d22ebcab6da447db9db34fc60f6f07470dd8ee92a266f87749 2024-05-24T11:36:28.828 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 24 11:36:28 smithi039 systemd[1]: Started Ceph mds.d for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:29.481 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: Deploying daemon mds.e on smithi053 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.39:6836/3631576511,v1:172.21.15.39:6837/3631576511] up:boot 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: fsmap 4 up:standby 2024-05-24T11:36:29.496 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:29 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-24T11:36:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.622+0000 7fb174cd3700 1 -- 172.21.15.39:0/2913451586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1700ffd90 msgr2=0x7fb1701001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.622+0000 7fb174cd3700 1 --2- 172.21.15.39:0/2913451586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1700ffd90 0x7fb1701001b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1580099c0 tx=0x7fb158009cd0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.622+0000 7fb174cd3700 1 -- 172.21.15.39:0/2913451586 shutdown_connections 2024-05-24T11:36:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.622+0000 7fb174cd3700 1 --2- 172.21.15.39:0/2913451586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb170101920 0x7fb1701081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.622+0000 7fb174cd3700 1 --2- 172.21.15.39:0/2913451586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1700ffd90 0x7fb1701001b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.622+0000 7fb174cd3700 1 --2- 172.21.15.39:0/2913451586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb170100f80 0x7fb1701013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.622+0000 7fb174cd3700 1 -- 172.21.15.39:0/2913451586 >> 172.21.15.39:0/2913451586 conn(0x7fb1700fb300 msgr2=0x7fb1700fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.622+0000 7fb174cd3700 1 -- 172.21.15.39:0/2913451586 shutdown_connections 2024-05-24T11:36:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.622+0000 7fb174cd3700 1 -- 172.21.15.39:0/2913451586 wait complete. 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.623+0000 7fb174cd3700 1 Processor -- start 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb174cd3700 1 -- start start 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb174cd3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1700ffd90 0x7fb170196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb174cd3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb170100f80 0x7fb1701971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb174cd3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb170101920 0x7fb17019e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb174cd3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17019e7a0 con 0x7fb170100f80 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb174cd3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17019e910 con 0x7fb1700ffd90 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb174cd3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17019ead0 con 0x7fb170101920 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb170100f80 0x7fb1701971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb170100f80 0x7fb1701971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35980/0 (socket says 172.21.15.39:35980) 2024-05-24T11:36:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16dd9b700 1 -- 172.21.15.39:0/1751948072 learned_addr learned my addr 172.21.15.39:0/1751948072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16e59c700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1700ffd90 0x7fb170196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16ed9d700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb170101920 0x7fb17019e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16dd9b700 1 -- 172.21.15.39:0/1751948072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb170101920 msgr2=0x7fb17019e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16dd9b700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb170101920 0x7fb17019e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16dd9b700 1 -- 172.21.15.39:0/1751948072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1700ffd90 msgr2=0x7fb170196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16dd9b700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1700ffd90 0x7fb170196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16dd9b700 1 -- 172.21.15.39:0/1751948072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1580095f0 con 0x7fb170100f80 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16ed9d700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb170101920 0x7fb17019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.624+0000 7fb16dd9b700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb170100f80 0x7fb1701971f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c9b0 tx=0x7fb16000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.625+0000 7fb1677fe700 1 -- 172.21.15.39:0/1751948072 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160007b00 con 0x7fb170100f80 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.625+0000 7fb1677fe700 1 -- 172.21.15.39:0/1751948072 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb160007c60 con 0x7fb170100f80 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.625+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17019edb0 con 0x7fb170100f80 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.625+0000 7fb1677fe700 1 -- 172.21.15.39:0/1751948072 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160018660 con 0x7fb170100f80 2024-05-24T11:36:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.625+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17019f1d0 con 0x7fb170100f80 2024-05-24T11:36:29.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.626+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb170067380 con 0x7fb170100f80 2024-05-24T11:36:29.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.627+0000 7fb1677fe700 1 -- 172.21.15.39:0/1751948072 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb16000f450 con 0x7fb170100f80 2024-05-24T11:36:29.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.627+0000 7fb1677fe700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb15c0a1120 0x7fb15c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:29.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.628+0000 7fb16e59c700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb15c0a1120 0x7fb15c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.628+0000 7fb1677fe700 1 -- 172.21.15.39:0/1751948072 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb1600cc6a0 con 0x7fb170100f80 2024-05-24T11:36:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.628+0000 7fb16e59c700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb15c0a1120 0x7fb15c0a35e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb158006010 tx=0x7fb15800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.631+0000 7fb1677fe700 1 -- 172.21.15.39:0/1751948072 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb160096360 con 0x7fb170100f80 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: Deploying daemon mds.e on smithi053 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.39:6836/3631576511,v1:172.21.15.39:6837/3631576511] up:boot 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: fsmap 4 up:standby 2024-05-24T11:36:29.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:29 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-24T11:36:29.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:29.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:29.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:29.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:29.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:29.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:29.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:29.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: Deploying daemon mds.e on smithi053 2024-05-24T11:36:29.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.39:6836/3631576511,v1:172.21.15.39:6837/3631576511] up:boot 2024-05-24T11:36:29.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: fsmap 4 up:standby 2024-05-24T11:36:29.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:29 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-24T11:36:29.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.895+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb17010b4a0 con 0x7fb170100f80 2024-05-24T11:36:29.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.895+0000 7fb1677fe700 1 -- 172.21.15.39:0/1751948072 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3528 (secure 0 0 0) 0x7fb16000fe60 con 0x7fb170100f80 2024-05-24T11:36:29.900 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:29.901 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-24T11:36:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb15c0a1120 msgr2=0x7fb15c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb15c0a1120 0x7fb15c0a35e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb158006010 tx=0x7fb15800b540 comp rx=0 tx=0).stop 2024-05-24T11:36:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb170100f80 msgr2=0x7fb1701971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb170100f80 0x7fb1701971f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c9b0 tx=0x7fb16000ccc0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 shutdown_connections 2024-05-24T11:36:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb15c0a1120 0x7fb15c0a35e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb170101920 0x7fb17019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1700ffd90 0x7fb170196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 --2- 172.21.15.39:0/1751948072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb170100f80 0x7fb1701971f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.897+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 >> 172.21.15.39:0/1751948072 conn(0x7fb1700fb300 msgr2=0x7fb1700fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.898+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 shutdown_connections 2024-05-24T11:36:29.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:29.898+0000 7fb174cd3700 1 -- 172.21.15.39:0/1751948072 wait complete. 2024-05-24T11:36:29.905 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 5 2024-05-24T11:36:29.988 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 24 11:36:29 smithi053 systemd[1]: Starting Ceph mds.e for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:30.430 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 24 11:36:30 smithi053 bash[84030]: ec08fa38bfa10988b16544409a673242c08099265c2bdc3e0486e35c19f38e13 2024-05-24T11:36:30.430 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 24 11:36:30 smithi053 systemd[1]: Started Ceph mds.e for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:30.430 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 24 11:36:30 smithi053 ceph-mds[84068]: -- 172.21.15.53:0/3748192739 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5639352821a0 con 0x563935259000 2024-05-24T11:36:30.430 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 24 11:36:30 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-e[84045]: starting mds.e at 2024-05-24T11:36:30.690 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:30 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1751948072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:30.690 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:30.690 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:30.690 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:30.690 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:30.690 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:30.690 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:30 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:30 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1751948072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:30.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:30 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:30.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:30 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1751948072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:30.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:30.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:30.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:30.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:30.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:30.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:30 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:31.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:31 smithi053 ceph-mon[47448]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:31.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:31 smithi053 ceph-mon[47448]: Deploying daemon mds.f on smithi137 2024-05-24T11:36:31.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:31 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.53:6834/3629179598,v1:172.21.15.53:6835/3629179598] up:boot 2024-05-24T11:36:31.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:31 smithi053 ceph-mon[47448]: fsmap 5 up:standby 2024-05-24T11:36:31.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:31 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-24T11:36:31.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:31 smithi137 ceph-mon[47907]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:31.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:31 smithi137 ceph-mon[47907]: Deploying daemon mds.f on smithi137 2024-05-24T11:36:31.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:31 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.53:6834/3629179598,v1:172.21.15.53:6835/3629179598] up:boot 2024-05-24T11:36:31.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:31 smithi137 ceph-mon[47907]: fsmap 5 up:standby 2024-05-24T11:36:31.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:31 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-24T11:36:31.745 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:36:31 smithi137 systemd[1]: Starting Ceph mds.f for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:31.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:31 smithi039 ceph-mon[42919]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:31.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:31 smithi039 ceph-mon[42919]: Deploying daemon mds.f on smithi137 2024-05-24T11:36:31.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:31 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.53:6834/3629179598,v1:172.21.15.53:6835/3629179598] up:boot 2024-05-24T11:36:31.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:31 smithi039 ceph-mon[42919]: fsmap 5 up:standby 2024-05-24T11:36:31.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:31 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-24T11:36:31.915 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:32.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.064+0000 7f400d59e700 1 -- 172.21.15.39:0/736676452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40080739a0 msgr2=0x7f4008073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.064+0000 7f400d59e700 1 --2- 172.21.15.39:0/736676452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40080739a0 0x7f4008073e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009990 tx=0x7f3ff8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.064+0000 7f400d59e700 1 -- 172.21.15.39:0/736676452 shutdown_connections 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.064+0000 7f400d59e700 1 --2- 172.21.15.39:0/736676452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4008108d20 0x7f400810b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.064+0000 7f400d59e700 1 --2- 172.21.15.39:0/736676452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008074f80 0x7f40080733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.064+0000 7f400d59e700 1 --2- 172.21.15.39:0/736676452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40080739a0 0x7f4008073e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.064+0000 7f400d59e700 1 -- 172.21.15.39:0/736676452 >> 172.21.15.39:0/736676452 conn(0x7f40080fc3e0 msgr2=0x7f40080fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.064+0000 7f400d59e700 1 -- 172.21.15.39:0/736676452 shutdown_connections 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.064+0000 7f400d59e700 1 -- 172.21.15.39:0/736676452 wait complete. 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400d59e700 1 Processor -- start 2024-05-24T11:36:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400d59e700 1 -- start start 2024-05-24T11:36:32.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400d59e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40080739a0 0x7f4008198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:32.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400d59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008074f80 0x7f40081993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:32.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400d59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4008108d20 0x7f40081a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400d59e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081a09e0 con 0x7f4008108d20 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400d59e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081a0b20 con 0x7f4008074f80 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400d59e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081a0cb0 con 0x7f40080739a0 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400cd9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4008108d20 0x7f40081a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400cd9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4008108d20 0x7f40081a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36010/0 (socket says 172.21.15.39:36010) 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400cd9d700 1 -- 172.21.15.39:0/2815137835 learned_addr learned my addr 172.21.15.39:0/2815137835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f4007fff700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40080739a0 0x7f4008198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f40077fe700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008074f80 0x7f40081993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400cd9d700 1 -- 172.21.15.39:0/2815137835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40080739a0 msgr2=0x7f4008198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400cd9d700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40080739a0 0x7f4008198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400cd9d700 1 -- 172.21.15.39:0/2815137835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008074f80 msgr2=0x7f40081993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400cd9d700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008074f80 0x7f40081993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f400cd9d700 1 -- 172.21.15.39:0/2815137835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff80095f0 con 0x7f4008108d20 2024-05-24T11:36:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.065+0000 7f4007fff700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40080739a0 0x7f4008198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:36:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.066+0000 7f400cd9d700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4008108d20 0x7f40081a0410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f400000ef30 tx=0x7f400000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.066+0000 7f40057fa700 1 -- 172.21.15.39:0/2815137835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000018070 con 0x7f4008108d20 2024-05-24T11:36:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.066+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40081a0f30 con 0x7f4008108d20 2024-05-24T11:36:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.066+0000 7f40057fa700 1 -- 172.21.15.39:0/2815137835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f400000f040 con 0x7f4008108d20 2024-05-24T11:36:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.066+0000 7f40057fa700 1 -- 172.21.15.39:0/2815137835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000cbc0 con 0x7f4008108d20 2024-05-24T11:36:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.066+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40081a13b0 con 0x7f4008108d20 2024-05-24T11:36:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.067+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008107b30 con 0x7f4008108d20 2024-05-24T11:36:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.068+0000 7f40057fa700 1 -- 172.21.15.39:0/2815137835 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f400000cd40 con 0x7f4008108d20 2024-05-24T11:36:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.068+0000 7f40057fa700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3ff40a1120 0x7f3ff40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.069+0000 7f40057fa700 1 -- 172.21.15.39:0/2815137835 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f40000cc4b0 con 0x7f4008108d20 2024-05-24T11:36:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.069+0000 7f4007fff700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3ff40a1120 0x7f3ff40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.069+0000 7f4007fff700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3ff40a1120 0x7f3ff40a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0011490 tx=0x7f3ff0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:32.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.072+0000 7f40057fa700 1 -- 172.21.15.39:0/2815137835 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4000096170 con 0x7f4008108d20 2024-05-24T11:36:32.245 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:36:31 smithi137 bash[84870]: b4da550cea9b75813de06fd109e097bdda79e117c5a9c960427d4dbee822fd22 2024-05-24T11:36:32.245 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:36:31 smithi137 systemd[1]: Started Ceph mds.f for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:32.245 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:36:31 smithi137 ceph-mds[84908]: -- 172.21.15.137:0/3038191989 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d757a7e1a0 con 0x55d757a55000 2024-05-24T11:36:32.245 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:36:31 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: starting mds.f at 2024-05-24T11:36:32.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.330+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4008067310 con 0x7f4008108d20 2024-05-24T11:36:32.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.331+0000 7f40057fa700 1 -- 172.21.15.39:0/2815137835 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4285 (secure 0 0 0) 0x7f4008067310 con 0x7f4008108d20 2024-05-24T11:36:32.335 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:32.335 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-05-24T11:36:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3ff40a1120 msgr2=0x7f3ff40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3ff40a1120 0x7f3ff40a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0011490 tx=0x7f3ff0009260 comp rx=0 tx=0).stop 2024-05-24T11:36:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4008108d20 msgr2=0x7f40081a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4008108d20 0x7f40081a0410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f400000ef30 tx=0x7f400000c3a0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 shutdown_connections 2024-05-24T11:36:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3ff40a1120 0x7f3ff40a35e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40080739a0 0x7f4008198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4008108d20 0x7f40081a0410 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 --2- 172.21.15.39:0/2815137835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4008074f80 0x7f40081993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 >> 172.21.15.39:0/2815137835 conn(0x7f40080fc3e0 msgr2=0x7f4008104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 shutdown_connections 2024-05-24T11:36:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:32.333+0000 7f400d59e700 1 -- 172.21.15.39:0/2815137835 wait complete. 2024-05-24T11:36:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 6 2024-05-24T11:36:32.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:32.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: Deploying daemon mds.c on smithi137 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2815137835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6834/1490077205,v1:172.21.15.137:6835/1490077205] up:boot 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: fsmap 6 up:standby 2024-05-24T11:36:32.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:32 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:36:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: Deploying daemon mds.c on smithi137 2024-05-24T11:36:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2815137835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:33.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6834/1490077205,v1:172.21.15.137:6835/1490077205] up:boot 2024-05-24T11:36:33.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: fsmap 6 up:standby 2024-05-24T11:36:33.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:32 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:36:33.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:33.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:33.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:33.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:33.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:33.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:33.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:33.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: Deploying daemon mds.c on smithi137 2024-05-24T11:36:33.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2815137835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:33.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6834/1490077205,v1:172.21.15.137:6835/1490077205] up:boot 2024-05-24T11:36:33.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: fsmap 6 up:standby 2024-05-24T11:36:33.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:32 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:36:33.402 INFO:journalctl@ceph.mds.c.smithi137.stdout:May 24 11:36:33 smithi137 systemd[1]: Starting Ceph mds.c for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:33.995 INFO:journalctl@ceph.mds.c.smithi137.stdout:May 24 11:36:33 smithi137 bash[85211]: 1c8aca4d78e6afc43c468e492ebd0e6e80772af52893b63681428fc8a835a29f 2024-05-24T11:36:33.995 INFO:journalctl@ceph.mds.c.smithi137.stdout:May 24 11:36:33 smithi137 systemd[1]: Started Ceph mds.c for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:33.995 INFO:journalctl@ceph.mds.c.smithi137.stdout:May 24 11:36:33 smithi137 ceph-mds[85249]: -- 172.21.15.137:0/3584194988 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555e23e3c1a0 con 0x555e23e12c00 2024-05-24T11:36:33.995 INFO:journalctl@ceph.mds.c.smithi137.stdout:May 24 11:36:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-c[85226]: starting mds.c at 2024-05-24T11:36:34.350 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:34.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 -- 172.21.15.39:0/644048536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4102980 msgr2=0x7f08e41092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:34.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 --2- 172.21.15.39:0/644048536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4102980 0x7f08e41092a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b0a0 tx=0x7f08dc00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 -- 172.21.15.39:0/644048536 shutdown_connections 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 --2- 172.21.15.39:0/644048536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e4100e70 0x7f08e4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 --2- 172.21.15.39:0/644048536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4102980 0x7f08e41092a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 --2- 172.21.15.39:0/644048536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08e4101fc0 0x7f08e4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 -- 172.21.15.39:0/644048536 >> 172.21.15.39:0/644048536 conn(0x7f08e40fc400 msgr2=0x7f08e40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 -- 172.21.15.39:0/644048536 shutdown_connections 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 -- 172.21.15.39:0/644048536 wait complete. 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 Processor -- start 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.487+0000 7f08eb48e700 1 -- start start 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08eb48e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4100e70 0x7f08e4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08eb48e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08e4101fc0 0x7f08e4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08eb48e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e4102980 0x7f08e40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08eb48e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4074070 con 0x7f08e4100e70 2024-05-24T11:36:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08eb48e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40741e0 con 0x7f08e4101fc0 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08eb48e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e41a0e70 con 0x7f08e4102980 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e922a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4100e70 0x7f08e4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e922a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4100e70 0x7f08e4074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36016/0 (socket says 172.21.15.39:36016) 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e922a700 1 -- 172.21.15.39:0/1617208866 learned_addr learned my addr 172.21.15.39:0/1617208866 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e9a2b700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e4102980 0x7f08e40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e8a29700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08e4101fc0 0x7f08e4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e922a700 1 -- 172.21.15.39:0/1617208866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e4102980 msgr2=0x7f08e40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e922a700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e4102980 0x7f08e40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e922a700 1 -- 172.21.15.39:0/1617208866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08e4101fc0 msgr2=0x7f08e4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e922a700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08e4101fc0 0x7f08e4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e922a700 1 -- 172.21.15.39:0/1617208866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc009d00 con 0x7f08e4100e70 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08e922a700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4100e70 0x7f08e4074d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c920 tx=0x7f08e000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08da7fc700 1 -- 172.21.15.39:0/1617208866 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0007aa0 con 0x7f08e4100e70 2024-05-24T11:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08da7fc700 1 -- 172.21.15.39:0/1617208866 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08e0004cc0 con 0x7f08e4100e70 2024-05-24T11:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e41a1100 con 0x7f08e4100e70 2024-05-24T11:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08da7fc700 1 -- 172.21.15.39:0/1617208866 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0005520 con 0x7f08e4100e70 2024-05-24T11:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.488+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e41a1660 con 0x7f08e4100e70 2024-05-24T11:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.489+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e41082e0 con 0x7f08e4100e70 2024-05-24T11:36:34.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.491+0000 7f08da7fc700 1 -- 172.21.15.39:0/1617208866 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f08e0004750 con 0x7f08e4100e70 2024-05-24T11:36:34.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.491+0000 7f08da7fc700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08d00a11a0 0x7f08d00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:34.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.491+0000 7f08e8a29700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08d00a11a0 0x7f08d00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:34.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.491+0000 7f08da7fc700 1 -- 172.21.15.39:0/1617208866 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f08e00cc440 con 0x7f08e4100e70 2024-05-24T11:36:34.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.492+0000 7f08e8a29700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08d00a11a0 0x7f08d00a3660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f08d4005950 tx=0x7f08d40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.495+0000 7f08da7fc700 1 -- 172.21.15.39:0/1617208866 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f08e0096050 con 0x7f08e4100e70 2024-05-24T11:36:34.654 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:34.655 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:34.655 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:34.655 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:34.655 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:34.655 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:34.655 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: Deploying daemon mds.h on smithi053 2024-05-24T11:36:34.655 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] up:boot 2024-05-24T11:36:34.655 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: fsmap 7 up:standby 2024-05-24T11:36:34.655 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:34 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-24T11:36:34.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.750+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08e4067380 con 0x7f08e4100e70 2024-05-24T11:36:34.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.750+0000 7f08da7fc700 1 -- 172.21.15.39:0/1617208866 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5802 (secure 0 0 0) 0x7f08e0099670 con 0x7f08e4100e70 2024-05-24T11:36:34.755 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:34.755 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08d00a11a0 msgr2=0x7f08d00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08d00a11a0 0x7f08d00a3660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f08d4005950 tx=0x7f08d40058e0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4100e70 msgr2=0x7f08e4074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4100e70 0x7f08e4074d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c920 tx=0x7f08e000cce0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 shutdown_connections 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08d00a11a0 0x7f08d00a3660 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e4102980 0x7f08e40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08e4100e70 0x7f08e4074d00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 --2- 172.21.15.39:0/1617208866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08e4101fc0 0x7f08e4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:34.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 >> 172.21.15.39:0/1617208866 conn(0x7f08e40fc400 msgr2=0x7f08e4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:34.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 shutdown_connections 2024-05-24T11:36:34.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:34.752+0000 7f08eb48e700 1 -- 172.21.15.39:0/1617208866 wait complete. 2024-05-24T11:36:34.758 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: Deploying daemon mds.h on smithi053 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] up:boot 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: fsmap 7 up:standby 2024-05-24T11:36:34.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:34 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-24T11:36:34.946 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 24 11:36:34 smithi053 systemd[1]: Starting Ceph mds.h for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:34.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:34.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:34.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:34.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:34.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:34.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:34.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: Deploying daemon mds.h on smithi053 2024-05-24T11:36:34.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] up:boot 2024-05-24T11:36:34.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: fsmap 7 up:standby 2024-05-24T11:36:34.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:34 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-24T11:36:35.974 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:35 smithi137 ceph-mon[47907]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:35.974 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:35 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1617208866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:35.975 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:35.975 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:35.975 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:35.975 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:35.975 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:35.975 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:35 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:35.989 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 24 11:36:35 smithi053 bash[84379]: 03f129f03a6686264e5387ee02cc3e661358d8983205e38fb90de2333bfb0d81 2024-05-24T11:36:35.989 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 24 11:36:35 smithi053 systemd[1]: Started Ceph mds.h for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:35.989 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 24 11:36:35 smithi053 ceph-mds[84417]: -- 172.21.15.53:0/1735179876 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a436c801a0 con 0x55a436c56c00 2024-05-24T11:36:35.989 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 24 11:36:35 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-h[84394]: starting mds.h at 2024-05-24T11:36:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:35 smithi053 ceph-mon[47448]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:35 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1617208866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:35.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:35.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:35.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:35.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:35.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:35 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:36.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:35 smithi039 ceph-mon[42919]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:36.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:35 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1617208866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:36.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:36.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:36.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:36.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:36.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:36.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:35 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:36.770 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:36.837 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:36 smithi137 ceph-mon[47907]: Deploying daemon mds.i on smithi137 2024-05-24T11:36:36.837 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:36 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] up:boot 2024-05-24T11:36:36.837 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:36 smithi137 ceph-mon[47907]: fsmap 8 up:standby 2024-05-24T11:36:36.837 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:36 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-24T11:36:36.837 INFO:journalctl@ceph.mds.i.smithi137.stdout:May 24 11:36:36 smithi137 systemd[1]: Starting Ceph mds.i for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 -- 172.21.15.39:0/884813761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98100e90 msgr2=0x7f8f981012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/884813761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98100e90 0x7f8f981012b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0098f0 tx=0x7f8f8c009c00 comp rx=0 tx=0).stop 2024-05-24T11:36:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 -- 172.21.15.39:0/884813761 shutdown_connections 2024-05-24T11:36:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/884813761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f981029a0 0x7f8f981092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/884813761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98100e90 0x7f8f981012b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/884813761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f98101fe0 0x7f8f98102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 -- 172.21.15.39:0/884813761 >> 172.21.15.39:0/884813761 conn(0x7f8f980fc400 msgr2=0x7f8f980fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 -- 172.21.15.39:0/884813761 shutdown_connections 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 -- 172.21.15.39:0/884813761 wait complete. 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 Processor -- start 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.907+0000 7f8f9ea65700 1 -- start start 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9ea65700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f98100e90 0x7f8f98196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9ea65700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98101fe0 0x7f8f981972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9ea65700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f981029a0 0x7f8f9819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9ea65700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9819e870 con 0x7f8f98101fe0 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9ea65700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9819e9e0 con 0x7f8f981029a0 2024-05-24T11:36:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9ea65700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9819eba0 con 0x7f8f98100e90 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f97fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98101fe0 0x7f8f981972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9c801700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f98100e90 0x7f8f98196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f97fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98101fe0 0x7f8f981972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36030/0 (socket says 172.21.15.39:36030) 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f97fff700 1 -- 172.21.15.39:0/3117702771 learned_addr learned my addr 172.21.15.39:0/3117702771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9c801700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f98100e90 0x7f8f98196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:44926/0 (socket says 172.21.15.39:44926) 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9d002700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f981029a0 0x7f8f9819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f97fff700 1 -- 172.21.15.39:0/3117702771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f98100e90 msgr2=0x7f8f98196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f97fff700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f98100e90 0x7f8f98196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f97fff700 1 -- 172.21.15.39:0/3117702771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f981029a0 msgr2=0x7f8f9819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f97fff700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f981029a0 0x7f8f9819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f97fff700 1 -- 172.21.15.39:0/3117702771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840095f0 con 0x7f8f98101fe0 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9d002700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f981029a0 0x7f8f9819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9c801700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f98100e90 0x7f8f98196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:36:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f97fff700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98101fe0 0x7f8f981972c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400ec10 tx=0x7f8f8400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f95ffb700 1 -- 172.21.15.39:0/3117702771 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8400cd40 con 0x7f8f98101fe0 2024-05-24T11:36:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f95ffb700 1 -- 172.21.15.39:0/3117702771 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f840044b0 con 0x7f8f98101fe0 2024-05-24T11:36:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c009520 con 0x7f8f98101fe0 2024-05-24T11:36:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.908+0000 7f8f95ffb700 1 -- 172.21.15.39:0/3117702771 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84010430 con 0x7f8f98101fe0 2024-05-24T11:36:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.909+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9819f1a0 con 0x7f8f98101fe0 2024-05-24T11:36:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.910+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f981083f0 con 0x7f8f98101fe0 2024-05-24T11:36:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.911+0000 7f8f95ffb700 1 -- 172.21.15.39:0/3117702771 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8f84004000 con 0x7f8f98101fe0 2024-05-24T11:36:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.911+0000 7f8f95ffb700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8f880a11a0 0x7f8f880a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.911+0000 7f8f9c801700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8f880a11a0 0x7f8f880a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.911+0000 7f8f95ffb700 1 -- 172.21.15.39:0/3117702771 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8f84014070 con 0x7f8f98101fe0 2024-05-24T11:36:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.912+0000 7f8f9c801700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8f880a11a0 0x7f8f880a3660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0098c0 tx=0x7f8f8c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:36.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:36.915+0000 7f8f95ffb700 1 -- 172.21.15.39:0/3117702771 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8f84096520 con 0x7f8f98101fe0 2024-05-24T11:36:36.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:36 smithi053 ceph-mon[47448]: Deploying daemon mds.i on smithi137 2024-05-24T11:36:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:36 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] up:boot 2024-05-24T11:36:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:36 smithi053 ceph-mon[47448]: fsmap 8 up:standby 2024-05-24T11:36:36.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:36 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-24T11:36:37.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:36 smithi039 ceph-mon[42919]: Deploying daemon mds.i on smithi137 2024-05-24T11:36:37.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:36 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] up:boot 2024-05-24T11:36:37.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:36 smithi039 ceph-mon[42919]: fsmap 8 up:standby 2024-05-24T11:36:37.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:36 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-24T11:36:37.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.171+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f9819ed60 con 0x7f8f98101fe0 2024-05-24T11:36:37.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.171+0000 7f8f95ffb700 1 -- 172.21.15.39:0/3117702771 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6559 (secure 0 0 0) 0x7f8f84099b40 con 0x7f8f98101fe0 2024-05-24T11:36:37.176 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:37.176 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-24T11:36:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.173+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8f880a11a0 msgr2=0x7f8f880a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8f880a11a0 0x7f8f880a3660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0098c0 tx=0x7f8f8c019040 comp rx=0 tx=0).stop 2024-05-24T11:36:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98101fe0 msgr2=0x7f8f981972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98101fe0 0x7f8f981972c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400ec10 tx=0x7f8f8400ef20 comp rx=0 tx=0).stop 2024-05-24T11:36:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 shutdown_connections 2024-05-24T11:36:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8f880a11a0 0x7f8f880a3660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f98100e90 0x7f8f98196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f981029a0 0x7f8f9819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 --2- 172.21.15.39:0/3117702771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f98101fe0 0x7f8f981972c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 >> 172.21.15.39:0/3117702771 conn(0x7f8f980fc400 msgr2=0x7f8f98105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 shutdown_connections 2024-05-24T11:36:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:37.174+0000 7f8f9ea65700 1 -- 172.21.15.39:0/3117702771 wait complete. 2024-05-24T11:36:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 9 2024-05-24T11:36:37.495 INFO:journalctl@ceph.mds.i.smithi137.stdout:May 24 11:36:37 smithi137 bash[85554]: 1a56c573683321b7a3a218033ec5460c9248492dd785a2299db783d2ea73dfb9 2024-05-24T11:36:37.495 INFO:journalctl@ceph.mds.i.smithi137.stdout:May 24 11:36:37 smithi137 systemd[1]: Started Ceph mds.i for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:37.495 INFO:journalctl@ceph.mds.i.smithi137.stdout:May 24 11:36:37 smithi137 ceph-mds[85591]: -- 172.21.15.137:0/1547643004 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55829e2fe1a0 con 0x55829e2d4c00 2024-05-24T11:36:37.495 INFO:journalctl@ceph.mds.i.smithi137.stdout:May 24 11:36:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-i[85568]: starting mds.i at 2024-05-24T11:36:37.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:37 smithi039 ceph-mon[42919]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:37.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:37 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3117702771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:37.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:37.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:37.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:37.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:37.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:37.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:37 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:37 smithi053 ceph-mon[47448]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:37 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3117702771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:37.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:37 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:37.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:37 smithi137 ceph-mon[47907]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:37.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:37 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3117702771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:37.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:37.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:37.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:37.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:37.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:37.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:37 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:38 smithi053 ceph-mon[47448]: Deploying daemon mds.g on smithi039 2024-05-24T11:36:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:38 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] up:boot 2024-05-24T11:36:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:38 smithi053 ceph-mon[47448]: fsmap 9 up:standby 2024-05-24T11:36:38.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:38 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-24T11:36:38.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:38 smithi137 ceph-mon[47907]: Deploying daemon mds.g on smithi039 2024-05-24T11:36:38.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:38 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] up:boot 2024-05-24T11:36:38.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:38 smithi137 ceph-mon[47907]: fsmap 9 up:standby 2024-05-24T11:36:38.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:38 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-24T11:36:38.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:38 smithi039 ceph-mon[42919]: Deploying daemon mds.g on smithi039 2024-05-24T11:36:38.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:38 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] up:boot 2024-05-24T11:36:38.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:38 smithi039 ceph-mon[42919]: fsmap 9 up:standby 2024-05-24T11:36:38.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:38 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-24T11:36:38.997 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 24 11:36:38 smithi039 systemd[1]: Starting Ceph mds.g for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:39.191 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.325+0000 7fe26ee11700 1 -- 172.21.15.39:0/3902859028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe268108d40 msgr2=0x7fe26810b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.325+0000 7fe26ee11700 1 --2- 172.21.15.39:0/3902859028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe268108d40 0x7fe26810b1b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe26000b0a0 tx=0x7fe26000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.325+0000 7fe26ee11700 1 -- 172.21.15.39:0/3902859028 shutdown_connections 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.325+0000 7fe26ee11700 1 --2- 172.21.15.39:0/3902859028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe268074f80 0x7fe2680733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.325+0000 7fe26ee11700 1 --2- 172.21.15.39:0/3902859028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe268108d40 0x7fe26810b1b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.325+0000 7fe26ee11700 1 --2- 172.21.15.39:0/3902859028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2680739a0 0x7fe268073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.325+0000 7fe26ee11700 1 -- 172.21.15.39:0/3902859028 >> 172.21.15.39:0/3902859028 conn(0x7fe2680fc400 msgr2=0x7fe2680fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.325+0000 7fe26ee11700 1 -- 172.21.15.39:0/3902859028 shutdown_connections 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.325+0000 7fe26ee11700 1 -- 172.21.15.39:0/3902859028 wait complete. 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26ee11700 1 Processor -- start 2024-05-24T11:36:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26ee11700 1 -- start start 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26ee11700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680739a0 0x7fe268198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26ee11700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe268074f80 0x7fe2681994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26ee11700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe268108d40 0x7fe2681a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26ee11700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2681a0ad0 con 0x7fe2680739a0 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26ee11700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2681a0c10 con 0x7fe268074f80 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26ee11700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2681a0da0 con 0x7fe268108d40 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26cbad700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680739a0 0x7fe268198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26cbad700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680739a0 0x7fe268198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38670/0 (socket says 172.21.15.39:38670) 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26cbad700 1 -- 172.21.15.39:0/2816023059 learned_addr learned my addr 172.21.15.39:0/2816023059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe267fff700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe268074f80 0x7fe2681994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26d3ae700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe268108d40 0x7fe2681a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26cbad700 1 -- 172.21.15.39:0/2816023059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe268108d40 msgr2=0x7fe2681a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26cbad700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe268108d40 0x7fe2681a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26cbad700 1 -- 172.21.15.39:0/2816023059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe268074f80 msgr2=0x7fe2681994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26cbad700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe268074f80 0x7fe2681994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe26cbad700 1 -- 172.21.15.39:0/2816023059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe260009d00 con 0x7fe2680739a0 2024-05-24T11:36:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.326+0000 7fe267fff700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe268074f80 0x7fe2681994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.327+0000 7fe26cbad700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680739a0 0x7fe268198fa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00d950 tx=0x7fe25c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.327+0000 7fe265ffb700 1 -- 172.21.15.39:0/2816023059 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c009b40 con 0x7fe2680739a0 2024-05-24T11:36:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.327+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2681a1080 con 0x7fe2680739a0 2024-05-24T11:36:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.327+0000 7fe265ffb700 1 -- 172.21.15.39:0/2816023059 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe25c009ca0 con 0x7fe2680739a0 2024-05-24T11:36:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.327+0000 7fe265ffb700 1 -- 172.21.15.39:0/2816023059 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c00f620 con 0x7fe2680739a0 2024-05-24T11:36:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.327+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2681a15e0 con 0x7fe2680739a0 2024-05-24T11:36:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.328+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe268107b50 con 0x7fe2680739a0 2024-05-24T11:36:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.329+0000 7fe265ffb700 1 -- 172.21.15.39:0/2816023059 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe25c010460 con 0x7fe2680739a0 2024-05-24T11:36:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.330+0000 7fe265ffb700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2580a1120 0x7fe2580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.330+0000 7fe265ffb700 1 -- 172.21.15.39:0/2816023059 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe25c0cc8c0 con 0x7fe2680739a0 2024-05-24T11:36:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.330+0000 7fe267fff700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2580a1120 0x7fe2580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.330+0000 7fe267fff700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2580a1120 0x7fe2580a35e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe254009cb0 tx=0x7fe254009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:39.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.337+0000 7fe265ffb700 1 -- 172.21.15.39:0/2816023059 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe25c096450 con 0x7fe2680739a0 2024-05-24T11:36:39.577 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 24 11:36:39 smithi039 ceph-mds[94237]: -- 172.21.15.39:0/2310994353 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56529fa221a0 con 0x56529f9f8c00 2024-05-24T11:36:39.577 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 24 11:36:39 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-g[94214]: starting mds.g at 2024-05-24T11:36:39.578 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 24 11:36:39 smithi039 bash[94199]: ee672293445dd318d000200a7877e9ee46b3ad12f2f3aba588541fa65a8b5bc0 2024-05-24T11:36:39.578 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 24 11:36:39 smithi039 systemd[1]: Started Ceph mds.g for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:39.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.611+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe26810e940 con 0x7fe2680739a0 2024-05-24T11:36:39.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.612+0000 7fe265ffb700 1 -- 172.21.15.39:0/2816023059 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7318 (secure 0 0 0) 0x7fe25c099a70 con 0x7fe2680739a0 2024-05-24T11:36:39.616 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:39.617 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/218688120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":218688120},{"type":"v1","addr":"172.21.15.137:6839","nonce":218688120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-24T11:36:39.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2580a1120 msgr2=0x7fe2580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2580a1120 0x7fe2580a35e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe254009cb0 tx=0x7fe254009260 comp rx=0 tx=0).stop 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680739a0 msgr2=0x7fe268198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680739a0 0x7fe268198fa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00d950 tx=0x7fe25c00dd10 comp rx=0 tx=0).stop 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 shutdown_connections 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2580a1120 0x7fe2580a35e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe268108d40 0x7fe2681a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2680739a0 0x7fe268198fa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 --2- 172.21.15.39:0/2816023059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe268074f80 0x7fe2681994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 >> 172.21.15.39:0/2816023059 conn(0x7fe2680fc400 msgr2=0x7fe268104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 shutdown_connections 2024-05-24T11:36:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:39.614+0000 7fe26ee11700 1 -- 172.21.15.39:0/2816023059 wait complete. 2024-05-24T11:36:39.620 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 10 2024-05-24T11:36:39.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:39 smithi053 ceph-mon[47448]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:39.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:39.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:39.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:39.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:39.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:39.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:39 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:39.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:39 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2816023059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:39.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:39 smithi137 ceph-mon[47907]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:39.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:39.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:39.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:39.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:39.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:39.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:39 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:39.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:39 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2816023059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:40.048 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:39 smithi039 ceph-mon[42919]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:40.048 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:40.048 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:40.048 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:40.049 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:40.049 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:40.049 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:39 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:40.049 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:39 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2816023059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:40.610 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 24 11:36:40 smithi039 systemd[1]: Starting Ceph mds.a for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:40.910 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:40 smithi039 ceph-mon[42919]: Deploying daemon mds.a on smithi039 2024-05-24T11:36:40.911 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:40 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] up:boot 2024-05-24T11:36:40.911 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:40 smithi039 ceph-mon[42919]: fsmap 10 up:standby 2024-05-24T11:36:40.911 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:40 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-24T11:36:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:40 smithi053 ceph-mon[47448]: Deploying daemon mds.a on smithi039 2024-05-24T11:36:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:40 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] up:boot 2024-05-24T11:36:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:40 smithi053 ceph-mon[47448]: fsmap 10 up:standby 2024-05-24T11:36:40.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:40 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-24T11:36:40.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:40 smithi137 ceph-mon[47907]: Deploying daemon mds.a on smithi039 2024-05-24T11:36:40.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:40 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] up:boot 2024-05-24T11:36:40.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:40 smithi137 ceph-mon[47907]: fsmap 10 up:standby 2024-05-24T11:36:40.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:40 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-24T11:36:41.327 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 24 11:36:41 smithi039 ceph-mds[94623]: -- 172.21.15.39:0/1237397601 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e7efbf81a0 con 0x55e7efbcec00 2024-05-24T11:36:41.394 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 24 11:36:41 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-a[94600]: starting mds.a at 2024-05-24T11:36:41.394 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 24 11:36:41 smithi039 bash[94585]: 0ecd24dfb557746ecf3cf487eaf363d1b3c8488e0b37a66c026ec5812cc32771 2024-05-24T11:36:41.394 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 24 11:36:41 smithi039 systemd[1]: Started Ceph mds.a for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:41.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:41.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.767+0000 7f5bd4db7700 1 -- 172.21.15.39:0/2847589134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 msgr2=0x7f5bd01013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.767+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/2847589134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 0x7f5bd01013b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80099c0 tx=0x7f5bb8009cd0 comp rx=0 tx=0).stop 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.767+0000 7f5bd4db7700 1 -- 172.21.15.39:0/2847589134 shutdown_connections 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.767+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/2847589134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd01018f0 0x7f5bd01081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.767+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/2847589134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd00ffd60 0x7f5bd0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.767+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/2847589134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 0x7f5bd01013b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.767+0000 7f5bd4db7700 1 -- 172.21.15.39:0/2847589134 >> 172.21.15.39:0/2847589134 conn(0x7f5bd00fb2f0 msgr2=0x7f5bd00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.767+0000 7f5bd4db7700 1 -- 172.21.15.39:0/2847589134 shutdown_connections 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.767+0000 7f5bd4db7700 1 -- 172.21.15.39:0/2847589134 wait complete. 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bd4db7700 1 Processor -- start 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bd4db7700 1 -- start start 2024-05-24T11:36:41.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bd4db7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd00ffd60 0x7f5bd0103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bd4db7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 0x7f5bd0102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bd4db7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd01018f0 0x7f5bd0102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bd4db7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0103010 con 0x7f5bd0100f50 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bd4db7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0103180 con 0x7f5bd00ffd60 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bd4db7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd01a0df0 con 0x7f5bd01018f0 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bcdd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 0x7f5bd0102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bcdd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 0x7f5bd0102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38710/0 (socket says 172.21.15.39:38710) 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bcdd9b700 1 -- 172.21.15.39:0/3787059726 learned_addr learned my addr 172.21.15.39:0/3787059726 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bce59c700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd00ffd60 0x7f5bd0103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bced9d700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd01018f0 0x7f5bd0102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bcdd9b700 1 -- 172.21.15.39:0/3787059726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd01018f0 msgr2=0x7f5bd0102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bcdd9b700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd01018f0 0x7f5bd0102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bcdd9b700 1 -- 172.21.15.39:0/3787059726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd00ffd60 msgr2=0x7f5bd0103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bcdd9b700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd00ffd60 0x7f5bd0103ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.768+0000 7f5bcdd9b700 1 -- 172.21.15.39:0/3787059726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb80095f0 con 0x7f5bd0100f50 2024-05-24T11:36:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.769+0000 7f5bcdd9b700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 0x7f5bd0102330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009990 tx=0x7f5bb800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.769+0000 7f5bc77fe700 1 -- 172.21.15.39:0/3787059726 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb801d070 con 0x7f5bd0100f50 2024-05-24T11:36:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.769+0000 7f5bc77fe700 1 -- 172.21.15.39:0/3787059726 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bb800bd10 con 0x7f5bd0100f50 2024-05-24T11:36:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.769+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd01a1020 con 0x7f5bd0100f50 2024-05-24T11:36:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.769+0000 7f5bc77fe700 1 -- 172.21.15.39:0/3787059726 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8017650 con 0x7f5bd0100f50 2024-05-24T11:36:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.769+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd01a1440 con 0x7f5bd0100f50 2024-05-24T11:36:41.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.770+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd0067c10 con 0x7f5bd0100f50 2024-05-24T11:36:41.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.771+0000 7f5bc77fe700 1 -- 172.21.15.39:0/3787059726 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5bb8022470 con 0x7f5bd0100f50 2024-05-24T11:36:41.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.771+0000 7f5bc77fe700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f5bbc0a11a0 0x7f5bbc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:41.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.772+0000 7f5bce59c700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f5bbc0a11a0 0x7f5bbc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:41.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.772+0000 7f5bc77fe700 1 -- 172.21.15.39:0/3787059726 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5bb80cdae0 con 0x7f5bd0100f50 2024-05-24T11:36:41.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.772+0000 7f5bce59c700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f5bbc0a11a0 0x7f5bbc0a3660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0100db0 tx=0x7f5bc0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:41.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:41.775+0000 7f5bc77fe700 1 -- 172.21.15.39:0/3787059726 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5bb80977a0 con 0x7f5bd0100f50 2024-05-24T11:36:41.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:41 smithi053 ceph-mon[47448]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:41.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:41.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:41.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:41.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:41.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:41.881 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:41 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:41.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:41 smithi137 ceph-mon[47907]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:41.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:41.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:41.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:41.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:41.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:41.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:41 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:42.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.034+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bd010b470 con 0x7f5bd0100f50 2024-05-24T11:36:42.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.035+0000 7f5bc77fe700 1 -- 172.21.15.39:0/3787059726 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8834 (secure 0 0 0) 0x7f5bb809adc0 con 0x7f5bd0100f50 2024-05-24T11:36:42.039 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:42.039 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/3006971240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3006971240},{"type":"v1","addr":"172.21.15.39:6839","nonce":3006971240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/3024712815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3024712815},{"type":"v1","addr":"172.21.15.39:6841","nonce":3024712815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/218688120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":218688120},{"type":"v1","addr":"172.21.15.137:6839","nonce":218688120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-24T11:36:42.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f5bbc0a11a0 msgr2=0x7f5bbc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:42.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f5bbc0a11a0 0x7f5bbc0a3660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0100db0 tx=0x7f5bc0009260 comp rx=0 tx=0).stop 2024-05-24T11:36:42.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 msgr2=0x7f5bd0102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:42.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 0x7f5bd0102330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009990 tx=0x7f5bb800fa00 comp rx=0 tx=0).stop 2024-05-24T11:36:42.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 shutdown_connections 2024-05-24T11:36:42.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f5bbc0a11a0 0x7f5bbc0a3660 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:42.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd01018f0 0x7f5bd0102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:42.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd00ffd60 0x7f5bd0103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:42.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 --2- 172.21.15.39:0/3787059726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd0100f50 0x7f5bd0102330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:42.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 >> 172.21.15.39:0/3787059726 conn(0x7f5bd00fb2f0 msgr2=0x7f5bd0068060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:42.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 shutdown_connections 2024-05-24T11:36:42.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:42.037+0000 7f5bd4db7700 1 -- 172.21.15.39:0/3787059726 wait complete. 2024-05-24T11:36:42.042 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 12 2024-05-24T11:36:42.053 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:41 smithi039 ceph-mon[42919]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:42.053 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:42.053 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:42.053 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:42.053 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-24T11:36:42.053 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-24T11:36:42.053 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:41 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:42.872 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:42 smithi053 ceph-mon[47448]: Deploying daemon mds.b on smithi053 2024-05-24T11:36:42.872 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:42 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.39:6840/3024712815,v1:172.21.15.39:6841/3024712815] up:boot 2024-05-24T11:36:42.872 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:42 smithi053 ceph-mon[47448]: fsmap 11 up:standby 2024-05-24T11:36:42.872 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:42 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-24T11:36:42.872 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:42 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3787059726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:42.872 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:42 smithi053 systemd[1]: Starting Ceph mds.b for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:36:42.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:42 smithi137 ceph-mon[47907]: Deploying daemon mds.b on smithi053 2024-05-24T11:36:42.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:42 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.39:6840/3024712815,v1:172.21.15.39:6841/3024712815] up:boot 2024-05-24T11:36:42.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:42 smithi137 ceph-mon[47907]: fsmap 11 up:standby 2024-05-24T11:36:42.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:42 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-24T11:36:42.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:42 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3787059726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:43.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:42 smithi039 ceph-mon[42919]: Deploying daemon mds.b on smithi053 2024-05-24T11:36:43.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:42 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.39:6840/3024712815,v1:172.21.15.39:6841/3024712815] up:boot 2024-05-24T11:36:43.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:42 smithi039 ceph-mon[42919]: fsmap 11 up:standby 2024-05-24T11:36:43.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-24T11:36:43.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:42 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3787059726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:43.239 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:42 smithi053 ceph-mds[84759]: -- 172.21.15.53:0/2299005948 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f5226ca1a0 con 0x55f5226a0c00 2024-05-24T11:36:43.239 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:42 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-b[84735]: starting mds.b at 2024-05-24T11:36:43.239 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:42 smithi053 bash[84720]: 48a982d25118161bc8f0519de0f2598613f1990561493e3090e3587ebe51f8f4 2024-05-24T11:36:43.239 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:42 smithi053 systemd[1]: Started Ceph mds.b for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:36:43.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:43 smithi053 ceph-mon[47448]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:43.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:43.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:43 smithi137 ceph-mon[47907]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:43.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:43.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:44.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:44.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:43 smithi039 ceph-mon[42919]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:44.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.200+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/4261031497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a940ff730 msgr2=0x7f2a940ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.200+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/4261031497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a940ff730 0x7f2a940ffb50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2a880099c0 tx=0x7f2a88009cd0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/4261031497 shutdown_connections 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/4261031497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a94100a50 0x7f2a94100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/4261031497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a940ff730 0x7f2a940ffb50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/4261031497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a94100090 0x7f2a94100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/4261031497 >> 172.21.15.39:0/4261031497 conn(0x7f2a940fb2f0 msgr2=0x7f2a940fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/4261031497 shutdown_connections 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/4261031497 wait complete. 2024-05-24T11:36:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 Processor -- start 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 -- start start 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a940ff730 0x7f2a94198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a94100090 0x7f2a94199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a94100a50 0x7f2a941a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a0a10 con 0x7f2a94100a50 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a0b50 con 0x7f2a94100090 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.201+0000 7f2a9b1bb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a0ce0 con 0x7f2a940ff730 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a99758700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a94100a50 0x7f2a941a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a99758700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a94100a50 0x7f2a941a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38728/0 (socket says 172.21.15.39:38728) 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a98f57700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a940ff730 0x7f2a94198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a99758700 1 -- 172.21.15.39:0/1695917167 learned_addr learned my addr 172.21.15.39:0/1695917167 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a93fff700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a94100090 0x7f2a94199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a99758700 1 -- 172.21.15.39:0/1695917167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a940ff730 msgr2=0x7f2a94198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a99758700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a940ff730 0x7f2a94198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a99758700 1 -- 172.21.15.39:0/1695917167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a94100090 msgr2=0x7f2a94199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a99758700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a94100090 0x7f2a94199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a99758700 1 -- 172.21.15.39:0/1695917167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880095f0 con 0x7f2a94100a50 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a99758700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a94100a50 0x7f2a941a0440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00ef30 tx=0x7f2a8c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a91ffb700 1 -- 172.21.15.39:0/1695917167 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c018070 con 0x7f2a94100a50 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a91ffb700 1 -- 172.21.15.39:0/1695917167 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a8c00f040 con 0x7f2a94100a50 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a941a0f60 con 0x7f2a94100a50 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a91ffb700 1 -- 172.21.15.39:0/1695917167 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00cbc0 con 0x7f2a94100a50 2024-05-24T11:36:44.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.202+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a941a1390 con 0x7f2a94100a50 2024-05-24T11:36:44.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.204+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9410a830 con 0x7f2a94100a50 2024-05-24T11:36:44.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.205+0000 7f2a91ffb700 1 -- 172.21.15.39:0/1695917167 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2a8c00cd80 con 0x7f2a94100a50 2024-05-24T11:36:44.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.205+0000 7f2a91ffb700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2a840a1120 0x7f2a840a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:44.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.205+0000 7f2a91ffb700 1 -- 172.21.15.39:0/1695917167 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a8c0cbe70 con 0x7f2a94100a50 2024-05-24T11:36:44.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.205+0000 7f2a98f57700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2a840a1120 0x7f2a840a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:44.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.206+0000 7f2a98f57700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2a840a1120 0x7f2a840a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a88006010 tx=0x7f2a88003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:44.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.210+0000 7f2a91ffb700 1 -- 172.21.15.39:0/1695917167 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2a8c095b30 con 0x7f2a94100a50 2024-05-24T11:36:44.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.492+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a9410e700 con 0x7f2a94100a50 2024-05-24T11:36:44.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.492+0000 7f2a91ffb700 1 -- 172.21.15.39:0/1695917167 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9592 (secure 0 0 0) 0x7f2a8c099150 con 0x7f2a94100a50 2024-05-24T11:36:44.497 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:44.497 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/3006971240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3006971240},{"type":"v1","addr":"172.21.15.39:6839","nonce":3006971240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/3024712815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3024712815},{"type":"v1","addr":"172.21.15.39:6841","nonce":3024712815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/218688120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":218688120},{"type":"v1","addr":"172.21.15.137:6839","nonce":218688120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/3196235839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3196235839},{"type":"v1","addr":"172.21.15.53:6839","nonce":3196235839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-24T11:36:44.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.494+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2a840a1120 msgr2=0x7f2a840a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.494+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2a840a1120 0x7f2a840a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a88006010 tx=0x7f2a88003680 comp rx=0 tx=0).stop 2024-05-24T11:36:44.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.494+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a94100a50 msgr2=0x7f2a941a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.494+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a94100a50 0x7f2a941a0440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00ef30 tx=0x7f2a8c00c3a0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.495+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 shutdown_connections 2024-05-24T11:36:44.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.495+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2a840a1120 0x7f2a840a35e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.495+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a940ff730 0x7f2a94198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.495+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a94100a50 0x7f2a941a0440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.495+0000 7f2a9b1bb700 1 --2- 172.21.15.39:0/1695917167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a94100090 0x7f2a94199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.495+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 >> 172.21.15.39:0/1695917167 conn(0x7f2a940fb2f0 msgr2=0x7f2a940fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:44.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.495+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 shutdown_connections 2024-05-24T11:36:44.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.495+0000 7f2a9b1bb700 1 -- 172.21.15.39:0/1695917167 wait complete. 2024-05-24T11:36:44.500 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 13 2024-05-24T11:36:44.512 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:36:44.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.675+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1519993550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc101900 msgr2=0x7fcacc1081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.675+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1519993550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc101900 0x7fcacc1081b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcac400b0d0 tx=0x7fcac400b3e0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.675+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1519993550 shutdown_connections 2024-05-24T11:36:44.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.675+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1519993550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcacc0ffd70 0x7fcacc100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.675+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1519993550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc101900 0x7fcacc1081b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.675+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1519993550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc100f60 0x7fcacc1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.675+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1519993550 >> 172.21.15.39:0/1519993550 conn(0x7fcacc0fb2e0 msgr2=0x7fcacc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.675+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1519993550 shutdown_connections 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1519993550 wait complete. 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad3d1f700 1 Processor -- start 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad3d1f700 1 -- start start 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad3d1f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc0ffd70 0x7fcacc069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad3d1f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc100f60 0x7fcacc19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad3d1f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcacc101900 0x7fcacc19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad3d1f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc19e8c0 con 0x7fcacc0ffd70 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad3d1f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc19ea30 con 0x7fcacc100f60 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad3d1f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc19ebf0 con 0x7fcacc101900 2024-05-24T11:36:44.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad1abb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc0ffd70 0x7fcacc069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad1abb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc0ffd70 0x7fcacc069fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38742/0 (socket says 172.21.15.39:38742) 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad1abb700 1 -- 172.21.15.39:0/1654558432 learned_addr learned my addr 172.21.15.39:0/1654558432 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad12ba700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc100f60 0x7fcacc19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.676+0000 7fcad22bc700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcacc101900 0x7fcacc19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcad1abb700 1 -- 172.21.15.39:0/1654558432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcacc101900 msgr2=0x7fcacc19e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcad1abb700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcacc101900 0x7fcacc19e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcad1abb700 1 -- 172.21.15.39:0/1654558432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc100f60 msgr2=0x7fcacc19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcad1abb700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc100f60 0x7fcacc19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcad1abb700 1 -- 172.21.15.39:0/1654558432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac4009d00 con 0x7fcacc0ffd70 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcad12ba700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc100f60 0x7fcacc19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcad1abb700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc0ffd70 0x7fcacc069fb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcac800e970 tx=0x7fcac800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:44.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcac2ffd700 1 -- 172.21.15.39:0/1654558432 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac8009790 con 0x7fcacc0ffd70 2024-05-24T11:36:44.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcac2ffd700 1 -- 172.21.15.39:0/1654558432 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcac800c7e0 con 0x7fcacc0ffd70 2024-05-24T11:36:44.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc19eed0 con 0x7fcacc0ffd70 2024-05-24T11:36:44.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcac2ffd700 1 -- 172.21.15.39:0/1654558432 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac8003bd0 con 0x7fcacc0ffd70 2024-05-24T11:36:44.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.677+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc19f300 con 0x7fcacc0ffd70 2024-05-24T11:36:44.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.678+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc107210 con 0x7fcacc0ffd70 2024-05-24T11:36:44.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.679+0000 7fcac2ffd700 1 -- 172.21.15.39:0/1654558432 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcac800c950 con 0x7fcacc0ffd70 2024-05-24T11:36:44.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.680+0000 7fcac2ffd700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcab80a1120 0x7fcab80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:44.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.680+0000 7fcac2ffd700 1 -- 172.21.15.39:0/1654558432 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcac8014070 con 0x7fcacc0ffd70 2024-05-24T11:36:44.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.680+0000 7fcad12ba700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcab80a1120 0x7fcab80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:44.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.680+0000 7fcad12ba700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcab80a1120 0x7fcab80a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcacc06a510 tx=0x7fcabc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:44.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.684+0000 7fcac2ffd700 1 -- 172.21.15.39:0/1654558432 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcac80963d0 con 0x7fcacc0ffd70 2024-05-24T11:36:44.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] up:boot 2024-05-24T11:36:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: fsmap 12 up:standby 2024-05-24T11:36:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-24T11:36:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:44 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1695917167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:44.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.926+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcacc068060 con 0x7fcacc0ffd70 2024-05-24T11:36:44.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.927+0000 7fcac2ffd700 1 -- 172.21.15.39:0/1654558432 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18364 (secure 0 0 0) 0x7fcac80999f0 con 0x7fcacc0ffd70 2024-05-24T11:36:44.931 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:44.931 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":73,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:15.592615+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:36:44.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcab80a1120 msgr2=0x7fcab80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcab80a1120 0x7fcab80a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcacc06a510 tx=0x7fcabc009260 comp rx=0 tx=0).stop 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc0ffd70 msgr2=0x7fcacc069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc0ffd70 0x7fcacc069fb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcac800e970 tx=0x7fcac800ec80 comp rx=0 tx=0).stop 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 shutdown_connections 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fcab80a1120 0x7fcab80a35e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcacc101900 0x7fcacc19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcacc0ffd70 0x7fcacc069fb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 --2- 172.21.15.39:0/1654558432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc100f60 0x7fcacc19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 >> 172.21.15.39:0/1654558432 conn(0x7fcacc0fb2e0 msgr2=0x7fcacc104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 shutdown_connections 2024-05-24T11:36:44.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:44.929+0000 7fcad3d1f700 1 -- 172.21.15.39:0/1654558432 wait complete. 2024-05-24T11:36:44.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:36:44.947 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-24T11:36:44.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] up:boot 2024-05-24T11:36:44.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: fsmap 12 up:standby 2024-05-24T11:36:44.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-24T11:36:44.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:44 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1695917167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] up:boot 2024-05-24T11:36:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: fsmap 12 up:standby 2024-05-24T11:36:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-24T11:36:44.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:44.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:44 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1695917167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.074+0000 7f41b83ec700 1 -- 172.21.15.39:0/212615110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0104330 msgr2=0x7f41b0106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.074+0000 7f41b83ec700 1 --2- 172.21.15.39:0/212615110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0104330 0x7f41b0106720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009990 tx=0x7f41a0009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.074+0000 7f41b83ec700 1 -- 172.21.15.39:0/212615110 shutdown_connections 2024-05-24T11:36:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.074+0000 7f41b83ec700 1 --2- 172.21.15.39:0/212615110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b0106c60 0x7f41b01090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.074+0000 7f41b83ec700 1 --2- 172.21.15.39:0/212615110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b0101a00 0x7f41b0103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.074+0000 7f41b83ec700 1 --2- 172.21.15.39:0/212615110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0104330 0x7f41b0106720 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.074+0000 7f41b83ec700 1 -- 172.21.15.39:0/212615110 >> 172.21.15.39:0/212615110 conn(0x7f41b00fb3f0 msgr2=0x7f41b00fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:45.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.074+0000 7f41b83ec700 1 -- 172.21.15.39:0/212615110 shutdown_connections 2024-05-24T11:36:45.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.074+0000 7f41b83ec700 1 -- 172.21.15.39:0/212615110 wait complete. 2024-05-24T11:36:45.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.075+0000 7f41b83ec700 1 Processor -- start 2024-05-24T11:36:45.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.075+0000 7f41b83ec700 1 -- start start 2024-05-24T11:36:45.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b83ec700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0101a00 0x7f41b0196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b83ec700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b0104330 0x7f41b0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b83ec700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b0106c60 0x7f41b019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b83ec700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b019e900 con 0x7f41b0101a00 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b83ec700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b019ea70 con 0x7f41b0104330 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b83ec700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b019ec30 con 0x7f41b0106c60 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6188700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0101a00 0x7f41b0196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6188700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0101a00 0x7f41b0196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38764/0 (socket says 172.21.15.39:38764) 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6188700 1 -- 172.21.15.39:0/2397897252 learned_addr learned my addr 172.21.15.39:0/2397897252 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6989700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b0106c60 0x7f41b019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b5987700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b0104330 0x7f41b0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6188700 1 -- 172.21.15.39:0/2397897252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b0106c60 msgr2=0x7f41b019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6188700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b0106c60 0x7f41b019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6188700 1 -- 172.21.15.39:0/2397897252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b0104330 msgr2=0x7f41b0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6188700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b0104330 0x7f41b0197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6188700 1 -- 172.21.15.39:0/2397897252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a00095f0 con 0x7f41b0101a00 2024-05-24T11:36:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b6188700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0101a00 0x7f41b0196e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00e9a0 tx=0x7f41ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41a77fe700 1 -- 172.21.15.39:0/2397897252 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac009790 con 0x7f41b0101a00 2024-05-24T11:36:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41a77fe700 1 -- 172.21.15.39:0/2397897252 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41ac010430 con 0x7f41b0101a00 2024-05-24T11:36:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b019ef10 con 0x7f41b0101a00 2024-05-24T11:36:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.076+0000 7f41a77fe700 1 -- 172.21.15.39:0/2397897252 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00cea0 con 0x7f41b0101a00 2024-05-24T11:36:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.077+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b019f340 con 0x7f41b0101a00 2024-05-24T11:36:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.078+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b00fcf70 con 0x7f41b0101a00 2024-05-24T11:36:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.078+0000 7f41a77fe700 1 -- 172.21.15.39:0/2397897252 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f41ac0105a0 con 0x7f41b0101a00 2024-05-24T11:36:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.079+0000 7f41a77fe700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f419c0a11a0 0x7f419c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.079+0000 7f41b5987700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f419c0a11a0 0x7f419c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.079+0000 7f41a77fe700 1 -- 172.21.15.39:0/2397897252 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f41ac014070 con 0x7f41b0101a00 2024-05-24T11:36:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.080+0000 7f41b5987700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f419c0a11a0 0x7f419c0a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005910 tx=0x7f41a000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:45.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.084+0000 7f41a77fe700 1 -- 172.21.15.39:0/2397897252 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f41ac052d60 con 0x7f41b0101a00 2024-05-24T11:36:45.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.294+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41b0067460 con 0x7f41b0101a00 2024-05-24T11:36:45.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.295+0000 7f41a77fe700 1 -- 172.21.15.39:0/2397897252 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41ac096190 con 0x7f41b0101a00 2024-05-24T11:36:45.299 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:36:45.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f419c0a11a0 msgr2=0x7f419c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f419c0a11a0 0x7f419c0a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005910 tx=0x7f41a000b610 comp rx=0 tx=0).stop 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0101a00 msgr2=0x7f41b0196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0101a00 0x7f41b0196e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00e9a0 tx=0x7f41ac00ecb0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 shutdown_connections 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f419c0a11a0 0x7f419c0a3660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b0106c60 0x7f41b019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b0101a00 0x7f41b0196e10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 --2- 172.21.15.39:0/2397897252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b0104330 0x7f41b0197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 >> 172.21.15.39:0/2397897252 conn(0x7f41b00fb3f0 msgr2=0x7f41b0103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 shutdown_connections 2024-05-24T11:36:45.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.297+0000 7f41b83ec700 1 -- 172.21.15.39:0/2397897252 wait complete. 2024-05-24T11:36:45.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-24T11:36:45.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.449+0000 7fa70f245700 1 -- 172.21.15.39:0/2984112875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7080ffd60 msgr2=0x7fa708100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.449+0000 7fa70f245700 1 --2- 172.21.15.39:0/2984112875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7080ffd60 0x7fa708100180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80099c0 tx=0x7fa6f8009cd0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.449+0000 7fa70f245700 1 -- 172.21.15.39:0/2984112875 shutdown_connections 2024-05-24T11:36:45.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.449+0000 7fa70f245700 1 --2- 172.21.15.39:0/2984112875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7081018f0 0x7fa7081081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.449+0000 7fa70f245700 1 --2- 172.21.15.39:0/2984112875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7080ffd60 0x7fa708100180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.449+0000 7fa70f245700 1 --2- 172.21.15.39:0/2984112875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa708100f50 0x7fa7081013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.449+0000 7fa70f245700 1 -- 172.21.15.39:0/2984112875 >> 172.21.15.39:0/2984112875 conn(0x7fa7080fb2f0 msgr2=0x7fa7080fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:45.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.449+0000 7fa70f245700 1 -- 172.21.15.39:0/2984112875 shutdown_connections 2024-05-24T11:36:45.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.449+0000 7fa70f245700 1 -- 172.21.15.39:0/2984112875 wait complete. 2024-05-24T11:36:45.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70f245700 1 Processor -- start 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70f245700 1 -- start start 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70f245700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7080ffd60 0x7fa708195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70f245700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa708100f50 0x7fa7081961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70f245700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7081018f0 0x7fa70819d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70f245700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70819d760 con 0x7fa7081018f0 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70f245700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70819d8d0 con 0x7fa7080ffd60 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70f245700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70819da90 con 0x7fa708100f50 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70d7e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7081018f0 0x7fa70819d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70d7e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7081018f0 0x7fa70819d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38792/0 (socket says 172.21.15.39:38792) 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa70d7e2700 1 -- 172.21.15.39:0/1867261235 learned_addr learned my addr 172.21.15.39:0/1867261235 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.450+0000 7fa707fff700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa708100f50 0x7fa7081961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa70cfe1700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7080ffd60 0x7fa708195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa70d7e2700 1 -- 172.21.15.39:0/1867261235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa708100f50 msgr2=0x7fa7081961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa70d7e2700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa708100f50 0x7fa7081961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa70d7e2700 1 -- 172.21.15.39:0/1867261235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7080ffd60 msgr2=0x7fa708195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa70d7e2700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7080ffd60 0x7fa708195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa70d7e2700 1 -- 172.21.15.39:0/1867261235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80095f0 con 0x7fa7081018f0 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa707fff700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa708100f50 0x7fa7081961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa70d7e2700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7081018f0 0x7fa70819d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa700007ee0 tx=0x7fa70000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa705ffb700 1 -- 172.21.15.39:0/1867261235 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700017070 con 0x7fa7081018f0 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa705ffb700 1 -- 172.21.15.39:0/1867261235 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa70000f040 con 0x7fa7081018f0 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70819dd70 con 0x7fa7081018f0 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa705ffb700 1 -- 172.21.15.39:0/1867261235 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700005710 con 0x7fa7081018f0 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.451+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70819e1a0 con 0x7fa7081018f0 2024-05-24T11:36:45.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.452+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7081072a0 con 0x7fa7081018f0 2024-05-24T11:36:45.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.453+0000 7fa705ffb700 1 -- 172.21.15.39:0/1867261235 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa700005870 con 0x7fa7081018f0 2024-05-24T11:36:45.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.454+0000 7fa705ffb700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa6f40a1120 0x7fa6f40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.454+0000 7fa705ffb700 1 -- 172.21.15.39:0/1867261235 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7000cbce0 con 0x7fa7081018f0 2024-05-24T11:36:45.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.454+0000 7fa70cfe1700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa6f40a1120 0x7fa6f40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.454+0000 7fa70cfe1700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa6f40a1120 0x7fa6f40a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009990 tx=0x7fa6f801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.458+0000 7fa705ffb700 1 -- 172.21.15.39:0/1867261235 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa700095870 con 0x7fa7081018f0 2024-05-24T11:36:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.676+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa708067c10 con 0x7fa7081018f0 2024-05-24T11:36:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.676+0000 7fa705ffb700 1 -- 172.21.15.39:0/1867261235 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa700098e90 con 0x7fa7081018f0 2024-05-24T11:36:45.681 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:45.681 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-05-24T11:36:45.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:36:45.681 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-05-24T11:36:45.681 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-05-24T11:36:45.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:36:45.681 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-05-24T11:36:45.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa6f40a1120 msgr2=0x7fa6f40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa6f40a1120 0x7fa6f40a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009990 tx=0x7fa6f801a040 comp rx=0 tx=0).stop 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7081018f0 msgr2=0x7fa70819d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7081018f0 0x7fa70819d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa700007ee0 tx=0x7fa70000d3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 shutdown_connections 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa6f40a1120 0x7fa6f40a35e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7081018f0 0x7fa70819d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7080ffd60 0x7fa708195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 --2- 172.21.15.39:0/1867261235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa708100f50 0x7fa7081961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 >> 172.21.15.39:0/1867261235 conn(0x7fa7080fb2f0 msgr2=0x7fa7081043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 shutdown_connections 2024-05-24T11:36:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.679+0000 7fa70f245700 1 -- 172.21.15.39:0/1867261235 wait complete. 2024-05-24T11:36:45.695 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-24T11:36:45.696 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-24T11:36:45.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.833+0000 7f8e0522a700 1 -- 172.21.15.39:0/3409680891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 msgr2=0x7f8e00106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.833+0000 7f8e0522a700 1 --2- 172.21.15.39:0/3409680891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 0x7f8e00106780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009990 tx=0x7f8df4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.833+0000 7f8e0522a700 1 -- 172.21.15.39:0/3409680891 shutdown_connections 2024-05-24T11:36:45.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.833+0000 7f8e0522a700 1 --2- 172.21.15.39:0/3409680891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e00106cc0 0x7f8e001090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.833+0000 7f8e0522a700 1 --2- 172.21.15.39:0/3409680891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101a60 0x7f8e00103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.833+0000 7f8e0522a700 1 --2- 172.21.15.39:0/3409680891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 0x7f8e00106780 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.833+0000 7f8e0522a700 1 -- 172.21.15.39:0/3409680891 >> 172.21.15.39:0/3409680891 conn(0x7f8e000fb3f0 msgr2=0x7f8e000fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:45.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.833+0000 7f8e0522a700 1 -- 172.21.15.39:0/3409680891 shutdown_connections 2024-05-24T11:36:45.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.833+0000 7f8e0522a700 1 -- 172.21.15.39:0/3409680891 wait complete. 2024-05-24T11:36:45.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.834+0000 7f8e0522a700 1 Processor -- start 2024-05-24T11:36:45.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.834+0000 7f8e0522a700 1 -- start start 2024-05-24T11:36:45.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8e0522a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101a60 0x7f8e00196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8e0522a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 0x7f8e00197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8e0522a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e00106cc0 0x7f8e0019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8e0522a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0019e980 con 0x7f8e00104390 2024-05-24T11:36:45.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8e0522a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0019eac0 con 0x7f8e00101a60 2024-05-24T11:36:45.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8e0522a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0019ec50 con 0x7f8e00106cc0 2024-05-24T11:36:45.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfe59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 0x7f8e00197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dff59e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e00106cc0 0x7f8e0019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfe59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 0x7f8e00197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38798/0 (socket says 172.21.15.39:38798) 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101a60 0x7f8e00196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfe59c700 1 -- 172.21.15.39:0/1535649082 learned_addr learned my addr 172.21.15.39:0/1535649082 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfe59c700 1 -- 172.21.15.39:0/1535649082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e00106cc0 msgr2=0x7f8e0019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfe59c700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e00106cc0 0x7f8e0019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfe59c700 1 -- 172.21.15.39:0/1535649082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101a60 msgr2=0x7f8e00196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfe59c700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101a60 0x7f8e00196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfe59c700 1 -- 172.21.15.39:0/1535649082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40095f0 con 0x7f8e00104390 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dff59e700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e00106cc0 0x7f8e0019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.835+0000 7f8dfe59c700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 0x7f8e00197390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006010 tx=0x7f8df4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:45.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.836+0000 7f8de7fff700 1 -- 172.21.15.39:0/1535649082 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df401d070 con 0x7f8e00104390 2024-05-24T11:36:45.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.836+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0019eed0 con 0x7f8e00104390 2024-05-24T11:36:45.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.836+0000 7f8de7fff700 1 -- 172.21.15.39:0/1535649082 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8df400bdb0 con 0x7f8e00104390 2024-05-24T11:36:45.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.836+0000 7f8de7fff700 1 -- 172.21.15.39:0/1535649082 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df400fda0 con 0x7f8e00104390 2024-05-24T11:36:45.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.836+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0019f2a0 con 0x7f8e00104390 2024-05-24T11:36:45.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.837+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e000fcf70 con 0x7f8e00104390 2024-05-24T11:36:45.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.838+0000 7f8de7fff700 1 -- 172.21.15.39:0/1535649082 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8df400f460 con 0x7f8e00104390 2024-05-24T11:36:45.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.838+0000 7f8de7fff700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8de80a1120 0x7f8de80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:45.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.839+0000 7f8dfed9d700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8de80a1120 0x7f8de80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:45.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.839+0000 7f8de7fff700 1 -- 172.21.15.39:0/1535649082 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8df40cdb00 con 0x7f8e00104390 2024-05-24T11:36:45.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.839+0000 7f8dfed9d700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8de80a1120 0x7f8de80a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009c60 tx=0x7f8df0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:45.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:45.846+0000 7f8de7fff700 1 -- 172.21.15.39:0/1535649082 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8df40977c0 con 0x7f8e00104390 2024-05-24T11:36:45.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:45 smithi053 ceph-mon[47448]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:45.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:45 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:45.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:45 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:45.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:45 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:45.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:45 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1654558432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:45.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:45 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:45.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:45 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2397897252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:36:45.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:45 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1867261235' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-24T11:36:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:45 smithi137 ceph-mon[47907]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:45 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:45 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:45 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:45 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1654558432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:45 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:45 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2397897252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:36:45.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:45 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1867261235' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-24T11:36:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:45 smithi039 ceph-mon[42919]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:45 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1654558432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:45 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:36:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:45 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2397897252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:36:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:45 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1867261235' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-24T11:36:46.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:46.076+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8e00068140 con 0x7f8e00104390 2024-05-24T11:36:47.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.063+0000 7f8de7fff700 1 -- 172.21.15.39:0/1535649082 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8df409ade0 con 0x7f8e00104390 2024-05-24T11:36:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.067+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8de80a1120 msgr2=0x7f8de80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.067+0000 7f8e0522a700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8de80a1120 0x7f8de80a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009c60 tx=0x7f8df0009260 comp rx=0 tx=0).stop 2024-05-24T11:36:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.068+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 msgr2=0x7f8e00197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.068+0000 7f8e0522a700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 0x7f8e00197390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006010 tx=0x7f8df4004730 comp rx=0 tx=0).stop 2024-05-24T11:36:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.068+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 shutdown_connections 2024-05-24T11:36:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.068+0000 7f8e0522a700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8de80a1120 0x7f8de80a35e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.068+0000 7f8e0522a700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e00106cc0 0x7f8e0019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.068+0000 7f8e0522a700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101a60 0x7f8e00196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.068+0000 7f8e0522a700 1 --2- 172.21.15.39:0/1535649082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e00104390 0x7f8e00197390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.068+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 >> 172.21.15.39:0/1535649082 conn(0x7f8e000fb3f0 msgr2=0x7f8e00103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:47.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.069+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 shutdown_connections 2024-05-24T11:36:47.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.069+0000 7f8e0522a700 1 -- 172.21.15.39:0/1535649082 wait complete. 2024-05-24T11:36:47.075 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-05-24T11:36:47.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-24T11:36:47.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.216+0000 7fb07cb2d700 1 -- 172.21.15.39:0/14436934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb078101170 msgr2=0x7fb078107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:47.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.216+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/14436934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb078101170 0x7fb078107a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb07000b0d0 tx=0x7fb07000b3e0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.216+0000 7fb07cb2d700 1 -- 172.21.15.39:0/14436934 shutdown_connections 2024-05-24T11:36:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.216+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/14436934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb078101170 0x7fb078107a20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.216+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/14436934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0780ff5e0 0x7fb0780ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.216+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/14436934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0781007d0 0x7fb078100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.216+0000 7fb07cb2d700 1 -- 172.21.15.39:0/14436934 >> 172.21.15.39:0/14436934 conn(0x7fb0780fab50 msgr2=0x7fb0780fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.216+0000 7fb07cb2d700 1 -- 172.21.15.39:0/14436934 shutdown_connections 2024-05-24T11:36:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.216+0000 7fb07cb2d700 1 -- 172.21.15.39:0/14436934 wait complete. 2024-05-24T11:36:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07cb2d700 1 Processor -- start 2024-05-24T11:36:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07cb2d700 1 -- start start 2024-05-24T11:36:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07cb2d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0780ff5e0 0x7fb0780fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07cb2d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0781007d0 0x7fb07819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07cb2d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb078101170 0x7fb0781a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07cb2d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0781a0c30 con 0x7fb0780ff5e0 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07cb2d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0781a0da0 con 0x7fb078101170 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07cb2d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0781a0f60 con 0x7fb0781007d0 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0780ff5e0 0x7fb0780fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0780ff5e0 0x7fb0780fccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38810/0 (socket says 172.21.15.39:38810) 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb07659c700 1 -- 172.21.15.39:0/1311092911 learned_addr learned my addr 172.21.15.39:0/1311092911 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb076d9d700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb078101170 0x7fb0781a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.217+0000 7fb06dd9b700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0781007d0 0x7fb07819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:47.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb07659c700 1 -- 172.21.15.39:0/1311092911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0781007d0 msgr2=0x7fb07819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb07659c700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0781007d0 0x7fb07819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb07659c700 1 -- 172.21.15.39:0/1311092911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb078101170 msgr2=0x7fb0781a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb07659c700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb078101170 0x7fb0781a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb07659c700 1 -- 172.21.15.39:0/1311092911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb070009d00 con 0x7fb0780ff5e0 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb07659c700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0780ff5e0 0x7fb0780fccf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d980 tx=0x7fb06800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb06ffff700 1 -- 172.21.15.39:0/1311092911 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068004020 con 0x7fb0780ff5e0 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0781a1240 con 0x7fb0780ff5e0 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb06ffff700 1 -- 172.21.15.39:0/1311092911 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb06800f690 con 0x7fb0780ff5e0 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb06ffff700 1 -- 172.21.15.39:0/1311092911 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0680039e0 con 0x7fb0780ff5e0 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.218+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0781a17a0 con 0x7fb0780ff5e0 2024-05-24T11:36:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.219+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb078107230 con 0x7fb0780ff5e0 2024-05-24T11:36:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.221+0000 7fb06ffff700 1 -- 172.21.15.39:0/1311092911 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb06800f800 con 0x7fb0780ff5e0 2024-05-24T11:36:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.221+0000 7fb06ffff700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb0640a1120 0x7fb0640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.222+0000 7fb06dd9b700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb0640a1120 0x7fb0640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.222+0000 7fb06ffff700 1 -- 172.21.15.39:0/1311092911 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fb0680cc590 con 0x7fb0780ff5e0 2024-05-24T11:36:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.222+0000 7fb06dd9b700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb0640a1120 0x7fb0640a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0780fd2b0 tx=0x7fb06000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:47.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.225+0000 7fb06ffff700 1 -- 172.21.15.39:0/1311092911 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb068095fb0 con 0x7fb0780ff5e0 2024-05-24T11:36:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1535649082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-24T11:36:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:47.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:47.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:47 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:47.434+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb07810aee0 con 0x7fb0780ff5e0 2024-05-24T11:36:47.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:47.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1535649082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-24T11:36:47.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:47 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:47.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1535649082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-24T11:36:47.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:36:47.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:36:47.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:47.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:47 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:36:48.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.076+0000 7fb06ffff700 1 -- 172.21.15.39:0/1311092911 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb0680995d0 con 0x7fb0780ff5e0 2024-05-24T11:36:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.081+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb0640a1120 msgr2=0x7fb0640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.081+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb0640a1120 0x7fb0640a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0780fd2b0 tx=0x7fb06000b410 comp rx=0 tx=0).stop 2024-05-24T11:36:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.081+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0780ff5e0 msgr2=0x7fb0780fccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.081+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0780ff5e0 0x7fb0780fccf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d980 tx=0x7fb06800dc90 comp rx=0 tx=0).stop 2024-05-24T11:36:48.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.084+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 shutdown_connections 2024-05-24T11:36:48.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.084+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb0640a1120 0x7fb0640a35e0 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0780fd2b0 tx=0x7fb06000b410 comp rx=0 tx=0).stop 2024-05-24T11:36:48.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.084+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb078101170 0x7fb0781a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.084+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0780ff5e0 0x7fb0780fccf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.084+0000 7fb07cb2d700 1 --2- 172.21.15.39:0/1311092911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0781007d0 0x7fb07819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.084+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 >> 172.21.15.39:0/1311092911 conn(0x7fb0780fab50 msgr2=0x7fb078101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.089+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 shutdown_connections 2024-05-24T11:36:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.090+0000 7fb07cb2d700 1 -- 172.21.15.39:0/1311092911 wait complete. 2024-05-24T11:36:48.098 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-05-24T11:36:48.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-24T11:36:48.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.233+0000 7f187c864700 1 -- 172.21.15.39:0/4109508689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18780ffe60 msgr2=0x7f1878100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.233+0000 7f187c864700 1 --2- 172.21.15.39:0/4109508689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18780ffe60 0x7f1878100280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1868009990 tx=0x7f1868009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.234+0000 7f187c864700 1 -- 172.21.15.39:0/4109508689 shutdown_connections 2024-05-24T11:36:48.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.234+0000 7f187c864700 1 --2- 172.21.15.39:0/4109508689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1878101970 0x7f1878108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.234+0000 7f187c864700 1 --2- 172.21.15.39:0/4109508689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18780ffe60 0x7f1878100280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.234+0000 7f187c864700 1 --2- 172.21.15.39:0/4109508689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1878100fb0 0x7f1878101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.234+0000 7f187c864700 1 -- 172.21.15.39:0/4109508689 >> 172.21.15.39:0/4109508689 conn(0x7f18780fb3f0 msgr2=0x7f18780fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:48.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.234+0000 7f187c864700 1 -- 172.21.15.39:0/4109508689 shutdown_connections 2024-05-24T11:36:48.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.234+0000 7f187c864700 1 -- 172.21.15.39:0/4109508689 wait complete. 2024-05-24T11:36:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187c864700 1 Processor -- start 2024-05-24T11:36:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187c864700 1 -- start start 2024-05-24T11:36:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187c864700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18780ffe60 0x7f1878195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:48.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187c864700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1878100fb0 0x7f18781961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187c864700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1878101970 0x7f187819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187c864700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187819d7a0 con 0x7f1878101970 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187c864700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187819d910 con 0x7f18780ffe60 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187c864700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187819dad0 con 0x7f1878100fb0 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f1876d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1878101970 0x7f187819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f1876d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1878101970 0x7f187819d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54162/0 (socket says 172.21.15.39:54162) 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f1876d9d700 1 -- 172.21.15.39:0/1072444052 learned_addr learned my addr 172.21.15.39:0/1072444052 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187659c700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18780ffe60 0x7f1878195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f1875d9b700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1878100fb0 0x7f18781961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f1876d9d700 1 -- 172.21.15.39:0/1072444052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1878100fb0 msgr2=0x7f18781961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f1876d9d700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1878100fb0 0x7f18781961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f1876d9d700 1 -- 172.21.15.39:0/1072444052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18780ffe60 msgr2=0x7f1878195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f1876d9d700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18780ffe60 0x7f1878195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f1876d9d700 1 -- 172.21.15.39:0/1072444052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18680095f0 con 0x7f1878101970 2024-05-24T11:36:48.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.235+0000 7f187659c700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18780ffe60 0x7f1878195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:48.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.236+0000 7f1876d9d700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1878101970 0x7f187819d1d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f187000b970 tx=0x7f187000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:48.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.236+0000 7f18677fe700 1 -- 172.21.15.39:0/1072444052 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187000c9b0 con 0x7f1878101970 2024-05-24T11:36:48.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.236+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187819ddb0 con 0x7f1878101970 2024-05-24T11:36:48.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.236+0000 7f18677fe700 1 -- 172.21.15.39:0/1072444052 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1870014440 con 0x7f1878101970 2024-05-24T11:36:48.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.236+0000 7f18677fe700 1 -- 172.21.15.39:0/1072444052 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18700075d0 con 0x7f1878101970 2024-05-24T11:36:48.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.236+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187819e1b0 con 0x7f1878101970 2024-05-24T11:36:48.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.237+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18781072d0 con 0x7f1878101970 2024-05-24T11:36:48.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.238+0000 7f18677fe700 1 -- 172.21.15.39:0/1072444052 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f18700145b0 con 0x7f1878101970 2024-05-24T11:36:48.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.239+0000 7f18677fe700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f18600a1120 0x7f18600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:48.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.239+0000 7f187659c700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f18600a1120 0x7f18600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:48.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.239+0000 7f18677fe700 1 -- 172.21.15.39:0/1072444052 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f1870006e40 con 0x7f1878101970 2024-05-24T11:36:48.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.239+0000 7f187659c700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f18600a1120 0x7f18600a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f186800c010 tx=0x7f186800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:48.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.242+0000 7f18677fe700 1 -- 172.21.15.39:0/1072444052 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1870096080 con 0x7f1878101970 2024-05-24T11:36:48.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:48 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1535649082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-24T11:36:48.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:48 smithi039 ceph-mon[42919]: osdmap e74: 12 total, 12 up, 12 in 2024-05-24T11:36:48.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:48 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1311092911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-24T11:36:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:48 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1535649082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-24T11:36:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:48 smithi053 ceph-mon[47448]: osdmap e74: 12 total, 12 up, 12 in 2024-05-24T11:36:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:48 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1311092911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-24T11:36:48.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:48 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1535649082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-24T11:36:48.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:48 smithi137 ceph-mon[47907]: osdmap e74: 12 total, 12 up, 12 in 2024-05-24T11:36:48.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:48 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1311092911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-24T11:36:48.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.498+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f187810b590 con 0x7f1878101970 2024-05-24T11:36:48.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.507+0000 7f18677fe700 1 -- 172.21.15.39:0/1072444052 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f18700190c0 con 0x7f1878101970 2024-05-24T11:36:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.517+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f18600a1120 msgr2=0x7f18600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.517+0000 7f187c864700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f18600a1120 0x7f18600a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f186800c010 tx=0x7f186800b540 comp rx=0 tx=0).stop 2024-05-24T11:36:48.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.517+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1878101970 msgr2=0x7f187819d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.517+0000 7f187c864700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1878101970 0x7f187819d1d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f187000b970 tx=0x7f187000bd30 comp rx=0 tx=0).stop 2024-05-24T11:36:48.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.519+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 shutdown_connections 2024-05-24T11:36:48.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.519+0000 7f187c864700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f18600a1120 0x7f18600a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.519+0000 7f187c864700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1878101970 0x7f187819d1d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.519+0000 7f187c864700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18780ffe60 0x7f1878195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.519+0000 7f187c864700 1 --2- 172.21.15.39:0/1072444052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1878100fb0 0x7f18781961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.519+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 >> 172.21.15.39:0/1072444052 conn(0x7f18780fb3f0 msgr2=0x7f1878104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.519+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 shutdown_connections 2024-05-24T11:36:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.519+0000 7f187c864700 1 -- 172.21.15.39:0/1072444052 wait complete. 2024-05-24T11:36:48.526 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 2 and data pool 3 2024-05-24T11:36:48.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:36:48.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 -- 172.21.15.39:0/2227042221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d781029a0 msgr2=0x7f2d781092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/2227042221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d781029a0 0x7f2d781092c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b0a0 tx=0x7f2d7000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 -- 172.21.15.39:0/2227042221 shutdown_connections 2024-05-24T11:36:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/2227042221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d78100e90 0x7f2d781012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/2227042221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d781029a0 0x7f2d781092c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/2227042221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d78101fe0 0x7f2d78102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 -- 172.21.15.39:0/2227042221 >> 172.21.15.39:0/2227042221 conn(0x7f2d780fc400 msgr2=0x7f2d780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 -- 172.21.15.39:0/2227042221 shutdown_connections 2024-05-24T11:36:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 -- 172.21.15.39:0/2227042221 wait complete. 2024-05-24T11:36:48.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.666+0000 7f2d7f25b700 1 Processor -- start 2024-05-24T11:36:48.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7f25b700 1 -- start start 2024-05-24T11:36:48.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7f25b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d78100e90 0x7f2d78196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:48.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7f25b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d78101fe0 0x7f2d78197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7f25b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d781029a0 0x7f2d7819e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7f25b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7819e930 con 0x7f2d78100e90 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7f25b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7819ea70 con 0x7f2d78101fe0 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7f25b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7819ec00 con 0x7f2d781029a0 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7cff7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d78100e90 0x7f2d78196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7cff7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d78100e90 0x7f2d78196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54188/0 (socket says 172.21.15.39:54188) 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7cff7700 1 -- 172.21.15.39:0/3546376273 learned_addr learned my addr 172.21.15.39:0/3546376273 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d77fff700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d78101fe0 0x7f2d78197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7d7f8700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d781029a0 0x7f2d7819e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7cff7700 1 -- 172.21.15.39:0/3546376273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d781029a0 msgr2=0x7f2d7819e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7cff7700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d781029a0 0x7f2d7819e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7cff7700 1 -- 172.21.15.39:0/3546376273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d78101fe0 msgr2=0x7f2d78197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7cff7700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d78101fe0 0x7f2d78197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7cff7700 1 -- 172.21.15.39:0/3546376273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70009d00 con 0x7f2d78100e90 2024-05-24T11:36:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7d7f8700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d781029a0 0x7f2d7819e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:36:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.667+0000 7f2d7cff7700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d78100e90 0x7f2d78196e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00d950 tx=0x7f2d6c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.668+0000 7f2d75ffb700 1 -- 172.21.15.39:0/3546376273 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c009940 con 0x7f2d78100e90 2024-05-24T11:36:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.668+0000 7f2d75ffb700 1 -- 172.21.15.39:0/3546376273 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d6c010460 con 0x7f2d78100e90 2024-05-24T11:36:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.668+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7819eee0 con 0x7f2d78100e90 2024-05-24T11:36:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.668+0000 7f2d75ffb700 1 -- 172.21.15.39:0/3546376273 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c00f620 con 0x7f2d78100e90 2024-05-24T11:36:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.668+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7819f300 con 0x7f2d78100e90 2024-05-24T11:36:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.669+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d78067c80 con 0x7f2d78100e90 2024-05-24T11:36:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.670+0000 7f2d75ffb700 1 -- 172.21.15.39:0/3546376273 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2d6c0105d0 con 0x7f2d78100e90 2024-05-24T11:36:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.671+0000 7f2d75ffb700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2d680a1120 0x7f2d680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.671+0000 7f2d77fff700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2d680a1120 0x7f2d680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.671+0000 7f2d75ffb700 1 -- 172.21.15.39:0/3546376273 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2d6c0cc650 con 0x7f2d78100e90 2024-05-24T11:36:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.671+0000 7f2d77fff700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2d680a1120 0x7f2d680a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005950 tx=0x7f2d6400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:48.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.675+0000 7f2d75ffb700 1 -- 172.21.15.39:0/3546376273 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2d78067c80 con 0x7f2d78100e90 2024-05-24T11:36:48.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.889+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d78108300 con 0x7f2d78100e90 2024-05-24T11:36:48.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.891+0000 7f2d75ffb700 1 -- 172.21.15.39:0/3546376273 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+22074 (secure 0 0 0) 0x7f2d6c095e90 con 0x7f2d78100e90 2024-05-24T11:36:48.896 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:48.897 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":76,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:48.500311+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:36:48.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2d680a1120 msgr2=0x7f2d680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2d680a1120 0x7f2d680a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005950 tx=0x7f2d6400b410 comp rx=0 tx=0).stop 2024-05-24T11:36:48.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d78100e90 msgr2=0x7f2d78196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:48.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d78100e90 0x7f2d78196e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00d950 tx=0x7f2d6c00dc60 comp rx=0 tx=0).stop 2024-05-24T11:36:48.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 shutdown_connections 2024-05-24T11:36:48.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2d680a1120 0x7f2d680a35e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d781029a0 0x7f2d7819e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d78100e90 0x7f2d78196e00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 --2- 172.21.15.39:0/3546376273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d78101fe0 0x7f2d78197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:48.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 >> 172.21.15.39:0/3546376273 conn(0x7f2d780fc400 msgr2=0x7f2d78105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:48.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 shutdown_connections 2024-05-24T11:36:48.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:48.893+0000 7f2d7f25b700 1 -- 172.21.15.39:0/3546376273 wait complete. 2024-05-24T11:36:48.911 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:36:49.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.033+0000 7f087563d700 1 -- 172.21.15.39:0/2040778044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 msgr2=0x7f08701081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:49.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.033+0000 7f087563d700 1 --2- 172.21.15.39:0/2040778044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 0x7f08701081e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f086800b0d0 tx=0x7f086800b3e0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.033+0000 7f087563d700 1 -- 172.21.15.39:0/2040778044 shutdown_connections 2024-05-24T11:36:49.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.033+0000 7f087563d700 1 --2- 172.21.15.39:0/2040778044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08700ffda0 0x7f08701001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.033+0000 7f087563d700 1 --2- 172.21.15.39:0/2040778044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 0x7f08701081e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.033+0000 7f087563d700 1 --2- 172.21.15.39:0/2040778044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0870100f90 0x7f08701013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 -- 172.21.15.39:0/2040778044 >> 172.21.15.39:0/2040778044 conn(0x7f08700fb330 msgr2=0x7f08700fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 -- 172.21.15.39:0/2040778044 shutdown_connections 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 -- 172.21.15.39:0/2040778044 wait complete. 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 Processor -- start 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 -- start start 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08700ffda0 0x7f0870103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0870100f90 0x7f0870102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 0x7f08701028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0870103050 con 0x7f0870101930 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08701031c0 con 0x7f0870100f90 2024-05-24T11:36:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.034+0000 7f087563d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08701a0e40 con 0x7f08700ffda0 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 0x7f08701028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 0x7f08701028b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54218/0 (socket says 172.21.15.39:54218) 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086f7fe700 1 -- 172.21.15.39:0/4051647065 learned_addr learned my addr 172.21.15.39:0/4051647065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086e7fc700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0870100f90 0x7f0870102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086effd700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08700ffda0 0x7f0870103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086f7fe700 1 -- 172.21.15.39:0/4051647065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08700ffda0 msgr2=0x7f0870103ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086f7fe700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08700ffda0 0x7f0870103ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086f7fe700 1 -- 172.21.15.39:0/4051647065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0870100f90 msgr2=0x7f0870102370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086f7fe700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0870100f90 0x7f0870102370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086f7fe700 1 -- 172.21.15.39:0/4051647065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868009d00 con 0x7f0870101930 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086e7fc700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0870100f90 0x7f0870102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:36:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f086f7fe700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 0x7f08701028b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f08680079c0 tx=0x7f086800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f0857fff700 1 -- 172.21.15.39:0/4051647065 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086800e040 con 0x7f0870101930 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08701a1070 con 0x7f0870101930 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f0857fff700 1 -- 172.21.15.39:0/4051647065 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0868004e60 con 0x7f0870101930 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f0857fff700 1 -- 172.21.15.39:0/4051647065 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086801b6b0 con 0x7f0870101930 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.035+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08701a1490 con 0x7f0870101930 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.036+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0870068060 con 0x7f0870101930 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.037+0000 7f0857fff700 1 -- 172.21.15.39:0/4051647065 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f086801d480 con 0x7f0870101930 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.038+0000 7f0857fff700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08580a1120 0x7f08580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.038+0000 7f086effd700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08580a1120 0x7f08580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.038+0000 7f0857fff700 1 -- 172.21.15.39:0/4051647065 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f08680ce160 con 0x7f0870101930 2024-05-24T11:36:49.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.038+0000 7f086effd700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08580a1120 0x7f08580a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0870100df0 tx=0x7f086000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:49.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.042+0000 7f0857fff700 1 -- 172.21.15.39:0/4051647065 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0868097ad0 con 0x7f0870101930 2024-05-24T11:36:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.258+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f087010b600 con 0x7f0870101930 2024-05-24T11:36:49.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.260+0000 7f0857fff700 1 -- 172.21.15.39:0/4051647065 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+22074 (secure 0 0 0) 0x7f086809b0f0 con 0x7f0870101930 2024-05-24T11:36:49.265 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:49.266 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":76,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:48.500311+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08580a1120 msgr2=0x7f08580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08580a1120 0x7f08580a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0870100df0 tx=0x7f086000b410 comp rx=0 tx=0).stop 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 msgr2=0x7f08701028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 0x7f08701028b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f08680079c0 tx=0x7f086800ccc0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 shutdown_connections 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f08580a1120 0x7f08580a35e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08700ffda0 0x7f0870103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0870101930 0x7f08701028b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 --2- 172.21.15.39:0/4051647065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0870100f90 0x7f0870102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 >> 172.21.15.39:0/4051647065 conn(0x7f08700fb330 msgr2=0x7f08700fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 shutdown_connections 2024-05-24T11:36:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.262+0000 7f087563d700 1 -- 172.21.15.39:0/4051647065 wait complete. 2024-05-24T11:36:49.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: pgmap v217: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:49.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-24T11:36:49.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1311092911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-24T11:36:49.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: osdmap e75: 12 total, 12 up, 12 in 2024-05-24T11:36:49.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1072444052' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-24T11:36:49.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-24T11:36:49.280 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: osdmap e76: 12 total, 12 up, 12 in 2024-05-24T11:36:49.280 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1072444052' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-24T11:36:49.280 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-24T11:36:49.280 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-24T11:36:49.280 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:49 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3546376273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:49.281 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-24T11:36:49.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3743334987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 msgr2=0x7fe0e40ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:49.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3743334987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 0x7fe0e40ff640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009990 tx=0x7fe0e0009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3743334987 shutdown_connections 2024-05-24T11:36:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3743334987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e4102b90 0x7fe0e4109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3743334987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 0x7fe0e40ff640 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3743334987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0e41021f0 0x7fe0e4102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3743334987 >> 172.21.15.39:0/3743334987 conn(0x7fe0e40fab50 msgr2=0x7fe0e40fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3743334987 shutdown_connections 2024-05-24T11:36:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3743334987 wait complete. 2024-05-24T11:36:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 Processor -- start 2024-05-24T11:36:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 -- start start 2024-05-24T11:36:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 0x7fe0e4101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.406+0000 7fe0ec6f7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e41021f0 0x7fe0e41001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ec6f7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0e4102b90 0x7fe0e41006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ec6f7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e4100e80 con 0x7fe0e40ff220 2024-05-24T11:36:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ec6f7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e4100ff0 con 0x7fe0e41021f0 2024-05-24T11:36:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ec6f7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e41a0e60 con 0x7fe0e4102b90 2024-05-24T11:36:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ea493700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 0x7fe0e4101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ea493700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 0x7fe0e4101b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54236/0 (socket says 172.21.15.39:54236) 2024-05-24T11:36:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ea493700 1 -- 172.21.15.39:0/3059874023 learned_addr learned my addr 172.21.15.39:0/3059874023 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0eac94700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0e4102b90 0x7fe0e41006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0e9c92700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e41021f0 0x7fe0e41001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ea493700 1 -- 172.21.15.39:0/3059874023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0e4102b90 msgr2=0x7fe0e41006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ea493700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0e4102b90 0x7fe0e41006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ea493700 1 -- 172.21.15.39:0/3059874023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e41021f0 msgr2=0x7fe0e41001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ea493700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e41021f0 0x7fe0e41001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ea493700 1 -- 172.21.15.39:0/3059874023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e00095f0 con 0x7fe0e40ff220 2024-05-24T11:36:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0eac94700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0e4102b90 0x7fe0e41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0e9c92700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e41021f0 0x7fe0e41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:36:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ea493700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 0x7fe0e4101b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c010 tx=0x7fe0e0003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0db7fe700 1 -- 172.21.15.39:0/3059874023 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e001d070 con 0x7fe0e40ff220 2024-05-24T11:36:49.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0db7fe700 1 -- 172.21.15.39:0/3059874023 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0e000bd70 con 0x7fe0e40ff220 2024-05-24T11:36:49.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0db7fe700 1 -- 172.21.15.39:0/3059874023 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e000f460 con 0x7fe0e40ff220 2024-05-24T11:36:49.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e41a1090 con 0x7fe0e40ff220 2024-05-24T11:36:49.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.407+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e41a1400 con 0x7fe0e40ff220 2024-05-24T11:36:49.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.408+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e4108470 con 0x7fe0e40ff220 2024-05-24T11:36:49.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.409+0000 7fe0db7fe700 1 -- 172.21.15.39:0/3059874023 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe0e0022470 con 0x7fe0e40ff220 2024-05-24T11:36:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.410+0000 7fe0db7fe700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0d00a1120 0x7fe0d00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.410+0000 7fe0db7fe700 1 -- 172.21.15.39:0/3059874023 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe0e00ce200 con 0x7fe0e40ff220 2024-05-24T11:36:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.410+0000 7fe0e9c92700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0d00a1120 0x7fe0d00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.410+0000 7fe0e9c92700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0d00a1120 0x7fe0d00a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4005fd0 tx=0x7fe0d4005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.413+0000 7fe0db7fe700 1 -- 172.21.15.39:0/3059874023 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe0e0097b70 con 0x7fe0e40ff220 2024-05-24T11:36:49.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: pgmap v217: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:49.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-24T11:36:49.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1311092911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-24T11:36:49.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: osdmap e75: 12 total, 12 up, 12 in 2024-05-24T11:36:49.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1072444052' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-24T11:36:49.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-24T11:36:49.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: osdmap e76: 12 total, 12 up, 12 in 2024-05-24T11:36:49.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1072444052' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-24T11:36:49.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-24T11:36:49.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-24T11:36:49.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:49 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3546376273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:49.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: pgmap v217: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:36:49.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-24T11:36:49.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1311092911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-24T11:36:49.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: osdmap e75: 12 total, 12 up, 12 in 2024-05-24T11:36:49.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1072444052' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-24T11:36:49.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-24T11:36:49.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: osdmap e76: 12 total, 12 up, 12 in 2024-05-24T11:36:49.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1072444052' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-24T11:36:49.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-24T11:36:49.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-24T11:36:49.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:49 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3546376273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:49.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:49.675+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe0e4067460 con 0x7fe0e40ff220 2024-05-24T11:36:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:50 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4051647065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:50 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] up:active 2024-05-24T11:36:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:50 smithi053 ceph-mon[47448]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:50 smithi053 ceph-mon[47448]: osdmap e77: 12 total, 12 up, 12 in 2024-05-24T11:36:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:50 smithi053 ceph-mon[47448]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:50 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3059874023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-24T11:36:50.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:50 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4051647065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:50.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:50 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] up:active 2024-05-24T11:36:50.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:50 smithi137 ceph-mon[47907]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:50.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:50 smithi137 ceph-mon[47907]: osdmap e77: 12 total, 12 up, 12 in 2024-05-24T11:36:50.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:50 smithi137 ceph-mon[47907]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:50.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:50 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3059874023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-24T11:36:50.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:50 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4051647065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:50.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:50 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] up:active 2024-05-24T11:36:50.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:50 smithi039 ceph-mon[42919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:50.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:50 smithi039 ceph-mon[42919]: osdmap e77: 12 total, 12 up, 12 in 2024-05-24T11:36:50.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:50 smithi039 ceph-mon[42919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:50.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:50 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3059874023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-24T11:36:50.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.582+0000 7fe0db7fe700 1 -- 172.21.15.39:0/3059874023 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe0e0022720 con 0x7fe0e40ff220 2024-05-24T11:36:50.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0d00a1120 msgr2=0x7fe0d00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:50.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0d00a1120 0x7fe0d00a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4005fd0 tx=0x7fe0d4005dc0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 msgr2=0x7fe0e4101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 0x7fe0e4101b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c010 tx=0x7fe0e0003b00 comp rx=0 tx=0).stop 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 shutdown_connections 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe0d00a1120 0x7fe0d00a35e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0e4102b90 0x7fe0e41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e40ff220 0x7fe0e4101b10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 --2- 172.21.15.39:0/3059874023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0e41021f0 0x7fe0e41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 >> 172.21.15.39:0/3059874023 conn(0x7fe0e40fab50 msgr2=0x7fe0e41055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 shutdown_connections 2024-05-24T11:36:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.585+0000 7fe0ec6f7700 1 -- 172.21.15.39:0/3059874023 wait complete. 2024-05-24T11:36:50.605 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-24T11:36:50.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-24T11:36:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.736+0000 7f2664059700 1 -- 172.21.15.39:0/4167416601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c100fb0 msgr2=0x7f265c101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.736+0000 7f2664059700 1 --2- 172.21.15.39:0/4167416601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c100fb0 0x7f265c101430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2658009990 tx=0x7f2658009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.736+0000 7f2664059700 1 -- 172.21.15.39:0/4167416601 shutdown_connections 2024-05-24T11:36:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.736+0000 7f2664059700 1 --2- 172.21.15.39:0/4167416601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c0ffe60 0x7f265c100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.736+0000 7f2664059700 1 --2- 172.21.15.39:0/4167416601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f265c101970 0x7f265c108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.736+0000 7f2664059700 1 --2- 172.21.15.39:0/4167416601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c100fb0 0x7f265c101430 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.736+0000 7f2664059700 1 -- 172.21.15.39:0/4167416601 >> 172.21.15.39:0/4167416601 conn(0x7f265c0fb3f0 msgr2=0x7f265c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.736+0000 7f2664059700 1 -- 172.21.15.39:0/4167416601 shutdown_connections 2024-05-24T11:36:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.736+0000 7f2664059700 1 -- 172.21.15.39:0/4167416601 wait complete. 2024-05-24T11:36:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f2664059700 1 Processor -- start 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f2664059700 1 -- start start 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f2664059700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f265c0ffe60 0x7f265c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f2664059700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c100fb0 0x7f265c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f2664059700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c101970 0x7f265c19d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f2664059700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c19d850 con 0x7f265c101970 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f2664059700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c19d9c0 con 0x7f265c0ffe60 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f2664059700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c19db80 con 0x7f265c100fb0 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f26625f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c101970 0x7f265c19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f26625f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c101970 0x7f265c19d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54254/0 (socket says 172.21.15.39:54254) 2024-05-24T11:36:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f26625f6700 1 -- 172.21.15.39:0/3166878369 learned_addr learned my addr 172.21.15.39:0/3166878369 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f26615f4700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c100fb0 0x7f265c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.737+0000 7f2661df5700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f265c0ffe60 0x7f265c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f26625f6700 1 -- 172.21.15.39:0/3166878369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c100fb0 msgr2=0x7f265c1962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f26625f6700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c100fb0 0x7f265c1962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f26625f6700 1 -- 172.21.15.39:0/3166878369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f265c0ffe60 msgr2=0x7f265c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f26625f6700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f265c0ffe60 0x7f265c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f26625f6700 1 -- 172.21.15.39:0/3166878369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26580095f0 con 0x7f265c101970 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f26625f6700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c101970 0x7f265c19d280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2654007ee0 tx=0x7f265400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f2652ffd700 1 -- 172.21.15.39:0/3166878369 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265400dd20 con 0x7f265c101970 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f2652ffd700 1 -- 172.21.15.39:0/3166878369 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f265400f040 con 0x7f265c101970 2024-05-24T11:36:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c19de60 con 0x7f265c101970 2024-05-24T11:36:50.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f2652ffd700 1 -- 172.21.15.39:0/3166878369 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654004750 con 0x7f265c101970 2024-05-24T11:36:50.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.738+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c19e280 con 0x7f265c101970 2024-05-24T11:36:50.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.739+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265c107320 con 0x7f265c101970 2024-05-24T11:36:50.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.741+0000 7f2652ffd700 1 -- 172.21.15.39:0/3166878369 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2654003a00 con 0x7f265c101970 2024-05-24T11:36:50.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.741+0000 7f2652ffd700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f26480a1120 0x7f26480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:50.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.741+0000 7f2652ffd700 1 -- 172.21.15.39:0/3166878369 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f26540cc770 con 0x7f265c101970 2024-05-24T11:36:50.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.741+0000 7f2661df5700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f26480a1120 0x7f26480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:50.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.742+0000 7f2661df5700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f26480a1120 0x7f26480a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f265c1971e0 tx=0x7f264c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:50.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:50.744+0000 7f2652ffd700 1 -- 172.21.15.39:0/3166878369 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2654096050 con 0x7f265c101970 2024-05-24T11:36:51.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.000+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f265c10b670 con 0x7f265c101970 2024-05-24T11:36:51.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:51 smithi053 ceph-mon[47448]: pgmap v221: 97 pgs: 11 active+clean, 11 creating+peering, 75 unknown; 578 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-05-24T11:36:51.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:51 smithi053 ceph-mon[47448]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-24T11:36:51.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:51 smithi053 ceph-mon[47448]: Cluster is now healthy 2024-05-24T11:36:51.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:51 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3059874023' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-24T11:36:51.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:51 smithi053 ceph-mon[47448]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:51.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:51 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3166878369' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-24T11:36:51.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:51 smithi137 ceph-mon[47907]: pgmap v221: 97 pgs: 11 active+clean, 11 creating+peering, 75 unknown; 578 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-05-24T11:36:51.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:51 smithi137 ceph-mon[47907]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-24T11:36:51.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:51 smithi137 ceph-mon[47907]: Cluster is now healthy 2024-05-24T11:36:51.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:51 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3059874023' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-24T11:36:51.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:51 smithi137 ceph-mon[47907]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:51.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:51 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3166878369' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-24T11:36:51.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:51 smithi039 ceph-mon[42919]: pgmap v221: 97 pgs: 11 active+clean, 11 creating+peering, 75 unknown; 578 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-05-24T11:36:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:51 smithi039 ceph-mon[42919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-24T11:36:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:51 smithi039 ceph-mon[42919]: Cluster is now healthy 2024-05-24T11:36:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:51 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3059874023' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-24T11:36:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:51 smithi039 ceph-mon[42919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:51 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3166878369' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-24T11:36:51.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.584+0000 7f2652ffd700 1 -- 172.21.15.39:0/3166878369 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f26540230b0 con 0x7f265c101970 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f26480a1120 msgr2=0x7f26480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f26480a1120 0x7f26480a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f265c1971e0 tx=0x7f264c009260 comp rx=0 tx=0).stop 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c101970 msgr2=0x7f265c19d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c101970 0x7f265c19d280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2654007ee0 tx=0x7f265400d3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 shutdown_connections 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f26480a1120 0x7f26480a35e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f265c101970 0x7f265c19d280 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f265c0ffe60 0x7f265c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 --2- 172.21.15.39:0/3166878369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c100fb0 0x7f265c1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 >> 172.21.15.39:0/3166878369 conn(0x7f265c0fb3f0 msgr2=0x7f265c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:51.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 shutdown_connections 2024-05-24T11:36:51.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.587+0000 7f2664059700 1 -- 172.21.15.39:0/3166878369 wait complete. 2024-05-24T11:36:51.611 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-24T11:36:51.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 -- 172.21.15.39:0/1665939451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 msgr2=0x7fb484103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/1665939451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 0x7fb484103e30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb474009990 tx=0x7fb474009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 -- 172.21.15.39:0/1665939451 shutdown_connections 2024-05-24T11:36:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/1665939451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb484106ca0 0x7fb4841090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/1665939451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 0x7fb484103e30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/1665939451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb484104370 0x7fb484106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 -- 172.21.15.39:0/1665939451 >> 172.21.15.39:0/1665939451 conn(0x7fb4840fb3f0 msgr2=0x7fb4840fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 -- 172.21.15.39:0/1665939451 shutdown_connections 2024-05-24T11:36:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 -- 172.21.15.39:0/1665939451 wait complete. 2024-05-24T11:36:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.742+0000 7fb48a4b2700 1 Processor -- start 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb48a4b2700 1 -- start start 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb48a4b2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 0x7fb484101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb48a4b2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb484104370 0x7fb4840ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb48a4b2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb484106ca0 0x7fb4840fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb48a4b2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4841006d0 con 0x7fb484101a40 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb48a4b2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb484100840 con 0x7fb484106ca0 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb48a4b2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4841a0e10 con 0x7fb484104370 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb483fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 0x7fb484101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb483fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 0x7fb484101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54268/0 (socket says 172.21.15.39:54268) 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb483fff700 1 -- 172.21.15.39:0/159383123 learned_addr learned my addr 172.21.15.39:0/159383123 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb488a4f700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb484106ca0 0x7fb4840fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb4837fe700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb484104370 0x7fb4840ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb483fff700 1 -- 172.21.15.39:0/159383123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb484104370 msgr2=0x7fb4840ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb483fff700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb484104370 0x7fb4840ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb483fff700 1 -- 172.21.15.39:0/159383123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb484106ca0 msgr2=0x7fb4840fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb483fff700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb484106ca0 0x7fb4840fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.743+0000 7fb483fff700 1 -- 172.21.15.39:0/159383123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4740095f0 con 0x7fb484101a40 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.744+0000 7fb483fff700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 0x7fb484101360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb474004710 tx=0x7fb474009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.744+0000 7fb4817fa700 1 -- 172.21.15.39:0/159383123 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47401d070 con 0x7fb484101a40 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.744+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4841a1040 con 0x7fb484101a40 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.744+0000 7fb4817fa700 1 -- 172.21.15.39:0/159383123 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb47400be70 con 0x7fb484101a40 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.744+0000 7fb4817fa700 1 -- 172.21.15.39:0/159383123 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474021410 con 0x7fb484101a40 2024-05-24T11:36:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.744+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4841a1400 con 0x7fb484101a40 2024-05-24T11:36:51.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.745+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4840fcf70 con 0x7fb484101a40 2024-05-24T11:36:51.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.746+0000 7fb4817fa700 1 -- 172.21.15.39:0/159383123 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb47400f680 con 0x7fb484101a40 2024-05-24T11:36:51.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.746+0000 7fb4817fa700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb4700a1120 0x7fb4700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:51.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.747+0000 7fb4837fe700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb4700a1120 0x7fb4700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:51.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.747+0000 7fb4817fa700 1 -- 172.21.15.39:0/159383123 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb4740cda60 con 0x7fb484101a40 2024-05-24T11:36:51.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.747+0000 7fb4837fe700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb4700a1120 0x7fb4700a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009c60 tx=0x7fb46c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:51.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:51.751+0000 7fb4817fa700 1 -- 172.21.15.39:0/159383123 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb4740973d0 con 0x7fb484101a40 2024-05-24T11:36:52.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.009+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb484067cf0 con 0x7fb484101a40 2024-05-24T11:36:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:52 smithi053 ceph-mon[47448]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-24T11:36:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:52 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3166878369' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-24T11:36:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:52 smithi053 ceph-mon[47448]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-24T11:36:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:52 smithi053 ceph-mon[47448]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:52 smithi053 ceph-mon[47448]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-24T11:36:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:52 smithi053 ceph-mon[47448]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-24T11:36:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:52 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/159383123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-24T11:36:52.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:52 smithi137 ceph-mon[47907]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-24T11:36:52.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:52 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3166878369' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-24T11:36:52.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:52 smithi137 ceph-mon[47907]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-24T11:36:52.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:52 smithi137 ceph-mon[47907]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:52.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:52 smithi137 ceph-mon[47907]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-24T11:36:52.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:52 smithi137 ceph-mon[47907]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-24T11:36:52.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:52 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/159383123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-24T11:36:52.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:52 smithi039 ceph-mon[42919]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-24T11:36:52.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:52 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3166878369' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-24T11:36:52.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:52 smithi039 ceph-mon[42919]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-24T11:36:52.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:52 smithi039 ceph-mon[42919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-24T11:36:52.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:52 smithi039 ceph-mon[42919]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-24T11:36:52.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:52 smithi039 ceph-mon[42919]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-24T11:36:52.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:52 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/159383123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-24T11:36:52.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.597+0000 7fb4817fa700 1 -- 172.21.15.39:0/159383123 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb47409a9f0 con 0x7fb484101a40 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb4700a1120 msgr2=0x7fb4700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb4700a1120 0x7fb4700a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009c60 tx=0x7fb46c009260 comp rx=0 tx=0).stop 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 msgr2=0x7fb484101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 0x7fb484101360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb474004710 tx=0x7fb474009580 comp rx=0 tx=0).stop 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 shutdown_connections 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb4700a1120 0x7fb4700a35e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb484106ca0 0x7fb4840fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb484101a40 0x7fb484101360 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 --2- 172.21.15.39:0/159383123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb484104370 0x7fb4840ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 >> 172.21.15.39:0/159383123 conn(0x7fb4840fb3f0 msgr2=0x7fb484103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 shutdown_connections 2024-05-24T11:36:52.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.600+0000 7fb48a4b2700 1 -- 172.21.15.39:0/159383123 wait complete. 2024-05-24T11:36:52.624 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-24T11:36:52.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/1826613477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0101a00 msgr2=0x7fc2d0103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:52.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/1826613477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0101a00 0x7fc2d0103df0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009990 tx=0x7fc2c0009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/1826613477 shutdown_connections 2024-05-24T11:36:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/1826613477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d0106c60 0x7fc2d01090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/1826613477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0101a00 0x7fc2d0103df0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/1826613477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104330 0x7fc2d0106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/1826613477 >> 172.21.15.39:0/1826613477 conn(0x7fc2d00fb3f0 msgr2=0x7fc2d00fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/1826613477 shutdown_connections 2024-05-24T11:36:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/1826613477 wait complete. 2024-05-24T11:36:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.752+0000 7fc2d6a0d700 1 Processor -- start 2024-05-24T11:36:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d6a0d700 1 -- start start 2024-05-24T11:36:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d6a0d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d0101a00 0x7fc2d0196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d6a0d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104330 0x7fc2d01973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d6a0d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0106c60 0x7fc2d019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d6a0d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d019e9c0 con 0x7fc2d0106c60 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d6a0d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d019eb00 con 0x7fc2d0101a00 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d6a0d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d019ec90 con 0x7fc2d0104330 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d4faa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0106c60 0x7fc2d019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d4faa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0106c60 0x7fc2d019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54292/0 (socket says 172.21.15.39:54292) 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d4faa700 1 -- 172.21.15.39:0/2129242348 learned_addr learned my addr 172.21.15.39:0/2129242348 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2cffff700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d0101a00 0x7fc2d0196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2cf7fe700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104330 0x7fc2d01973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d4faa700 1 -- 172.21.15.39:0/2129242348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104330 msgr2=0x7fc2d01973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d4faa700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104330 0x7fc2d01973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d4faa700 1 -- 172.21.15.39:0/2129242348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d0101a00 msgr2=0x7fc2d0196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d4faa700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d0101a00 0x7fc2d0196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.753+0000 7fc2d4faa700 1 -- 172.21.15.39:0/2129242348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c00095f0 con 0x7fc2d0106c60 2024-05-24T11:36:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.754+0000 7fc2d4faa700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0106c60 0x7fc2d019e3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8007ee0 tx=0x7fc2c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.754+0000 7fc2cd7fa700 1 -- 172.21.15.39:0/2129242348 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8017070 con 0x7fc2d0106c60 2024-05-24T11:36:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.754+0000 7fc2cd7fa700 1 -- 172.21.15.39:0/2129242348 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2c800f040 con 0x7fc2d0106c60 2024-05-24T11:36:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.754+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d019ef70 con 0x7fc2d0106c60 2024-05-24T11:36:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.754+0000 7fc2cd7fa700 1 -- 172.21.15.39:0/2129242348 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8005710 con 0x7fc2d0106c60 2024-05-24T11:36:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.754+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d019f3a0 con 0x7fc2d0106c60 2024-05-24T11:36:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.755+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d00fcf70 con 0x7fc2d0106c60 2024-05-24T11:36:52.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.756+0000 7fc2cd7fa700 1 -- 172.21.15.39:0/2129242348 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc2c8005870 con 0x7fc2d0106c60 2024-05-24T11:36:52.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.757+0000 7fc2cd7fa700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc2bc0a1120 0x7fc2bc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:52.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.757+0000 7fc2cffff700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc2bc0a1120 0x7fc2bc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:52.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.757+0000 7fc2cd7fa700 1 -- 172.21.15.39:0/2129242348 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc2c80cc590 con 0x7fc2d0106c60 2024-05-24T11:36:52.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.757+0000 7fc2cffff700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc2bc0a1120 0x7fc2bc0a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0003780 tx=0x7fc2c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:52.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:52.760+0000 7fc2cd7fa700 1 -- 172.21.15.39:0/2129242348 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc2c8095e70 con 0x7fc2d0106c60 2024-05-24T11:36:53.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.022+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc2d010c6e0 con 0x7fc2d0106c60 2024-05-24T11:36:53.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.603+0000 7fc2cd7fa700 1 -- 172.21.15.39:0/2129242348 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc2c8099490 con 0x7fc2d0106c60 2024-05-24T11:36:53.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.606+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc2bc0a1120 msgr2=0x7fc2bc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:53.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.606+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc2bc0a1120 0x7fc2bc0a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0003780 tx=0x7fc2c0005fb0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.606+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0106c60 msgr2=0x7fc2d019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:53.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.606+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0106c60 0x7fc2d019e3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8007ee0 tx=0x7fc2c800d3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.607+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 shutdown_connections 2024-05-24T11:36:53.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.607+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fc2bc0a1120 0x7fc2bc0a35e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.607+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0106c60 0x7fc2d019e3f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.607+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d0101a00 0x7fc2d0196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.607+0000 7fc2d6a0d700 1 --2- 172.21.15.39:0/2129242348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2d0104330 0x7fc2d01973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.607+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 >> 172.21.15.39:0/2129242348 conn(0x7fc2d00fb3f0 msgr2=0x7fc2d0103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:53.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.607+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 shutdown_connections 2024-05-24T11:36:53.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.607+0000 7fc2d6a0d700 1 -- 172.21.15.39:0/2129242348 wait complete. 2024-05-24T11:36:53.630 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-24T11:36:53.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2024-05-24T11:36:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.760+0000 7f841133e700 1 -- 172.21.15.39:0/3727226995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c100f20 msgr2=0x7f840c101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.760+0000 7f841133e700 1 --2- 172.21.15.39:0/3727226995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c100f20 0x7f840c101360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009990 tx=0x7f83f4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.760+0000 7f841133e700 1 -- 172.21.15.39:0/3727226995 shutdown_connections 2024-05-24T11:36:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.760+0000 7f841133e700 1 --2- 172.21.15.39:0/3727226995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c102ab0 0x7f840c1093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.760+0000 7f841133e700 1 --2- 172.21.15.39:0/3727226995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c100f20 0x7f840c101360 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.760+0000 7f841133e700 1 --2- 172.21.15.39:0/3727226995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c1020f0 0x7f840c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.760+0000 7f841133e700 1 -- 172.21.15.39:0/3727226995 >> 172.21.15.39:0/3727226995 conn(0x7f840c0fc4e0 msgr2=0x7f840c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.761+0000 7f841133e700 1 -- 172.21.15.39:0/3727226995 shutdown_connections 2024-05-24T11:36:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.761+0000 7f841133e700 1 -- 172.21.15.39:0/3727226995 wait complete. 2024-05-24T11:36:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.761+0000 7f841133e700 1 Processor -- start 2024-05-24T11:36:53.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.761+0000 7f841133e700 1 -- start start 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f841133e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100f20 0x7f840c196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f841133e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c1020f0 0x7f840c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f841133e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c102ab0 0x7f840c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f841133e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c19e9c0 con 0x7f840c102ab0 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f841133e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c19eb00 con 0x7f840c1020f0 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f841133e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c19ec90 con 0x7f840c100f20 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c102ab0 0x7f840c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100f20 0x7f840c196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c102ab0 0x7f840c19e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54308/0 (socket says 172.21.15.39:54308) 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100f20 0x7f840c196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:60122/0 (socket says 172.21.15.39:60122) 2024-05-24T11:36:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840b7fe700 1 -- 172.21.15.39:0/2446718233 learned_addr learned my addr 172.21.15.39:0/2446718233 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840a7fc700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c1020f0 0x7f840c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840b7fe700 1 -- 172.21.15.39:0/2446718233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100f20 msgr2=0x7f840c196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840b7fe700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100f20 0x7f840c196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840b7fe700 1 -- 172.21.15.39:0/2446718233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c1020f0 msgr2=0x7f840c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840b7fe700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c1020f0 0x7f840c197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840b7fe700 1 -- 172.21.15.39:0/2446718233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f40095f0 con 0x7f840c102ab0 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840a7fc700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c1020f0 0x7f840c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.762+0000 7f840b7fe700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c102ab0 0x7f840c19e3f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8404007ee0 tx=0x7f840400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.763+0000 7f8403fff700 1 -- 172.21.15.39:0/2446718233 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840400ddd0 con 0x7f840c102ab0 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.763+0000 7f8403fff700 1 -- 172.21.15.39:0/2446718233 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f840400f040 con 0x7f840c102ab0 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.763+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c19ef70 con 0x7f840c102ab0 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.763+0000 7f8403fff700 1 -- 172.21.15.39:0/2446718233 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84040047d0 con 0x7f840c102ab0 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.763+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c19f3f0 con 0x7f840c102ab0 2024-05-24T11:36:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.764+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c1084a0 con 0x7f840c102ab0 2024-05-24T11:36:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.765+0000 7f8403fff700 1 -- 172.21.15.39:0/2446718233 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8404003a00 con 0x7f840c102ab0 2024-05-24T11:36:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.765+0000 7f8403fff700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f83f80a11a0 0x7f83f80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.765+0000 7f8403fff700 1 -- 172.21.15.39:0/2446718233 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84040cc030 con 0x7f840c102ab0 2024-05-24T11:36:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.765+0000 7f840affd700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f83f80a11a0 0x7f83f80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:53.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.766+0000 7f840affd700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f83f80a11a0 0x7f83f80a3660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83f400c010 tx=0x7f83f4003720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:53.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:53.770+0000 7f8403fff700 1 -- 172.21.15.39:0/2446718233 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8404095910 con 0x7f840c102ab0 2024-05-24T11:36:53.989 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-b[84735]: 2024-05-24T11:36:53.717+0000 7f0de1fce700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-24T11:36:53.989 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-b[84735]: 2024-05-24T11:36:53.717+0000 7f0de1fce700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-24T11:36:53.989 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-b[84735]: 2024-05-24T11:36:53.717+0000 7f0de1fce700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-24T11:36:53.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-mon[47448]: pgmap v222: 97 pgs: 74 active+clean, 11 creating+peering, 12 unknown; 580 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-05-24T11:36:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] up:active 2024-05-24T11:36:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/159383123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-24T11:36:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-mon[47448]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-24T11:36:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-mon[47448]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-24T11:36:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-mon[47448]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-24T11:36:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-mon[47448]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-24T11:36:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:53 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2129242348' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-24T11:36:53.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:53 smithi137 ceph-mon[47907]: pgmap v222: 97 pgs: 74 active+clean, 11 creating+peering, 12 unknown; 580 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-05-24T11:36:53.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:53 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] up:active 2024-05-24T11:36:53.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:53 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/159383123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-24T11:36:53.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:53 smithi137 ceph-mon[47907]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-24T11:36:53.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:53 smithi137 ceph-mon[47907]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-24T11:36:53.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:53 smithi137 ceph-mon[47907]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-24T11:36:53.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:53 smithi137 ceph-mon[47907]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-24T11:36:53.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:53 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2129242348' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-24T11:36:54.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.014+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f840c0613e0 con 0x7f83f80a11a0 2024-05-24T11:36:54.025 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:53 smithi039 ceph-mon[42919]: pgmap v222: 97 pgs: 74 active+clean, 11 creating+peering, 12 unknown; 580 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-05-24T11:36:54.026 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:53 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] up:active 2024-05-24T11:36:54.026 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:53 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/159383123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-24T11:36:54.026 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:53 smithi039 ceph-mon[42919]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-24T11:36:54.026 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:53 smithi039 ceph-mon[42919]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-24T11:36:54.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:53 smithi039 ceph-mon[42919]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-24T11:36:54.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:53 smithi039 ceph-mon[42919]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-24T11:36:54.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:53 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2129242348' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-24T11:36:54.327 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:36:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:36:54.019+0000 7fb85b112700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:36:54.327 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:36:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:36:54.019+0000 7fb85b112700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:36:54.328 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:36:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:36:54.019+0000 7fb85b112700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:36:54.328 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:36:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:36:54.019+0000 7fb85b112700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:36:54.328 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:36:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:36:54.019+0000 7fb85b112700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:36:54.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.614+0000 7f8403fff700 1 -- 172.21.15.39:0/2446718233 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f840c0613e0 con 0x7f83f80a11a0 2024-05-24T11:36:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f83f80a11a0 msgr2=0x7f83f80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f83f80a11a0 0x7f83f80a3660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83f400c010 tx=0x7f83f4003720 comp rx=0 tx=0).stop 2024-05-24T11:36:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c102ab0 msgr2=0x7f840c19e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c102ab0 0x7f840c19e3f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8404007ee0 tx=0x7f840400d3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 shutdown_connections 2024-05-24T11:36:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f83f80a11a0 0x7f83f80a3660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f840c100f20 0x7f840c196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840c102ab0 0x7f840c19e3f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 --2- 172.21.15.39:0/2446718233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c1020f0 0x7f840c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 >> 172.21.15.39:0/2446718233 conn(0x7f840c0fc4e0 msgr2=0x7f840c1055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:54.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 shutdown_connections 2024-05-24T11:36:54.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.616+0000 7f841133e700 1 -- 172.21.15.39:0/2446718233 wait complete. 2024-05-24T11:36:54.635 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2024-05-24T11:36:54.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.759+0000 7fe244efe700 1 -- 172.21.15.39:0/4095699742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe240100fd0 msgr2=0x7fe240101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:54.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.759+0000 7fe244efe700 1 --2- 172.21.15.39:0/4095699742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe240100fd0 0x7fe240101450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe234009990 tx=0x7fe234009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.759+0000 7fe244efe700 1 -- 172.21.15.39:0/4095699742 shutdown_connections 2024-05-24T11:36:54.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.759+0000 7fe244efe700 1 --2- 172.21.15.39:0/4095699742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe240101990 0x7fe2401082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.759+0000 7fe244efe700 1 --2- 172.21.15.39:0/4095699742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2400ffe80 0x7fe2401002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.759+0000 7fe244efe700 1 --2- 172.21.15.39:0/4095699742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe240100fd0 0x7fe240101450 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.759+0000 7fe244efe700 1 -- 172.21.15.39:0/4095699742 >> 172.21.15.39:0/4095699742 conn(0x7fe2400fb3f0 msgr2=0x7fe2400fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:54.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.759+0000 7fe244efe700 1 -- 172.21.15.39:0/4095699742 shutdown_connections 2024-05-24T11:36:54.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.759+0000 7fe244efe700 1 -- 172.21.15.39:0/4095699742 wait complete. 2024-05-24T11:36:54.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.760+0000 7fe244efe700 1 Processor -- start 2024-05-24T11:36:54.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.760+0000 7fe244efe700 1 -- start start 2024-05-24T11:36:54.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe244efe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2400ffe80 0x7fe240195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:54.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe244efe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe240100fd0 0x7fe240196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:54.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe244efe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe240101990 0x7fe24019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:54.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe244efe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24019d880 con 0x7fe2400ffe80 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe244efe700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24019d9f0 con 0x7fe240100fd0 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe244efe700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24019dbb0 con 0x7fe240101990 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2400ffe80 0x7fe240195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2400ffe80 0x7fe240195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54336/0 (socket says 172.21.15.39:54336) 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23e59c700 1 -- 172.21.15.39:0/2412014732 learned_addr learned my addr 172.21.15.39:0/2412014732 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23e59c700 1 -- 172.21.15.39:0/2412014732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe240101990 msgr2=0x7fe24019d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23e59c700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe240101990 0x7fe24019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23e59c700 1 -- 172.21.15.39:0/2412014732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe240100fd0 msgr2=0x7fe240196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23dd9b700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe240100fd0 0x7fe240196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23e59c700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe240100fd0 0x7fe240196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23e59c700 1 -- 172.21.15.39:0/2412014732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2340095f0 con 0x7fe2400ffe80 2024-05-24T11:36:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe23e59c700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2400ffe80 0x7fe240195de0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe23000d950 tx=0x7fe23000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe22f7fe700 1 -- 172.21.15.39:0/2412014732 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230009b40 con 0x7fe2400ffe80 2024-05-24T11:36:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.761+0000 7fe22f7fe700 1 -- 172.21.15.39:0/2412014732 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe230009ca0 con 0x7fe2400ffe80 2024-05-24T11:36:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.762+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24019de90 con 0x7fe2400ffe80 2024-05-24T11:36:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.762+0000 7fe22f7fe700 1 -- 172.21.15.39:0/2412014732 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23000f620 con 0x7fe2400ffe80 2024-05-24T11:36:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.762+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24019e2b0 con 0x7fe2400ffe80 2024-05-24T11:36:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.763+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe240107380 con 0x7fe2400ffe80 2024-05-24T11:36:54.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.764+0000 7fe22f7fe700 1 -- 172.21.15.39:0/2412014732 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe230010460 con 0x7fe2400ffe80 2024-05-24T11:36:54.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.764+0000 7fe22f7fe700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2280a1120 0x7fe2280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:54.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.764+0000 7fe23dd9b700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2280a1120 0x7fe2280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:54.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.765+0000 7fe22f7fe700 1 -- 172.21.15.39:0/2412014732 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe2300ccb00 con 0x7fe2400ffe80 2024-05-24T11:36:54.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.765+0000 7fe23dd9b700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2280a1120 0x7fe2280a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe234006010 tx=0x7fe234005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:54.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:54.769+0000 7fe22f7fe700 1 -- 172.21.15.39:0/2412014732 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe2300963e0 con 0x7fe2400ffe80 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] up:active 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2129242348' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] up:active 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: daemon mds.g assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: Cluster is now healthy 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-05-24T11:36:54.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-24T11:36:54.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:54 smithi053 ceph-mon[47448]: daemon mds.g is now active in filesystem cephfs as rank 4 2024-05-24T11:36:54.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] up:active 2024-05-24T11:36:54.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2129242348' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-24T11:36:54.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-24T11:36:54.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-24T11:36:54.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-05-24T11:36:54.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-24T11:36:54.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] up:active 2024-05-24T11:36:54.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: daemon mds.g assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-24T11:36:54.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-24T11:36:54.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: Cluster is now healthy 2024-05-24T11:36:54.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-05-24T11:36:54.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-24T11:36:54.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:54 smithi137 ceph-mon[47907]: daemon mds.g is now active in filesystem cephfs as rank 4 2024-05-24T11:36:55.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.015+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe24002ce10 con 0x7fe2280a1120 2024-05-24T11:36:55.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.019+0000 7fe22f7fe700 1 -- 172.21.15.39:0/2412014732 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fe24002ce10 con 0x7fe2280a1120 2024-05-24T11:36:55.025 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/_nogroup/sv_0/650282f3-ff17-4113-9a25-acb8b2736148 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2280a1120 msgr2=0x7fe2280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2280a1120 0x7fe2280a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe234006010 tx=0x7fe234005c60 comp rx=0 tx=0).stop 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2400ffe80 msgr2=0x7fe240195de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2400ffe80 0x7fe240195de0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe23000d950 tx=0x7fe23000dd10 comp rx=0 tx=0).stop 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 shutdown_connections 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe2280a1120 0x7fe2280a35e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe240101990 0x7fe24019d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2400ffe80 0x7fe240195de0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 --2- 172.21.15.39:0/2412014732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe240100fd0 0x7fe240196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 >> 172.21.15.39:0/2412014732 conn(0x7fe2400fb3f0 msgr2=0x7fe2401044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 shutdown_connections 2024-05-24T11:36:55.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.021+0000 7fe244efe700 1 -- 172.21.15.39:0/2412014732 wait complete. 2024-05-24T11:36:55.038 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] up:active 2024-05-24T11:36:55.038 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2129242348' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-24T11:36:55.038 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-24T11:36:55.038 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-24T11:36:55.038 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-05-24T11:36:55.038 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-24T11:36:55.038 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] up:active 2024-05-24T11:36:55.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: daemon mds.g assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-24T11:36:55.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-24T11:36:55.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: Cluster is now healthy 2024-05-24T11:36:55.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-05-24T11:36:55.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-24T11:36:55.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:54 smithi039 ceph-mon[42919]: daemon mds.g is now active in filesystem cephfs as rank 4 2024-05-24T11:36:55.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2024-05-24T11:36:55.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.165+0000 7ffb09155700 1 -- 172.21.15.39:0/726612508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb04100fd0 msgr2=0x7ffb04101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.165+0000 7ffb09155700 1 --2- 172.21.15.39:0/726612508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb04100fd0 0x7ffb04101450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009990 tx=0x7ffaec009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.165+0000 7ffb09155700 1 -- 172.21.15.39:0/726612508 shutdown_connections 2024-05-24T11:36:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.165+0000 7ffb09155700 1 --2- 172.21.15.39:0/726612508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb040ffe80 0x7ffb041002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.165+0000 7ffb09155700 1 --2- 172.21.15.39:0/726612508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb04101990 0x7ffb041082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.165+0000 7ffb09155700 1 --2- 172.21.15.39:0/726612508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb04100fd0 0x7ffb04101450 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.165+0000 7ffb09155700 1 -- 172.21.15.39:0/726612508 >> 172.21.15.39:0/726612508 conn(0x7ffb040fb3f0 msgr2=0x7ffb040fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.165+0000 7ffb09155700 1 -- 172.21.15.39:0/726612508 shutdown_connections 2024-05-24T11:36:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.165+0000 7ffb09155700 1 -- 172.21.15.39:0/726612508 wait complete. 2024-05-24T11:36:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb09155700 1 Processor -- start 2024-05-24T11:36:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb09155700 1 -- start start 2024-05-24T11:36:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb09155700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb040ffe80 0x7ffb04199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb09155700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb04100fd0 0x7ffb041995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb09155700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04101990 0x7ffb041a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb09155700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb041a0ba0 con 0x7ffb040ffe80 2024-05-24T11:36:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb09155700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb041a0ce0 con 0x7ffb04100fd0 2024-05-24T11:36:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb09155700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb041a0e70 con 0x7ffb04101990 2024-05-24T11:36:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb02d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb040ffe80 0x7ffb04199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb02d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb040ffe80 0x7ffb04199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54364/0 (socket says 172.21.15.39:54364) 2024-05-24T11:36:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb0259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb04100fd0 0x7ffb041995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb02d9d700 1 -- 172.21.15.39:0/1946957241 learned_addr learned my addr 172.21.15.39:0/1946957241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb0359e700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04101990 0x7ffb041a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb02d9d700 1 -- 172.21.15.39:0/1946957241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04101990 msgr2=0x7ffb041a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb02d9d700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04101990 0x7ffb041a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb02d9d700 1 -- 172.21.15.39:0/1946957241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb04100fd0 msgr2=0x7ffb041995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.166+0000 7ffb02d9d700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb04100fd0 0x7ffb041995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.167+0000 7ffb02d9d700 1 -- 172.21.15.39:0/1946957241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec0095f0 con 0x7ffb040ffe80 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.167+0000 7ffb02d9d700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb040ffe80 0x7ffb04199070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400c920 tx=0x7ffaf400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.167+0000 7ffafbfff700 1 -- 172.21.15.39:0/1946957241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4007b00 con 0x7ffb040ffe80 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.167+0000 7ffafbfff700 1 -- 172.21.15.39:0/1946957241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffaf400ce70 con 0x7ffb040ffe80 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.167+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb041a1150 con 0x7ffb040ffe80 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.167+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb041a1660 con 0x7ffb040ffe80 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.167+0000 7ffafbfff700 1 -- 172.21.15.39:0/1946957241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40053e0 con 0x7ffb040ffe80 2024-05-24T11:36:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.168+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb04067460 con 0x7ffb040ffe80 2024-05-24T11:36:55.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.169+0000 7ffafbfff700 1 -- 172.21.15.39:0/1946957241 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ffaf4005540 con 0x7ffb040ffe80 2024-05-24T11:36:55.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.170+0000 7ffafbfff700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ffaf00a1120 0x7ffaf00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:55.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.170+0000 7ffb0259c700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ffaf00a1120 0x7ffaf00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:55.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.170+0000 7ffafbfff700 1 -- 172.21.15.39:0/1946957241 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffaf40cc860 con 0x7ffb040ffe80 2024-05-24T11:36:55.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.170+0000 7ffb0259c700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ffaf00a1120 0x7ffaf00a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00c010 tx=0x7ffaec005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:55.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.173+0000 7ffafbfff700 1 -- 172.21.15.39:0/1946957241 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffaf40960a0 con 0x7ffb040ffe80 2024-05-24T11:36:55.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.417+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb04069950 con 0x7ffaf00a1120 2024-05-24T11:36:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.441+0000 7ffafbfff700 1 -- 172.21.15.39:0/1946957241 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffb04069950 con 0x7ffaf00a1120 2024-05-24T11:36:55.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ffaf00a1120 msgr2=0x7ffaf00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ffaf00a1120 0x7ffaf00a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00c010 tx=0x7ffaec005fd0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb040ffe80 msgr2=0x7ffb04199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb040ffe80 0x7ffb04199070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400c920 tx=0x7ffaf400cc30 comp rx=0 tx=0).stop 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 shutdown_connections 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ffaf00a1120 0x7ffaf00a35e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04101990 0x7ffb041a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb040ffe80 0x7ffb04199070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 --2- 172.21.15.39:0/1946957241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb04100fd0 0x7ffb041995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 >> 172.21.15.39:0/1946957241 conn(0x7ffb040fb3f0 msgr2=0x7ffb04069240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 shutdown_connections 2024-05-24T11:36:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.444+0000 7ffb09155700 1 -- 172.21.15.39:0/1946957241 wait complete. 2024-05-24T11:36:55.462 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2024-05-24T11:36:55.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.589+0000 7fa282508700 1 -- 172.21.15.39:0/3526337596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c0ffe80 msgr2=0x7fa27c1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.589+0000 7fa282508700 1 --2- 172.21.15.39:0/3526337596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c0ffe80 0x7fa27c1002a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa270009990 tx=0x7fa270009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.590+0000 7fa282508700 1 -- 172.21.15.39:0/3526337596 shutdown_connections 2024-05-24T11:36:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.590+0000 7fa282508700 1 --2- 172.21.15.39:0/3526337596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c101990 0x7fa27c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.590+0000 7fa282508700 1 --2- 172.21.15.39:0/3526337596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c0ffe80 0x7fa27c1002a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.590+0000 7fa282508700 1 --2- 172.21.15.39:0/3526337596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27c100fd0 0x7fa27c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.590+0000 7fa282508700 1 -- 172.21.15.39:0/3526337596 >> 172.21.15.39:0/3526337596 conn(0x7fa27c0fb3f0 msgr2=0x7fa27c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.590+0000 7fa282508700 1 -- 172.21.15.39:0/3526337596 shutdown_connections 2024-05-24T11:36:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.590+0000 7fa282508700 1 -- 172.21.15.39:0/3526337596 wait complete. 2024-05-24T11:36:55.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.590+0000 7fa282508700 1 Processor -- start 2024-05-24T11:36:55.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.590+0000 7fa282508700 1 -- start start 2024-05-24T11:36:55.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa282508700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27c0ffe80 0x7fa27c10a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:55.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa282508700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c100fd0 0x7fa27c108c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:55.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa282508700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c101990 0x7fa27c109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:55.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa282508700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c109910 con 0x7fa27c100fd0 2024-05-24T11:36:55.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa282508700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c109a80 con 0x7fa27c101990 2024-05-24T11:36:55.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa282508700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c1a0e40 con 0x7fa27c0ffe80 2024-05-24T11:36:55.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c100fd0 0x7fa27c108c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c100fd0 0x7fa27c108c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54380/0 (socket says 172.21.15.39:54380) 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27b7fe700 1 -- 172.21.15.39:0/798042786 learned_addr learned my addr 172.21.15.39:0/798042786 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa280aa5700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c101990 0x7fa27c109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27bfff700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27c0ffe80 0x7fa27c10a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27b7fe700 1 -- 172.21.15.39:0/798042786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27c0ffe80 msgr2=0x7fa27c10a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27b7fe700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27c0ffe80 0x7fa27c10a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27b7fe700 1 -- 172.21.15.39:0/798042786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c101990 msgr2=0x7fa27c109170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27b7fe700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c101990 0x7fa27c109170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27b7fe700 1 -- 172.21.15.39:0/798042786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700095f0 con 0x7fa27c100fd0 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.591+0000 7fa27b7fe700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c100fd0 0x7fa27c108c30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00ef10 tx=0x7fa26c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.592+0000 7fa2797fa700 1 -- 172.21.15.39:0/798042786 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00c960 con 0x7fa27c100fd0 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.592+0000 7fa2797fa700 1 -- 172.21.15.39:0/798042786 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa26c00f040 con 0x7fa27c100fd0 2024-05-24T11:36:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.592+0000 7fa2797fa700 1 -- 172.21.15.39:0/798042786 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c007e60 con 0x7fa27c100fd0 2024-05-24T11:36:55.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.592+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c1a1070 con 0x7fa27c100fd0 2024-05-24T11:36:55.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.592+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c1a1530 con 0x7fa27c100fd0 2024-05-24T11:36:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.593+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c1072f0 con 0x7fa27c100fd0 2024-05-24T11:36:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.594+0000 7fa2797fa700 1 -- 172.21.15.39:0/798042786 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa26c007600 con 0x7fa27c100fd0 2024-05-24T11:36:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.595+0000 7fa2797fa700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2640a1120 0x7fa2640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.595+0000 7fa27bfff700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2640a1120 0x7fa2640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.595+0000 7fa2797fa700 1 -- 172.21.15.39:0/798042786 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa26c0cca10 con 0x7fa27c100fd0 2024-05-24T11:36:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.595+0000 7fa27bfff700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2640a1120 0x7fa2640a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa270005910 tx=0x7fa27000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.598+0000 7fa2797fa700 1 -- 172.21.15.39:0/798042786 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa26c0962f0 con 0x7fa27c100fd0 2024-05-24T11:36:55.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:55 smithi039 ceph-mon[42919]: pgmap v223: 97 pgs: 97 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 13 op/s 2024-05-24T11:36:55.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:55 smithi039 ceph-mon[42919]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:55.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:55 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] up:active 2024-05-24T11:36:55.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:55 smithi039 ceph-mon[42919]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:36:55.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:55 smithi039 ceph-mon[42919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:36:55.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.851+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa27c0613e0 con 0x7fa2640a1120 2024-05-24T11:36:55.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.855+0000 7fa2797fa700 1 -- 172.21.15.39:0/798042786 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fa27c0613e0 con 0x7fa2640a1120 2024-05-24T11:36:55.860 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/_nogroup/sv_1/d9e7be1e-3d2a-4b05-8ab7-b2f9780d822a 2024-05-24T11:36:55.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2640a1120 msgr2=0x7fa2640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2640a1120 0x7fa2640a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa270005910 tx=0x7fa27000b610 comp rx=0 tx=0).stop 2024-05-24T11:36:55.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c100fd0 msgr2=0x7fa27c108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:55.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c100fd0 0x7fa27c108c30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00ef10 tx=0x7fa26c00c3a0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 shutdown_connections 2024-05-24T11:36:55.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2640a1120 0x7fa2640a35e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27c0ffe80 0x7fa27c10a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c101990 0x7fa27c109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 --2- 172.21.15.39:0/798042786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa27c100fd0 0x7fa27c108c30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:55.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 >> 172.21.15.39:0/798042786 conn(0x7fa27c0fb3f0 msgr2=0x7fa27c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:55.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 shutdown_connections 2024-05-24T11:36:55.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:55.857+0000 7fa282508700 1 -- 172.21.15.39:0/798042786 wait complete. 2024-05-24T11:36:55.878 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:36:55.988 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:55 smithi053 ceph-mon[47448]: pgmap v223: 97 pgs: 97 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 13 op/s 2024-05-24T11:36:55.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:55 smithi053 ceph-mon[47448]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:55.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:55 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] up:active 2024-05-24T11:36:55.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:55 smithi053 ceph-mon[47448]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:36:55.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:55 smithi053 ceph-mon[47448]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:36:55.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:55 smithi137 ceph-mon[47907]: pgmap v223: 97 pgs: 97 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 13 op/s 2024-05-24T11:36:55.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:55 smithi137 ceph-mon[47907]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:55.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:55 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] up:active 2024-05-24T11:36:55.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:55 smithi137 ceph-mon[47907]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:36:55.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:55 smithi137 ceph-mon[47907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:36:56.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 -- 172.21.15.39:0/2138827915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c101920 msgr2=0x7f322c1081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 --2- 172.21.15.39:0/2138827915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c101920 0x7f322c1081d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f322400b0d0 tx=0x7f322400b3e0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 -- 172.21.15.39:0/2138827915 shutdown_connections 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 --2- 172.21.15.39:0/2138827915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c0ffd90 0x7f322c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 --2- 172.21.15.39:0/2138827915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c101920 0x7f322c1081d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 --2- 172.21.15.39:0/2138827915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c100f80 0x7f322c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 -- 172.21.15.39:0/2138827915 >> 172.21.15.39:0/2138827915 conn(0x7f322c0fb300 msgr2=0x7f322c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 -- 172.21.15.39:0/2138827915 shutdown_connections 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 -- 172.21.15.39:0/2138827915 wait complete. 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 Processor -- start 2024-05-24T11:36:56.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.001+0000 7f323266c700 1 -- start start 2024-05-24T11:36:56.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f323266c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c0ffd90 0x7f322c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f323266c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c100f80 0x7f322c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f323266c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c101920 0x7f322c19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f323266c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19e8d0 con 0x7f322c100f80 2024-05-24T11:36:56.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f323266c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19ea40 con 0x7f322c101920 2024-05-24T11:36:56.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f323266c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19ec00 con 0x7f322c0ffd90 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c100f80 0x7f322c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c100f80 0x7f322c19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54398/0 (socket says 172.21.15.39:54398) 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322b7fe700 1 -- 172.21.15.39:0/2742879770 learned_addr learned my addr 172.21.15.39:0/2742879770 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322bfff700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c0ffd90 0x7f322c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f3230c09700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c101920 0x7f322c19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322b7fe700 1 -- 172.21.15.39:0/2742879770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c0ffd90 msgr2=0x7f322c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322b7fe700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c0ffd90 0x7f322c069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322b7fe700 1 -- 172.21.15.39:0/2742879770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c101920 msgr2=0x7f322c19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322b7fe700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c101920 0x7f322c19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322b7fe700 1 -- 172.21.15.39:0/2742879770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224009d00 con 0x7f322c100f80 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f3230c09700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c101920 0x7f322c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f322b7fe700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c100f80 0x7f322c19bae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f321400e9a0 tx=0x7f321400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f32297fa700 1 -- 172.21.15.39:0/2742879770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214009790 con 0x7f322c100f80 2024-05-24T11:36:56.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f32297fa700 1 -- 172.21.15.39:0/2742879770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f321400cbd0 con 0x7f322c100f80 2024-05-24T11:36:56.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f32297fa700 1 -- 172.21.15.39:0/2742879770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214018680 con 0x7f322c100f80 2024-05-24T11:36:56.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c19eee0 con 0x7f322c100f80 2024-05-24T11:36:56.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.002+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c19f310 con 0x7f322c100f80 2024-05-24T11:36:56.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.003+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c1071c0 con 0x7f322c100f80 2024-05-24T11:36:56.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.005+0000 7f32297fa700 1 -- 172.21.15.39:0/2742879770 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f321400cd40 con 0x7f322c100f80 2024-05-24T11:36:56.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.005+0000 7f32297fa700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f32180a1120 0x7f32180a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.005+0000 7f322bfff700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f32180a1120 0x7f32180a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.005+0000 7f32297fa700 1 -- 172.21.15.39:0/2742879770 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3214014070 con 0x7f322c100f80 2024-05-24T11:36:56.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.006+0000 7f322bfff700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f32180a1120 0x7f32180a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f321c005950 tx=0x7f321c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:56.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.009+0000 7f32297fa700 1 -- 172.21.15.39:0/2742879770 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3214096590 con 0x7f322c100f80 2024-05-24T11:36:56.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.266+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f322c067380 con 0x7f322c100f80 2024-05-24T11:36:56.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.266+0000 7f32297fa700 1 -- 172.21.15.39:0/2742879770 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10906 (secure 0 0 0) 0x7f3214026020 con 0x7f322c100f80 2024-05-24T11:36:56.272 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:56.272 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-24T11:36:48.500682+0000","modified":"2024-05-24T11:36:55.584150+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24511,"mds_2":24506,"mds_3":24490,"mds_4":14631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/3006971240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3006971240},{"type":"v1","addr":"172.21.15.39:6839","nonce":3006971240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/3024712815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3024712815},{"type":"v1","addr":"172.21.15.39:6841","nonce":3024712815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/218688120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":218688120},{"type":"v1","addr":"172.21.15.137:6839","nonce":218688120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3196235839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3196235839},{"type":"v1","addr":"172.21.15.53:6839","nonce":3196235839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-24T11:36:56.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f32180a1120 msgr2=0x7f32180a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f32180a1120 0x7f32180a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f321c005950 tx=0x7f321c0058e0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c100f80 msgr2=0x7f322c19bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c100f80 0x7f322c19bae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f321400e9a0 tx=0x7f321400ed60 comp rx=0 tx=0).stop 2024-05-24T11:36:56.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 shutdown_connections 2024-05-24T11:36:56.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f32180a1120 0x7f32180a35e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c0ffd90 0x7f322c069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c101920 0x7f322c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 --2- 172.21.15.39:0/2742879770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322c100f80 0x7f322c19bae0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.268+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 >> 172.21.15.39:0/2742879770 conn(0x7f322c0fb300 msgr2=0x7f322c104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:56.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.269+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 shutdown_connections 2024-05-24T11:36:56.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.269+0000 7f323266c700 1 -- 172.21.15.39:0/2742879770 wait complete. 2024-05-24T11:36:56.275 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-24T11:36:56.287 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:36:56.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 -- 172.21.15.39:0/2061876395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97540ffd70 msgr2=0x7f9754100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 --2- 172.21.15.39:0/2061876395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97540ffd70 0x7f9754100190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f97500099c0 tx=0x7f9750009cd0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 -- 172.21.15.39:0/2061876395 shutdown_connections 2024-05-24T11:36:56.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 --2- 172.21.15.39:0/2061876395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9754101900 0x7f97541081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 --2- 172.21.15.39:0/2061876395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97540ffd70 0x7f9754100190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 --2- 172.21.15.39:0/2061876395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9754100f60 0x7f97541013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 -- 172.21.15.39:0/2061876395 >> 172.21.15.39:0/2061876395 conn(0x7f97540fb2e0 msgr2=0x7f97540fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 -- 172.21.15.39:0/2061876395 shutdown_connections 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 -- 172.21.15.39:0/2061876395 wait complete. 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 Processor -- start 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 -- start start 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97540ffd70 0x7f9754196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9754100f60 0x7f97541972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.408+0000 7f975c233700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9754101900 0x7f975419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975c233700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975419e870 con 0x7f9754101900 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975c233700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975419e9e0 con 0x7f9754100f60 2024-05-24T11:36:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975c233700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975419eba0 con 0x7f97540ffd70 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975a7d0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9754101900 0x7f975419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975a7d0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9754101900 0x7f975419e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54422/0 (socket says 172.21.15.39:54422) 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975a7d0700 1 -- 172.21.15.39:0/1033092850 learned_addr learned my addr 172.21.15.39:0/1033092850 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f9759fcf700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97540ffd70 0x7f9754196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f97597ce700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9754100f60 0x7f97541972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975a7d0700 1 -- 172.21.15.39:0/1033092850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97540ffd70 msgr2=0x7f9754196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975a7d0700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97540ffd70 0x7f9754196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975a7d0700 1 -- 172.21.15.39:0/1033092850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9754100f60 msgr2=0x7f97541972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975a7d0700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9754100f60 0x7f97541972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975a7d0700 1 -- 172.21.15.39:0/1033092850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97500095f0 con 0x7f9754101900 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f97597ce700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9754100f60 0x7f97541972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975a7d0700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9754101900 0x7f975419e2a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f974c007ee0 tx=0x7f974c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:56.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f974affd700 1 -- 172.21.15.39:0/1033092850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c017070 con 0x7f9754101900 2024-05-24T11:36:56.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f974affd700 1 -- 172.21.15.39:0/1033092850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f974c00f040 con 0x7f9754101900 2024-05-24T11:36:56.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975419ee80 con 0x7f9754101900 2024-05-24T11:36:56.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f974affd700 1 -- 172.21.15.39:0/1033092850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c005710 con 0x7f9754101900 2024-05-24T11:36:56.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.409+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975419f2a0 con 0x7f9754101900 2024-05-24T11:36:56.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.410+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9754068060 con 0x7f9754101900 2024-05-24T11:36:56.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.411+0000 7f974affd700 1 -- 172.21.15.39:0/1033092850 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f974c005870 con 0x7f9754101900 2024-05-24T11:36:56.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.412+0000 7f974affd700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f97400a11a0 0x7f97400a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.412+0000 7f9759fcf700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f97400a11a0 0x7f97400a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.412+0000 7f974affd700 1 -- 172.21.15.39:0/1033092850 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f974c0cc630 con 0x7f9754101900 2024-05-24T11:36:56.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.412+0000 7f9759fcf700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f97400a11a0 0x7f97400a3660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9750009990 tx=0x7f9750005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:56.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.415+0000 7f974affd700 1 -- 172.21.15.39:0/1033092850 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f974c095e70 con 0x7f9754101900 2024-05-24T11:36:56.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.625+0000 7f974affd700 1 -- 172.21.15.39:0/1033092850 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f974c096240 con 0x7f9754101900 2024-05-24T11:36:56.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.633+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f975410b6b0 con 0x7f9754101900 2024-05-24T11:36:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.634+0000 7f974affd700 1 -- 172.21.15.39:0/1033092850 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+22061 (secure 0 0 0) 0x7f974c099900 con 0x7f9754101900 2024-05-24T11:36:56.640 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:56.640 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:49.502598+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:36:56.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f97400a11a0 msgr2=0x7f97400a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f97400a11a0 0x7f97400a3660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9750009990 tx=0x7f9750005fb0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9754101900 msgr2=0x7f975419e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9754101900 0x7f975419e2a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f974c007ee0 tx=0x7f974c00d3b0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 shutdown_connections 2024-05-24T11:36:56.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f97400a11a0 0x7f97400a3660 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97540ffd70 0x7f9754196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9754101900 0x7f975419e2a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 --2- 172.21.15.39:0/1033092850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9754100f60 0x7f97541972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 >> 172.21.15.39:0/1033092850 conn(0x7f97540fb2e0 msgr2=0x7f9754069460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:56.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 shutdown_connections 2024-05-24T11:36:56.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.637+0000 7f975c233700 1 -- 172.21.15.39:0/1033092850 wait complete. 2024-05-24T11:36:56.655 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-24T11:36:56.655 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-24T11:36:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.776+0000 7f1b46a79700 1 -- 172.21.15.39:0/3135550023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b401018e0 msgr2=0x7f1b40103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.776+0000 7f1b46a79700 1 --2- 172.21.15.39:0/3135550023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b401018e0 0x7f1b40103cd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009990 tx=0x7f1b34009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.776+0000 7f1b46a79700 1 -- 172.21.15.39:0/3135550023 shutdown_connections 2024-05-24T11:36:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.776+0000 7f1b46a79700 1 --2- 172.21.15.39:0/3135550023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b40106b40 0x7f1b40108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.776+0000 7f1b46a79700 1 --2- 172.21.15.39:0/3135550023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b401018e0 0x7f1b40103cd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.776+0000 7f1b46a79700 1 --2- 172.21.15.39:0/3135550023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b40104210 0x7f1b40106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.776+0000 7f1b46a79700 1 -- 172.21.15.39:0/3135550023 >> 172.21.15.39:0/3135550023 conn(0x7f1b400fb2f0 msgr2=0x7f1b400fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.776+0000 7f1b46a79700 1 -- 172.21.15.39:0/3135550023 shutdown_connections 2024-05-24T11:36:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.776+0000 7f1b46a79700 1 -- 172.21.15.39:0/3135550023 wait complete. 2024-05-24T11:36:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b46a79700 1 Processor -- start 2024-05-24T11:36:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b46a79700 1 -- start start 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b46a79700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b401018e0 0x7f1b40196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b46a79700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b40104210 0x7f1b401972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b46a79700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b40106b40 0x7f1b4019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b46a79700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4019e850 con 0x7f1b40104210 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b46a79700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4019e9c0 con 0x7f1b40106b40 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b46a79700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4019eb80 con 0x7f1b401018e0 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b40104210 0x7f1b401972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b3ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b40104210 0x7f1b401972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54438/0 (socket says 172.21.15.39:54438) 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b3ffff700 1 -- 172.21.15.39:0/978907197 learned_addr learned my addr 172.21.15.39:0/978907197 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b44815700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b401018e0 0x7f1b40196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b3ffff700 1 -- 172.21.15.39:0/978907197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b401018e0 msgr2=0x7f1b40196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b3ffff700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b401018e0 0x7f1b40196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b45016700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b40106b40 0x7f1b4019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b3ffff700 1 -- 172.21.15.39:0/978907197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b40106b40 msgr2=0x7f1b4019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b3ffff700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b40106b40 0x7f1b4019e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.777+0000 7f1b3ffff700 1 -- 172.21.15.39:0/978907197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b340095f0 con 0x7f1b40104210 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.778+0000 7f1b3ffff700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b40104210 0x7f1b401972a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00d950 tx=0x7f1b2c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.778+0000 7f1b3dffb700 1 -- 172.21.15.39:0/978907197 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c009b40 con 0x7f1b40104210 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.778+0000 7f1b3dffb700 1 -- 172.21.15.39:0/978907197 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b2c009ca0 con 0x7f1b40104210 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.778+0000 7f1b3dffb700 1 -- 172.21.15.39:0/978907197 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00f620 con 0x7f1b40104210 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.778+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4019ee60 con 0x7f1b40104210 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.778+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4019f2b0 con 0x7f1b40104210 2024-05-24T11:36:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.779+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b400fce70 con 0x7f1b40104210 2024-05-24T11:36:56.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.780+0000 7f1b3dffb700 1 -- 172.21.15.39:0/978907197 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1b2c010460 con 0x7f1b40104210 2024-05-24T11:36:56.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.781+0000 7f1b3dffb700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b300a11b0 0x7f1b300a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:56.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.781+0000 7f1b44815700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b300a11b0 0x7f1b300a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:56.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.781+0000 7f1b3dffb700 1 -- 172.21.15.39:0/978907197 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1b2c0ccd30 con 0x7f1b40104210 2024-05-24T11:36:56.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.781+0000 7f1b44815700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b300a11b0 0x7f1b300a3670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b34006010 tx=0x7f1b34005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:56.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.786+0000 7f1b3dffb700 1 -- 172.21.15.39:0/978907197 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1b2c096410 con 0x7f1b40104210 2024-05-24T11:36:56.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:56 smithi053 ceph-mon[47448]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:56.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:56 smithi053 ceph-mon[47448]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:56.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:56 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2742879770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:56.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:56 smithi137 ceph-mon[47907]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:56.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:56 smithi137 ceph-mon[47907]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:56.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:56 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2742879770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:56.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:56.991+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b4002ceb0 con 0x7f1b300a11b0 2024-05-24T11:36:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.002+0000 7f1b3dffb700 1 -- 172.21.15.39:0/978907197 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328439 (secure 0 0 0) 0x7f1b4002ceb0 con 0x7f1b300a11b0 2024-05-24T11:36:57.013 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:57.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b300a11b0 msgr2=0x7f1b300a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:57.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b300a11b0 0x7f1b300a3670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b34006010 tx=0x7f1b34005ec0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b40104210 msgr2=0x7f1b401972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:57.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b40104210 0x7f1b401972a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00d950 tx=0x7f1b2c00dd10 comp rx=0 tx=0).stop 2024-05-24T11:36:57.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 shutdown_connections 2024-05-24T11:36:57.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1b300a11b0 0x7f1b300a3670 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b401018e0 0x7f1b40196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b40106b40 0x7f1b4019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 --2- 172.21.15.39:0/978907197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b40104210 0x7f1b401972a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 >> 172.21.15.39:0/978907197 conn(0x7f1b400fb2f0 msgr2=0x7f1b40102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:57.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 shutdown_connections 2024-05-24T11:36:57.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.006+0000 7f1b46a79700 1 -- 172.21.15.39:0/978907197 wait complete. 2024-05-24T11:36:57.018 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-24T11:36:57.023 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:56 smithi039 ceph-mon[42919]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:57.023 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:56 smithi039 ceph-mon[42919]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:57.023 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:56 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2742879770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:36:57.024 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-05-24T11:36:55.876816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596174,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":153,"num_read_kb":135,"num_write":337,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380876,"kb_used_data":11600,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124560948,"statfs":{"total":1151940427776,"available":1151550410752,"internally_reserved":0,"allocated":11878400,"data_stored":6554556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":377992517},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5806,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":17,"num_read_kb":16,"num_write":156,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.805385"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518423+0000","last_change":"2024-05-24T11:36:48.521973+0000","last_active":"2024-05-24T11:36:49.518423+0000","last_peered":"2024-05-24T11:36:49.518423+0000","last_clean":"2024-05-24T11:36:49.518423+0000","last_became_active":"2024-05-24T11:36:48.521587+0000","last_became_peered":"2024-05-24T11:36:48.521587+0000","last_unstale":"2024-05-24T11:36:49.518423+0000","last_undegraded":"2024-05-24T11:36:49.518423+0000","last_fullsized":"2024-05-24T11:36:49.518423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:25:29.234714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811230+0000","last_change":"2024-05-24T11:36:48.536290+0000","last_active":"2024-05-24T11:36:49.811230+0000","last_peered":"2024-05-24T11:36:49.811230+0000","last_clean":"2024-05-24T11:36:49.811230+0000","last_became_active":"2024-05-24T11:36:48.535961+0000","last_became_peered":"2024-05-24T11:36:48.535961+0000","last_unstale":"2024-05-24T11:36:49.811230+0000","last_undegraded":"2024-05-24T11:36:49.811230+0000","last_fullsized":"2024-05-24T11:36:49.811230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:52:37.951507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811216+0000","last_change":"2024-05-24T11:36:48.520039+0000","last_active":"2024-05-24T11:36:49.811216+0000","last_peered":"2024-05-24T11:36:49.811216+0000","last_clean":"2024-05-24T11:36:49.811216+0000","last_became_active":"2024-05-24T11:36:48.519809+0000","last_became_peered":"2024-05-24T11:36:48.519809+0000","last_unstale":"2024-05-24T11:36:49.811216+0000","last_undegraded":"2024-05-24T11:36:49.811216+0000","last_fullsized":"2024-05-24T11:36:49.811216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:44:20.133334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045745+0000","last_change":"2024-05-24T11:36:48.528100+0000","last_active":"2024-05-24T11:36:50.045745+0000","last_peered":"2024-05-24T11:36:50.045745+0000","last_clean":"2024-05-24T11:36:50.045745+0000","last_became_active":"2024-05-24T11:36:48.524918+0000","last_became_peered":"2024-05-24T11:36:48.524918+0000","last_unstale":"2024-05-24T11:36:50.045745+0000","last_undegraded":"2024-05-24T11:36:50.045745+0000","last_fullsized":"2024-05-24T11:36:50.045745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:20:34.628252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044676+0000","last_change":"2024-05-24T11:36:48.522735+0000","last_active":"2024-05-24T11:36:50.044676+0000","last_peered":"2024-05-24T11:36:50.044676+0000","last_clean":"2024-05-24T11:36:50.044676+0000","last_became_active":"2024-05-24T11:36:48.521657+0000","last_became_peered":"2024-05-24T11:36:48.521657+0000","last_unstale":"2024-05-24T11:36:50.044676+0000","last_undegraded":"2024-05-24T11:36:50.044676+0000","last_fullsized":"2024-05-24T11:36:50.044676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:03:32.056035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.835970+0000","last_change":"2024-05-24T11:36:48.534898+0000","last_active":"2024-05-24T11:36:48.835970+0000","last_peered":"2024-05-24T11:36:48.835970+0000","last_clean":"2024-05-24T11:36:48.835970+0000","last_became_active":"2024-05-24T11:36:48.534557+0000","last_became_peered":"2024-05-24T11:36:48.534557+0000","last_unstale":"2024-05-24T11:36:48.835970+0000","last_undegraded":"2024-05-24T11:36:48.835970+0000","last_fullsized":"2024-05-24T11:36:48.835970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:59:23.847279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.533394+0000","last_change":"2024-05-24T11:36:48.532435+0000","last_active":"2024-05-24T11:36:48.533394+0000","last_peered":"2024-05-24T11:36:48.533394+0000","last_clean":"2024-05-24T11:36:48.533394+0000","last_became_active":"2024-05-24T11:36:48.532058+0000","last_became_peered":"2024-05-24T11:36:48.532058+0000","last_unstale":"2024-05-24T11:36:48.533394+0000","last_undegraded":"2024-05-24T11:36:48.533394+0000","last_fullsized":"2024-05-24T11:36:48.533394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:33:26.178124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044447+0000","last_change":"2024-05-24T11:36:48.519054+0000","last_active":"2024-05-24T11:36:50.044447+0000","last_peered":"2024-05-24T11:36:50.044447+0000","last_clean":"2024-05-24T11:36:50.044447+0000","last_became_active":"2024-05-24T11:36:48.518756+0000","last_became_peered":"2024-05-24T11:36:48.518756+0000","last_unstale":"2024-05-24T11:36:50.044447+0000","last_undegraded":"2024-05-24T11:36:50.044447+0000","last_fullsized":"2024-05-24T11:36:50.044447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:56:19.599232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518761+0000","last_change":"2024-05-24T11:36:48.522937+0000","last_active":"2024-05-24T11:36:49.518761+0000","last_peered":"2024-05-24T11:36:49.518761+0000","last_clean":"2024-05-24T11:36:49.518761+0000","last_became_active":"2024-05-24T11:36:48.521531+0000","last_became_peered":"2024-05-24T11:36:48.521531+0000","last_unstale":"2024-05-24T11:36:49.518761+0000","last_undegraded":"2024-05-24T11:36:49.518761+0000","last_fullsized":"2024-05-24T11:36:49.518761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:16:54.392477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.836050+0000","last_change":"2024-05-24T11:36:48.536738+0000","last_active":"2024-05-24T11:36:48.836050+0000","last_peered":"2024-05-24T11:36:48.836050+0000","last_clean":"2024-05-24T11:36:48.836050+0000","last_became_active":"2024-05-24T11:36:48.536010+0000","last_became_peered":"2024-05-24T11:36:48.536010+0000","last_unstale":"2024-05-24T11:36:48.836050+0000","last_undegraded":"2024-05-24T11:36:48.836050+0000","last_fullsized":"2024-05-24T11:36:48.836050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:21:24.078676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.536403+0000","last_change":"2024-05-24T11:36:48.535036+0000","last_active":"2024-05-24T11:36:48.536403+0000","last_peered":"2024-05-24T11:36:48.536403+0000","last_clean":"2024-05-24T11:36:48.536403+0000","last_became_active":"2024-05-24T11:36:48.534232+0000","last_became_peered":"2024-05-24T11:36:48.534232+0000","last_unstale":"2024-05-24T11:36:48.536403+0000","last_undegraded":"2024-05-24T11:36:48.536403+0000","last_fullsized":"2024-05-24T11:36:48.536403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:56:28.111680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045685+0000","last_change":"2024-05-24T11:36:48.537837+0000","last_active":"2024-05-24T11:36:50.045685+0000","last_peered":"2024-05-24T11:36:50.045685+0000","last_clean":"2024-05-24T11:36:50.045685+0000","last_became_active":"2024-05-24T11:36:48.537570+0000","last_became_peered":"2024-05-24T11:36:48.537570+0000","last_unstale":"2024-05-24T11:36:50.045685+0000","last_undegraded":"2024-05-24T11:36:50.045685+0000","last_fullsized":"2024-05-24T11:36:50.045685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:47:14.769120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045004+0000","last_change":"2024-05-24T11:36:48.525333+0000","last_active":"2024-05-24T11:36:50.045004+0000","last_peered":"2024-05-24T11:36:50.045004+0000","last_clean":"2024-05-24T11:36:50.045004+0000","last_became_active":"2024-05-24T11:36:48.525146+0000","last_became_peered":"2024-05-24T11:36:48.525146+0000","last_unstale":"2024-05-24T11:36:50.045004+0000","last_undegraded":"2024-05-24T11:36:50.045004+0000","last_fullsized":"2024-05-24T11:36:50.045004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:22:58.950175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811013+0000","last_change":"2024-05-24T11:36:48.525438+0000","last_active":"2024-05-24T11:36:49.811013+0000","last_peered":"2024-05-24T11:36:49.811013+0000","last_clean":"2024-05-24T11:36:49.811013+0000","last_became_active":"2024-05-24T11:36:48.524866+0000","last_became_peered":"2024-05-24T11:36:48.524866+0000","last_unstale":"2024-05-24T11:36:49.811013+0000","last_undegraded":"2024-05-24T11:36:49.811013+0000","last_fullsized":"2024-05-24T11:36:49.811013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:39:11.826673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.810917+0000","last_change":"2024-05-24T11:36:48.529993+0000","last_active":"2024-05-24T11:36:49.810917+0000","last_peered":"2024-05-24T11:36:49.810917+0000","last_clean":"2024-05-24T11:36:49.810917+0000","last_became_active":"2024-05-24T11:36:48.529644+0000","last_became_peered":"2024-05-24T11:36:48.529644+0000","last_unstale":"2024-05-24T11:36:49.810917+0000","last_undegraded":"2024-05-24T11:36:49.810917+0000","last_fullsized":"2024-05-24T11:36:49.810917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:00:02.580642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518315+0000","last_change":"2024-05-24T11:36:48.521904+0000","last_active":"2024-05-24T11:36:49.518315+0000","last_peered":"2024-05-24T11:36:49.518315+0000","last_clean":"2024-05-24T11:36:49.518315+0000","last_became_active":"2024-05-24T11:36:48.521281+0000","last_became_peered":"2024-05-24T11:36:48.521281+0000","last_unstale":"2024-05-24T11:36:49.518315+0000","last_undegraded":"2024-05-24T11:36:49.518315+0000","last_fullsized":"2024-05-24T11:36:49.518315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:16:32.578404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518556+0000","last_change":"2024-05-24T11:36:48.526737+0000","last_active":"2024-05-24T11:36:49.518556+0000","last_peered":"2024-05-24T11:36:49.518556+0000","last_clean":"2024-05-24T11:36:49.518556+0000","last_became_active":"2024-05-24T11:36:48.526379+0000","last_became_peered":"2024-05-24T11:36:48.526379+0000","last_unstale":"2024-05-24T11:36:49.518556+0000","last_undegraded":"2024-05-24T11:36:49.518556+0000","last_fullsized":"2024-05-24T11:36:49.518556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:32:19.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.522182+0000","last_change":"2024-05-24T11:36:48.529315+0000","last_active":"2024-05-24T11:36:49.522182+0000","last_peered":"2024-05-24T11:36:49.522182+0000","last_clean":"2024-05-24T11:36:49.522182+0000","last_became_active":"2024-05-24T11:36:48.528963+0000","last_became_peered":"2024-05-24T11:36:48.528963+0000","last_unstale":"2024-05-24T11:36:49.522182+0000","last_undegraded":"2024-05-24T11:36:49.522182+0000","last_fullsized":"2024-05-24T11:36:49.522182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:54:23.922811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.544120+0000","last_change":"2024-05-24T11:36:48.520925+0000","last_active":"2024-05-24T11:36:49.544120+0000","last_peered":"2024-05-24T11:36:49.544120+0000","last_clean":"2024-05-24T11:36:49.544120+0000","last_became_active":"2024-05-24T11:36:48.520592+0000","last_became_peered":"2024-05-24T11:36:48.520592+0000","last_unstale":"2024-05-24T11:36:49.544120+0000","last_undegraded":"2024-05-24T11:36:49.544120+0000","last_fullsized":"2024-05-24T11:36:49.544120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:39:09.918746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.590609+0000","last_change":"2024-05-24T11:36:48.090815+0000","last_active":"2024-05-24T11:36:54.590609+0000","last_peered":"2024-05-24T11:36:54.590609+0000","last_clean":"2024-05-24T11:36:54.590609+0000","last_became_active":"2024-05-24T11:36:48.090481+0000","last_became_peered":"2024-05-24T11:36:48.090481+0000","last_unstale":"2024-05-24T11:36:54.590609+0000","last_undegraded":"2024-05-24T11:36:54.590609+0000","last_fullsized":"2024-05-24T11:36:54.590609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:32:40.850290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044588+0000","last_change":"2024-05-24T11:36:48.519300+0000","last_active":"2024-05-24T11:36:50.044588+0000","last_peered":"2024-05-24T11:36:50.044588+0000","last_clean":"2024-05-24T11:36:50.044588+0000","last_became_active":"2024-05-24T11:36:48.519004+0000","last_became_peered":"2024-05-24T11:36:48.519004+0000","last_unstale":"2024-05-24T11:36:50.044588+0000","last_undegraded":"2024-05-24T11:36:50.044588+0000","last_fullsized":"2024-05-24T11:36:50.044588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:15:58.098701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044840+0000","last_change":"2024-05-24T11:36:48.091695+0000","last_active":"2024-05-24T11:36:50.044840+0000","last_peered":"2024-05-24T11:36:50.044840+0000","last_clean":"2024-05-24T11:36:50.044840+0000","last_became_active":"2024-05-24T11:36:48.091379+0000","last_became_peered":"2024-05-24T11:36:48.091379+0000","last_unstale":"2024-05-24T11:36:50.044840+0000","last_undegraded":"2024-05-24T11:36:50.044840+0000","last_fullsized":"2024-05-24T11:36:50.044840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:07:55.039339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811338+0000","last_change":"2024-05-24T11:36:48.519313+0000","last_active":"2024-05-24T11:36:49.811338+0000","last_peered":"2024-05-24T11:36:49.811338+0000","last_clean":"2024-05-24T11:36:49.811338+0000","last_became_active":"2024-05-24T11:36:48.518968+0000","last_became_peered":"2024-05-24T11:36:48.518968+0000","last_unstale":"2024-05-24T11:36:49.811338+0000","last_undegraded":"2024-05-24T11:36:49.811338+0000","last_fullsized":"2024-05-24T11:36:49.811338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:51:23.712853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'11","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.590653+0000","last_change":"2024-05-24T11:36:48.096014+0000","last_active":"2024-05-24T11:36:54.590653+0000","last_peered":"2024-05-24T11:36:54.590653+0000","last_clean":"2024-05-24T11:36:54.590653+0000","last_became_active":"2024-05-24T11:36:48.095261+0000","last_became_peered":"2024-05-24T11:36:48.095261+0000","last_unstale":"2024-05-24T11:36:54.590653+0000","last_undegraded":"2024-05-24T11:36:54.590653+0000","last_fullsized":"2024-05-24T11:36:54.590653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:19:32.675770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518671+0000","last_change":"2024-05-24T11:36:48.524464+0000","last_active":"2024-05-24T11:36:49.518671+0000","last_peered":"2024-05-24T11:36:49.518671+0000","last_clean":"2024-05-24T11:36:49.518671+0000","last_became_active":"2024-05-24T11:36:48.524128+0000","last_became_peered":"2024-05-24T11:36:48.524128+0000","last_unstale":"2024-05-24T11:36:49.518671+0000","last_undegraded":"2024-05-24T11:36:49.518671+0000","last_fullsized":"2024-05-24T11:36:49.518671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:57:21.920169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.590552+0000","last_change":"2024-05-24T11:36:48.095739+0000","last_active":"2024-05-24T11:36:54.590552+0000","last_peered":"2024-05-24T11:36:54.590552+0000","last_clean":"2024-05-24T11:36:54.590552+0000","last_became_active":"2024-05-24T11:36:48.095313+0000","last_became_peered":"2024-05-24T11:36:48.095313+0000","last_unstale":"2024-05-24T11:36:54.590552+0000","last_undegraded":"2024-05-24T11:36:54.590552+0000","last_fullsized":"2024-05-24T11:36:54.590552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:59:55.283423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045238+0000","last_change":"2024-05-24T11:36:48.528788+0000","last_active":"2024-05-24T11:36:50.045238+0000","last_peered":"2024-05-24T11:36:50.045238+0000","last_clean":"2024-05-24T11:36:50.045238+0000","last_became_active":"2024-05-24T11:36:48.528358+0000","last_became_peered":"2024-05-24T11:36:48.528358+0000","last_unstale":"2024-05-24T11:36:50.045238+0000","last_undegraded":"2024-05-24T11:36:50.045238+0000","last_fullsized":"2024-05-24T11:36:50.045238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:19:37.870208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"77'5","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.601922+0000","last_change":"2024-05-24T11:36:48.093252+0000","last_active":"2024-05-24T11:36:51.601922+0000","last_peered":"2024-05-24T11:36:51.601922+0000","last_clean":"2024-05-24T11:36:51.601922+0000","last_became_active":"2024-05-24T11:36:48.092920+0000","last_became_peered":"2024-05-24T11:36:48.092920+0000","last_unstale":"2024-05-24T11:36:51.601922+0000","last_undegraded":"2024-05-24T11:36:51.601922+0000","last_fullsized":"2024-05-24T11:36:51.601922+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:51:39.032589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044926+0000","last_change":"2024-05-24T11:36:48.518108+0000","last_active":"2024-05-24T11:36:50.044926+0000","last_peered":"2024-05-24T11:36:50.044926+0000","last_clean":"2024-05-24T11:36:50.044926+0000","last_became_active":"2024-05-24T11:36:48.517786+0000","last_became_peered":"2024-05-24T11:36:48.517786+0000","last_unstale":"2024-05-24T11:36:50.044926+0000","last_undegraded":"2024-05-24T11:36:50.044926+0000","last_fullsized":"2024-05-24T11:36:50.044926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:41:44.779354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:53.618566+0000","last_change":"2024-05-24T11:36:48.082348+0000","last_active":"2024-05-24T11:36:53.618566+0000","last_peered":"2024-05-24T11:36:53.618566+0000","last_clean":"2024-05-24T11:36:53.618566+0000","last_became_active":"2024-05-24T11:36:48.081964+0000","last_became_peered":"2024-05-24T11:36:48.081964+0000","last_unstale":"2024-05-24T11:36:53.618566+0000","last_undegraded":"2024-05-24T11:36:53.618566+0000","last_fullsized":"2024-05-24T11:36:53.618566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:45:54.028437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045904+0000","last_change":"2024-05-24T11:36:48.528990+0000","last_active":"2024-05-24T11:36:50.045904+0000","last_peered":"2024-05-24T11:36:50.045904+0000","last_clean":"2024-05-24T11:36:50.045904+0000","last_became_active":"2024-05-24T11:36:48.528444+0000","last_became_peered":"2024-05-24T11:36:48.528444+0000","last_unstale":"2024-05-24T11:36:50.045904+0000","last_undegraded":"2024-05-24T11:36:50.045904+0000","last_fullsized":"2024-05-24T11:36:50.045904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:35:56.108817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045030+0000","last_change":"2024-05-24T11:36:48.079471+0000","last_active":"2024-05-24T11:36:50.045030+0000","last_peered":"2024-05-24T11:36:50.045030+0000","last_clean":"2024-05-24T11:36:50.045030+0000","last_became_active":"2024-05-24T11:36:48.079118+0000","last_became_peered":"2024-05-24T11:36:48.079118+0000","last_unstale":"2024-05-24T11:36:50.045030+0000","last_undegraded":"2024-05-24T11:36:50.045030+0000","last_fullsized":"2024-05-24T11:36:50.045030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:01:35.897437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044314+0000","last_change":"2024-05-24T11:36:48.517960+0000","last_active":"2024-05-24T11:36:50.044314+0000","last_peered":"2024-05-24T11:36:50.044314+0000","last_clean":"2024-05-24T11:36:50.044314+0000","last_became_active":"2024-05-24T11:36:48.517675+0000","last_became_peered":"2024-05-24T11:36:48.517675+0000","last_unstale":"2024-05-24T11:36:50.044314+0000","last_undegraded":"2024-05-24T11:36:50.044314+0000","last_fullsized":"2024-05-24T11:36:50.044314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:08:21.209080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599284+0000","last_change":"2024-05-24T11:36:48.091922+0000","last_active":"2024-05-24T11:36:51.599284+0000","last_peered":"2024-05-24T11:36:51.599284+0000","last_clean":"2024-05-24T11:36:51.599284+0000","last_became_active":"2024-05-24T11:36:48.091576+0000","last_became_peered":"2024-05-24T11:36:48.091576+0000","last_unstale":"2024-05-24T11:36:51.599284+0000","last_undegraded":"2024-05-24T11:36:51.599284+0000","last_fullsized":"2024-05-24T11:36:51.599284+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:01:39.386649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.521890+0000","last_change":"2024-05-24T11:36:48.523769+0000","last_active":"2024-05-24T11:36:49.521890+0000","last_peered":"2024-05-24T11:36:49.521890+0000","last_clean":"2024-05-24T11:36:49.521890+0000","last_became_active":"2024-05-24T11:36:48.523394+0000","last_became_peered":"2024-05-24T11:36:48.523394+0000","last_unstale":"2024-05-24T11:36:49.521890+0000","last_undegraded":"2024-05-24T11:36:49.521890+0000","last_fullsized":"2024-05-24T11:36:49.521890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:03:49.209246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.600064+0000","last_change":"2024-05-24T11:36:48.089651+0000","last_active":"2024-05-24T11:36:51.600064+0000","last_peered":"2024-05-24T11:36:51.600064+0000","last_clean":"2024-05-24T11:36:51.600064+0000","last_became_active":"2024-05-24T11:36:48.089195+0000","last_became_peered":"2024-05-24T11:36:48.089195+0000","last_unstale":"2024-05-24T11:36:51.600064+0000","last_undegraded":"2024-05-24T11:36:51.600064+0000","last_fullsized":"2024-05-24T11:36:51.600064+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:30:47.730594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518732+0000","last_change":"2024-05-24T11:36:48.531529+0000","last_active":"2024-05-24T11:36:49.518732+0000","last_peered":"2024-05-24T11:36:49.518732+0000","last_clean":"2024-05-24T11:36:49.518732+0000","last_became_active":"2024-05-24T11:36:48.530969+0000","last_became_peered":"2024-05-24T11:36:48.530969+0000","last_unstale":"2024-05-24T11:36:49.518732+0000","last_undegraded":"2024-05-24T11:36:49.518732+0000","last_fullsized":"2024-05-24T11:36:49.518732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:05:24.929278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599141+0000","last_change":"2024-05-24T11:36:48.094249+0000","last_active":"2024-05-24T11:36:51.599141+0000","last_peered":"2024-05-24T11:36:51.599141+0000","last_clean":"2024-05-24T11:36:51.599141+0000","last_became_active":"2024-05-24T11:36:48.093835+0000","last_became_peered":"2024-05-24T11:36:48.093835+0000","last_unstale":"2024-05-24T11:36:51.599141+0000","last_undegraded":"2024-05-24T11:36:51.599141+0000","last_fullsized":"2024-05-24T11:36:51.599141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:41:47.660513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811338+0000","last_change":"2024-05-24T11:36:48.531745+0000","last_active":"2024-05-24T11:36:49.811338+0000","last_peered":"2024-05-24T11:36:49.811338+0000","last_clean":"2024-05-24T11:36:49.811338+0000","last_became_active":"2024-05-24T11:36:48.531459+0000","last_became_peered":"2024-05-24T11:36:48.531459+0000","last_unstale":"2024-05-24T11:36:49.811338+0000","last_undegraded":"2024-05-24T11:36:49.811338+0000","last_fullsized":"2024-05-24T11:36:49.811338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:06:41.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.598752+0000","last_change":"2024-05-24T11:36:48.089567+0000","last_active":"2024-05-24T11:36:51.598752+0000","last_peered":"2024-05-24T11:36:51.598752+0000","last_clean":"2024-05-24T11:36:51.598752+0000","last_became_active":"2024-05-24T11:36:48.089248+0000","last_became_peered":"2024-05-24T11:36:48.089248+0000","last_unstale":"2024-05-24T11:36:51.598752+0000","last_undegraded":"2024-05-24T11:36:51.598752+0000","last_fullsized":"2024-05-24T11:36:51.598752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:49:06.307799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518786+0000","last_change":"2024-05-24T11:36:48.529531+0000","last_active":"2024-05-24T11:36:49.518786+0000","last_peered":"2024-05-24T11:36:49.518786+0000","last_clean":"2024-05-24T11:36:49.518786+0000","last_became_active":"2024-05-24T11:36:48.529222+0000","last_became_peered":"2024-05-24T11:36:48.529222+0000","last_unstale":"2024-05-24T11:36:49.518786+0000","last_undegraded":"2024-05-24T11:36:49.518786+0000","last_fullsized":"2024-05-24T11:36:49.518786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:18:49.571222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.600866+0000","last_change":"2024-05-24T11:36:48.086123+0000","last_active":"2024-05-24T11:36:51.600866+0000","last_peered":"2024-05-24T11:36:51.600866+0000","last_clean":"2024-05-24T11:36:51.600866+0000","last_became_active":"2024-05-24T11:36:48.085681+0000","last_became_peered":"2024-05-24T11:36:48.085681+0000","last_unstale":"2024-05-24T11:36:51.600866+0000","last_undegraded":"2024-05-24T11:36:51.600866+0000","last_fullsized":"2024-05-24T11:36:51.600866+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:17:12.646332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045057+0000","last_change":"2024-05-24T11:36:48.524883+0000","last_active":"2024-05-24T11:36:50.045057+0000","last_peered":"2024-05-24T11:36:50.045057+0000","last_clean":"2024-05-24T11:36:50.045057+0000","last_became_active":"2024-05-24T11:36:48.524578+0000","last_became_peered":"2024-05-24T11:36:48.524578+0000","last_unstale":"2024-05-24T11:36:50.045057+0000","last_undegraded":"2024-05-24T11:36:50.045057+0000","last_fullsized":"2024-05-24T11:36:50.045057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:54:15.419099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.610572+0000","last_change":"2024-05-24T11:36:48.090437+0000","last_active":"2024-05-24T11:36:52.610572+0000","last_peered":"2024-05-24T11:36:52.610572+0000","last_clean":"2024-05-24T11:36:52.610572+0000","last_became_active":"2024-05-24T11:36:48.088745+0000","last_became_peered":"2024-05-24T11:36:48.088745+0000","last_unstale":"2024-05-24T11:36:52.610572+0000","last_undegraded":"2024-05-24T11:36:52.610572+0000","last_fullsized":"2024-05-24T11:36:52.610572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:12:48.829945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044821+0000","last_change":"2024-05-24T11:36:48.525723+0000","last_active":"2024-05-24T11:36:50.044821+0000","last_peered":"2024-05-24T11:36:50.044821+0000","last_clean":"2024-05-24T11:36:50.044821+0000","last_became_active":"2024-05-24T11:36:48.525267+0000","last_became_peered":"2024-05-24T11:36:48.525267+0000","last_unstale":"2024-05-24T11:36:50.044821+0000","last_undegraded":"2024-05-24T11:36:50.044821+0000","last_fullsized":"2024-05-24T11:36:50.044821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:27:59.404214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.600200+0000","last_change":"2024-05-24T11:36:48.094426+0000","last_active":"2024-05-24T11:36:51.600200+0000","last_peered":"2024-05-24T11:36:51.600200+0000","last_clean":"2024-05-24T11:36:51.600200+0000","last_became_active":"2024-05-24T11:36:48.094169+0000","last_became_peered":"2024-05-24T11:36:48.094169+0000","last_unstale":"2024-05-24T11:36:51.600200+0000","last_undegraded":"2024-05-24T11:36:51.600200+0000","last_fullsized":"2024-05-24T11:36:51.600200+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:31:02.682818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.531031+0000","last_change":"2024-05-24T11:36:48.529851+0000","last_active":"2024-05-24T11:36:48.531031+0000","last_peered":"2024-05-24T11:36:48.531031+0000","last_clean":"2024-05-24T11:36:48.531031+0000","last_became_active":"2024-05-24T11:36:48.529540+0000","last_became_peered":"2024-05-24T11:36:48.529540+0000","last_unstale":"2024-05-24T11:36:48.531031+0000","last_undegraded":"2024-05-24T11:36:48.531031+0000","last_fullsized":"2024-05-24T11:36:48.531031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:22:12.101608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'5","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.583517+0000","last_change":"2024-05-24T11:36:48.090611+0000","last_active":"2024-05-24T11:36:54.583517+0000","last_peered":"2024-05-24T11:36:54.583517+0000","last_clean":"2024-05-24T11:36:54.583517+0000","last_became_active":"2024-05-24T11:36:48.090283+0000","last_became_peered":"2024-05-24T11:36:48.090283+0000","last_unstale":"2024-05-24T11:36:54.583517+0000","last_undegraded":"2024-05-24T11:36:54.583517+0000","last_fullsized":"2024-05-24T11:36:54.583517+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:24:57.272371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.537097+0000","last_change":"2024-05-24T11:36:48.536763+0000","last_active":"2024-05-24T11:36:48.537097+0000","last_peered":"2024-05-24T11:36:48.537097+0000","last_clean":"2024-05-24T11:36:48.537097+0000","last_became_active":"2024-05-24T11:36:48.536398+0000","last_became_peered":"2024-05-24T11:36:48.536398+0000","last_unstale":"2024-05-24T11:36:48.537097+0000","last_undegraded":"2024-05-24T11:36:48.537097+0000","last_fullsized":"2024-05-24T11:36:48.537097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:13:58.446126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.581083+0000","last_change":"2024-05-24T11:36:48.080590+0000","last_active":"2024-05-24T11:36:54.581083+0000","last_peered":"2024-05-24T11:36:54.581083+0000","last_clean":"2024-05-24T11:36:54.581083+0000","last_became_active":"2024-05-24T11:36:48.080192+0000","last_became_peered":"2024-05-24T11:36:48.080192+0000","last_unstale":"2024-05-24T11:36:54.581083+0000","last_undegraded":"2024-05-24T11:36:54.581083+0000","last_fullsized":"2024-05-24T11:36:54.581083+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:13:35.452878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044986+0000","last_change":"2024-05-24T11:36:48.530444+0000","last_active":"2024-05-24T11:36:50.044986+0000","last_peered":"2024-05-24T11:36:50.044986+0000","last_clean":"2024-05-24T11:36:50.044986+0000","last_became_active":"2024-05-24T11:36:48.530063+0000","last_became_peered":"2024-05-24T11:36:48.530063+0000","last_unstale":"2024-05-24T11:36:50.044986+0000","last_undegraded":"2024-05-24T11:36:50.044986+0000","last_fullsized":"2024-05-24T11:36:50.044986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:23:12.380170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.581432+0000","last_change":"2024-05-24T11:36:48.086566+0000","last_active":"2024-05-24T11:36:54.581432+0000","last_peered":"2024-05-24T11:36:54.581432+0000","last_clean":"2024-05-24T11:36:54.581432+0000","last_became_active":"2024-05-24T11:36:48.086213+0000","last_became_peered":"2024-05-24T11:36:48.086213+0000","last_unstale":"2024-05-24T11:36:54.581432+0000","last_undegraded":"2024-05-24T11:36:54.581432+0000","last_fullsized":"2024-05-24T11:36:54.581432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:09:53.744478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518496+0000","last_change":"2024-05-24T11:36:48.528746+0000","last_active":"2024-05-24T11:36:49.518496+0000","last_peered":"2024-05-24T11:36:49.518496+0000","last_clean":"2024-05-24T11:36:49.518496+0000","last_became_active":"2024-05-24T11:36:48.528399+0000","last_became_peered":"2024-05-24T11:36:48.528399+0000","last_unstale":"2024-05-24T11:36:49.518496+0000","last_undegraded":"2024-05-24T11:36:49.518496+0000","last_fullsized":"2024-05-24T11:36:49.518496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:51:28.838280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.790552+0000","last_change":"2024-05-24T11:36:48.084268+0000","last_active":"2024-05-24T11:36:50.790552+0000","last_peered":"2024-05-24T11:36:50.790552+0000","last_clean":"2024-05-24T11:36:50.790552+0000","last_became_active":"2024-05-24T11:36:48.083968+0000","last_became_peered":"2024-05-24T11:36:48.083968+0000","last_unstale":"2024-05-24T11:36:50.790552+0000","last_undegraded":"2024-05-24T11:36:50.790552+0000","last_fullsized":"2024-05-24T11:36:50.790552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:02:00.989621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.836040+0000","last_change":"2024-05-24T11:36:48.523784+0000","last_active":"2024-05-24T11:36:48.836040+0000","last_peered":"2024-05-24T11:36:48.836040+0000","last_clean":"2024-05-24T11:36:48.836040+0000","last_became_active":"2024-05-24T11:36:48.523430+0000","last_became_peered":"2024-05-24T11:36:48.523430+0000","last_unstale":"2024-05-24T11:36:48.836040+0000","last_undegraded":"2024-05-24T11:36:48.836040+0000","last_fullsized":"2024-05-24T11:36:48.836040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:02:33.070861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.048190+0000","last_change":"2024-05-24T11:36:48.092166+0000","last_active":"2024-05-24T11:36:50.048190+0000","last_peered":"2024-05-24T11:36:50.048190+0000","last_clean":"2024-05-24T11:36:50.048190+0000","last_became_active":"2024-05-24T11:36:48.091682+0000","last_became_peered":"2024-05-24T11:36:48.091682+0000","last_unstale":"2024-05-24T11:36:50.048190+0000","last_undegraded":"2024-05-24T11:36:50.048190+0000","last_fullsized":"2024-05-24T11:36:50.048190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:14:53.083387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.835997+0000","last_change":"2024-05-24T11:36:48.534618+0000","last_active":"2024-05-24T11:36:48.835997+0000","last_peered":"2024-05-24T11:36:48.835997+0000","last_clean":"2024-05-24T11:36:48.835997+0000","last_became_active":"2024-05-24T11:36:48.534251+0000","last_became_peered":"2024-05-24T11:36:48.534251+0000","last_unstale":"2024-05-24T11:36:48.835997+0000","last_undegraded":"2024-05-24T11:36:48.835997+0000","last_fullsized":"2024-05-24T11:36:48.835997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:14:46.889921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045023+0000","last_change":"2024-05-24T11:36:48.528214+0000","last_active":"2024-05-24T11:36:50.045023+0000","last_peered":"2024-05-24T11:36:50.045023+0000","last_clean":"2024-05-24T11:36:50.045023+0000","last_became_active":"2024-05-24T11:36:48.526711+0000","last_became_peered":"2024-05-24T11:36:48.526711+0000","last_unstale":"2024-05-24T11:36:50.045023+0000","last_undegraded":"2024-05-24T11:36:50.045023+0000","last_fullsized":"2024-05-24T11:36:50.045023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:06:27.935139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045003+0000","last_change":"2024-05-24T11:36:48.089263+0000","last_active":"2024-05-24T11:36:50.045003+0000","last_peered":"2024-05-24T11:36:50.045003+0000","last_clean":"2024-05-24T11:36:50.045003+0000","last_became_active":"2024-05-24T11:36:48.087905+0000","last_became_peered":"2024-05-24T11:36:48.087905+0000","last_unstale":"2024-05-24T11:36:50.045003+0000","last_undegraded":"2024-05-24T11:36:50.045003+0000","last_fullsized":"2024-05-24T11:36:50.045003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:06:37.941039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044877+0000","last_change":"2024-05-24T11:36:48.517569+0000","last_active":"2024-05-24T11:36:50.044877+0000","last_peered":"2024-05-24T11:36:50.044877+0000","last_clean":"2024-05-24T11:36:50.044877+0000","last_became_active":"2024-05-24T11:36:48.517202+0000","last_became_peered":"2024-05-24T11:36:48.517202+0000","last_unstale":"2024-05-24T11:36:50.044877+0000","last_undegraded":"2024-05-24T11:36:50.044877+0000","last_fullsized":"2024-05-24T11:36:50.044877+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:38:50.778284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.582717+0000","last_change":"2024-05-24T11:36:48.077543+0000","last_active":"2024-05-24T11:36:54.582717+0000","last_peered":"2024-05-24T11:36:54.582717+0000","last_clean":"2024-05-24T11:36:54.582717+0000","last_became_active":"2024-05-24T11:36:48.077058+0000","last_became_peered":"2024-05-24T11:36:48.077058+0000","last_unstale":"2024-05-24T11:36:54.582717+0000","last_undegraded":"2024-05-24T11:36:54.582717+0000","last_fullsized":"2024-05-24T11:36:54.582717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:19:49.486922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":327,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.515699+0000","last_change":"2024-05-24T11:36:00.577887+0000","last_active":"2024-05-24T11:36:48.515699+0000","last_peered":"2024-05-24T11:36:48.515699+0000","last_clean":"2024-05-24T11:36:48.515699+0000","last_became_active":"2024-05-24T11:36:00.577291+0000","last_became_peered":"2024-05-24T11:36:00.577291+0000","last_unstale":"2024-05-24T11:36:48.515699+0000","last_undegraded":"2024-05-24T11:36:48.515699+0000","last_fullsized":"2024-05-24T11:36:48.515699+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:33:34.896223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:33:34.896223+0000","last_clean_scrub_stamp":"2024-05-24T11:33:34.896223+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:11:36.393700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044963+0000","last_change":"2024-05-24T11:36:48.527294+0000","last_active":"2024-05-24T11:36:50.044963+0000","last_peered":"2024-05-24T11:36:50.044963+0000","last_clean":"2024-05-24T11:36:50.044963+0000","last_became_active":"2024-05-24T11:36:48.526983+0000","last_became_peered":"2024-05-24T11:36:48.526983+0000","last_unstale":"2024-05-24T11:36:50.044963+0000","last_undegraded":"2024-05-24T11:36:50.044963+0000","last_fullsized":"2024-05-24T11:36:50.044963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:23:26.492502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.610703+0000","last_change":"2024-05-24T11:36:48.097290+0000","last_active":"2024-05-24T11:36:52.610703+0000","last_peered":"2024-05-24T11:36:52.610703+0000","last_clean":"2024-05-24T11:36:52.610703+0000","last_became_active":"2024-05-24T11:36:48.096942+0000","last_became_peered":"2024-05-24T11:36:48.096942+0000","last_unstale":"2024-05-24T11:36:52.610703+0000","last_undegraded":"2024-05-24T11:36:52.610703+0000","last_fullsized":"2024-05-24T11:36:52.610703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:24:20.221023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.579815+0000","last_change":"2024-05-24T11:36:48.079294+0000","last_active":"2024-05-24T11:36:54.579815+0000","last_peered":"2024-05-24T11:36:54.579815+0000","last_clean":"2024-05-24T11:36:54.579815+0000","last_became_active":"2024-05-24T11:36:48.078919+0000","last_became_peered":"2024-05-24T11:36:48.078919+0000","last_unstale":"2024-05-24T11:36:54.579815+0000","last_undegraded":"2024-05-24T11:36:54.579815+0000","last_fullsized":"2024-05-24T11:36:54.579815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:22:32.262674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045222+0000","last_change":"2024-05-24T11:36:48.524666+0000","last_active":"2024-05-24T11:36:50.045222+0000","last_peered":"2024-05-24T11:36:50.045222+0000","last_clean":"2024-05-24T11:36:50.045222+0000","last_became_active":"2024-05-24T11:36:48.524347+0000","last_became_peered":"2024-05-24T11:36:48.524347+0000","last_unstale":"2024-05-24T11:36:50.045222+0000","last_undegraded":"2024-05-24T11:36:50.045222+0000","last_fullsized":"2024-05-24T11:36:50.045222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:14:13.716982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:53.617908+0000","last_change":"2024-05-24T11:36:48.085665+0000","last_active":"2024-05-24T11:36:53.617908+0000","last_peered":"2024-05-24T11:36:53.617908+0000","last_clean":"2024-05-24T11:36:53.617908+0000","last_became_active":"2024-05-24T11:36:48.085344+0000","last_became_peered":"2024-05-24T11:36:48.085344+0000","last_unstale":"2024-05-24T11:36:53.617908+0000","last_undegraded":"2024-05-24T11:36:53.617908+0000","last_fullsized":"2024-05-24T11:36:53.617908+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:11:29.930656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044736+0000","last_change":"2024-05-24T11:36:48.523060+0000","last_active":"2024-05-24T11:36:50.044736+0000","last_peered":"2024-05-24T11:36:50.044736+0000","last_clean":"2024-05-24T11:36:50.044736+0000","last_became_active":"2024-05-24T11:36:48.522712+0000","last_became_peered":"2024-05-24T11:36:48.522712+0000","last_unstale":"2024-05-24T11:36:50.044736+0000","last_undegraded":"2024-05-24T11:36:50.044736+0000","last_fullsized":"2024-05-24T11:36:50.044736+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:07:54.311695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.589339+0000","last_change":"2024-05-24T11:36:48.092809+0000","last_active":"2024-05-24T11:36:54.589339+0000","last_peered":"2024-05-24T11:36:54.589339+0000","last_clean":"2024-05-24T11:36:54.589339+0000","last_became_active":"2024-05-24T11:36:48.092447+0000","last_became_peered":"2024-05-24T11:36:48.092447+0000","last_unstale":"2024-05-24T11:36:54.589339+0000","last_undegraded":"2024-05-24T11:36:54.589339+0000","last_fullsized":"2024-05-24T11:36:54.589339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:27:12.916761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811262+0000","last_change":"2024-05-24T11:36:48.528514+0000","last_active":"2024-05-24T11:36:49.811262+0000","last_peered":"2024-05-24T11:36:49.811262+0000","last_clean":"2024-05-24T11:36:49.811262+0000","last_became_active":"2024-05-24T11:36:48.528244+0000","last_became_peered":"2024-05-24T11:36:48.528244+0000","last_unstale":"2024-05-24T11:36:49.811262+0000","last_undegraded":"2024-05-24T11:36:49.811262+0000","last_fullsized":"2024-05-24T11:36:49.811262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:53:05.993805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518367+0000","last_change":"2024-05-24T11:36:48.094084+0000","last_active":"2024-05-24T11:36:49.518367+0000","last_peered":"2024-05-24T11:36:49.518367+0000","last_clean":"2024-05-24T11:36:49.518367+0000","last_became_active":"2024-05-24T11:36:48.093254+0000","last_became_peered":"2024-05-24T11:36:48.093254+0000","last_unstale":"2024-05-24T11:36:49.518367+0000","last_undegraded":"2024-05-24T11:36:49.518367+0000","last_fullsized":"2024-05-24T11:36:49.518367+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:56:18.896200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044843+0000","last_change":"2024-05-24T11:36:48.522292+0000","last_active":"2024-05-24T11:36:50.044843+0000","last_peered":"2024-05-24T11:36:50.044843+0000","last_clean":"2024-05-24T11:36:50.044843+0000","last_became_active":"2024-05-24T11:36:48.521356+0000","last_became_peered":"2024-05-24T11:36:48.521356+0000","last_unstale":"2024-05-24T11:36:50.044843+0000","last_undegraded":"2024-05-24T11:36:50.044843+0000","last_fullsized":"2024-05-24T11:36:50.044843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:51:44.805455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044445+0000","last_change":"2024-05-24T11:36:48.086209+0000","last_active":"2024-05-24T11:36:50.044445+0000","last_peered":"2024-05-24T11:36:50.044445+0000","last_clean":"2024-05-24T11:36:50.044445+0000","last_became_active":"2024-05-24T11:36:48.084978+0000","last_became_peered":"2024-05-24T11:36:48.084978+0000","last_unstale":"2024-05-24T11:36:50.044445+0000","last_undegraded":"2024-05-24T11:36:50.044445+0000","last_fullsized":"2024-05-24T11:36:50.044445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:26:39.181832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518178+0000","last_change":"2024-05-24T11:36:48.520890+0000","last_active":"2024-05-24T11:36:49.518178+0000","last_peered":"2024-05-24T11:36:49.518178+0000","last_clean":"2024-05-24T11:36:49.518178+0000","last_became_active":"2024-05-24T11:36:48.519860+0000","last_became_peered":"2024-05-24T11:36:48.519860+0000","last_unstale":"2024-05-24T11:36:49.518178+0000","last_undegraded":"2024-05-24T11:36:49.518178+0000","last_fullsized":"2024-05-24T11:36:49.518178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:54:35.691695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.512436+0000","last_change":"2024-05-24T11:36:48.089884+0000","last_active":"2024-05-24T11:36:48.512436+0000","last_peered":"2024-05-24T11:36:48.512436+0000","last_clean":"2024-05-24T11:36:48.512436+0000","last_became_active":"2024-05-24T11:36:48.089386+0000","last_became_peered":"2024-05-24T11:36:48.089386+0000","last_unstale":"2024-05-24T11:36:48.512436+0000","last_undegraded":"2024-05-24T11:36:48.512436+0000","last_fullsized":"2024-05-24T11:36:48.512436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:56:14.463210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044730+0000","last_change":"2024-05-24T11:36:48.515178+0000","last_active":"2024-05-24T11:36:50.044730+0000","last_peered":"2024-05-24T11:36:50.044730+0000","last_clean":"2024-05-24T11:36:50.044730+0000","last_became_active":"2024-05-24T11:36:48.514277+0000","last_became_peered":"2024-05-24T11:36:48.514277+0000","last_unstale":"2024-05-24T11:36:50.044730+0000","last_undegraded":"2024-05-24T11:36:50.044730+0000","last_fullsized":"2024-05-24T11:36:50.044730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:46:38.526893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599557+0000","last_change":"2024-05-24T11:36:48.095493+0000","last_active":"2024-05-24T11:36:51.599557+0000","last_peered":"2024-05-24T11:36:51.599557+0000","last_clean":"2024-05-24T11:36:51.599557+0000","last_became_active":"2024-05-24T11:36:48.094863+0000","last_became_peered":"2024-05-24T11:36:48.094863+0000","last_unstale":"2024-05-24T11:36:51.599557+0000","last_undegraded":"2024-05-24T11:36:51.599557+0000","last_fullsized":"2024-05-24T11:36:51.599557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:45:06.827169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.530486+0000","last_change":"2024-05-24T11:36:48.528024+0000","last_active":"2024-05-24T11:36:48.530486+0000","last_peered":"2024-05-24T11:36:48.530486+0000","last_clean":"2024-05-24T11:36:48.530486+0000","last_became_active":"2024-05-24T11:36:48.527689+0000","last_became_peered":"2024-05-24T11:36:48.527689+0000","last_unstale":"2024-05-24T11:36:48.530486+0000","last_undegraded":"2024-05-24T11:36:48.530486+0000","last_fullsized":"2024-05-24T11:36:48.530486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:07:50.186661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.612656+0000","last_change":"2024-05-24T11:36:48.090660+0000","last_active":"2024-05-24T11:36:52.612656+0000","last_peered":"2024-05-24T11:36:52.612656+0000","last_clean":"2024-05-24T11:36:52.612656+0000","last_became_active":"2024-05-24T11:36:48.090298+0000","last_became_peered":"2024-05-24T11:36:48.090298+0000","last_unstale":"2024-05-24T11:36:52.612656+0000","last_undegraded":"2024-05-24T11:36:52.612656+0000","last_fullsized":"2024-05-24T11:36:52.612656+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:07:30.366506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.528160+0000","last_change":"2024-05-24T11:36:48.526287+0000","last_active":"2024-05-24T11:36:48.528160+0000","last_peered":"2024-05-24T11:36:48.528160+0000","last_clean":"2024-05-24T11:36:48.528160+0000","last_became_active":"2024-05-24T11:36:48.525951+0000","last_became_peered":"2024-05-24T11:36:48.525951+0000","last_unstale":"2024-05-24T11:36:48.528160+0000","last_undegraded":"2024-05-24T11:36:48.528160+0000","last_fullsized":"2024-05-24T11:36:48.528160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:38:11.562370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045141+0000","last_change":"2024-05-24T11:36:48.092808+0000","last_active":"2024-05-24T11:36:50.045141+0000","last_peered":"2024-05-24T11:36:50.045141+0000","last_clean":"2024-05-24T11:36:50.045141+0000","last_became_active":"2024-05-24T11:36:48.092466+0000","last_became_peered":"2024-05-24T11:36:48.092466+0000","last_unstale":"2024-05-24T11:36:50.045141+0000","last_undegraded":"2024-05-24T11:36:50.045141+0000","last_fullsized":"2024-05-24T11:36:50.045141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:27:08.423449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.535337+0000","last_change":"2024-05-24T11:36:48.533175+0000","last_active":"2024-05-24T11:36:48.535337+0000","last_peered":"2024-05-24T11:36:48.535337+0000","last_clean":"2024-05-24T11:36:48.535337+0000","last_became_active":"2024-05-24T11:36:48.532779+0000","last_became_peered":"2024-05-24T11:36:48.532779+0000","last_unstale":"2024-05-24T11:36:48.535337+0000","last_undegraded":"2024-05-24T11:36:48.535337+0000","last_fullsized":"2024-05-24T11:36:48.535337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:39:40.041747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045161+0000","last_change":"2024-05-24T11:36:48.080825+0000","last_active":"2024-05-24T11:36:50.045161+0000","last_peered":"2024-05-24T11:36:50.045161+0000","last_clean":"2024-05-24T11:36:50.045161+0000","last_became_active":"2024-05-24T11:36:48.080462+0000","last_became_peered":"2024-05-24T11:36:48.080462+0000","last_unstale":"2024-05-24T11:36:50.045161+0000","last_undegraded":"2024-05-24T11:36:50.045161+0000","last_fullsized":"2024-05-24T11:36:50.045161+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:05:25.665222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.536673+0000","last_change":"2024-05-24T11:36:48.535094+0000","last_active":"2024-05-24T11:36:48.536673+0000","last_peered":"2024-05-24T11:36:48.536673+0000","last_clean":"2024-05-24T11:36:48.536673+0000","last_became_active":"2024-05-24T11:36:48.534670+0000","last_became_peered":"2024-05-24T11:36:48.534670+0000","last_unstale":"2024-05-24T11:36:48.536673+0000","last_undegraded":"2024-05-24T11:36:48.536673+0000","last_fullsized":"2024-05-24T11:36:48.536673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:24:49.492069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518418+0000","last_change":"2024-05-24T11:36:48.521113+0000","last_active":"2024-05-24T11:36:49.518418+0000","last_peered":"2024-05-24T11:36:49.518418+0000","last_clean":"2024-05-24T11:36:49.518418+0000","last_became_active":"2024-05-24T11:36:48.520379+0000","last_became_peered":"2024-05-24T11:36:48.520379+0000","last_unstale":"2024-05-24T11:36:49.518418+0000","last_undegraded":"2024-05-24T11:36:49.518418+0000","last_fullsized":"2024-05-24T11:36:49.518418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:07:17.112794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.543923+0000","last_change":"2024-05-24T11:36:48.529329+0000","last_active":"2024-05-24T11:36:49.543923+0000","last_peered":"2024-05-24T11:36:49.543923+0000","last_clean":"2024-05-24T11:36:49.543923+0000","last_became_active":"2024-05-24T11:36:48.528994+0000","last_became_peered":"2024-05-24T11:36:48.528994+0000","last_unstale":"2024-05-24T11:36:49.543923+0000","last_undegraded":"2024-05-24T11:36:49.543923+0000","last_fullsized":"2024-05-24T11:36:49.543923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:23:37.166318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.543981+0000","last_change":"2024-05-24T11:36:48.519429+0000","last_active":"2024-05-24T11:36:49.543981+0000","last_peered":"2024-05-24T11:36:49.543981+0000","last_clean":"2024-05-24T11:36:49.543981+0000","last_became_active":"2024-05-24T11:36:48.519085+0000","last_became_peered":"2024-05-24T11:36:48.519085+0000","last_unstale":"2024-05-24T11:36:49.543981+0000","last_undegraded":"2024-05-24T11:36:49.543981+0000","last_fullsized":"2024-05-24T11:36:49.543981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:41:55.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811057+0000","last_change":"2024-05-24T11:36:48.528932+0000","last_active":"2024-05-24T11:36:49.811057+0000","last_peered":"2024-05-24T11:36:49.811057+0000","last_clean":"2024-05-24T11:36:49.811057+0000","last_became_active":"2024-05-24T11:36:48.528413+0000","last_became_peered":"2024-05-24T11:36:48.528413+0000","last_unstale":"2024-05-24T11:36:49.811057+0000","last_undegraded":"2024-05-24T11:36:49.811057+0000","last_fullsized":"2024-05-24T11:36:49.811057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:55:47.051779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045272+0000","last_change":"2024-05-24T11:36:48.526905+0000","last_active":"2024-05-24T11:36:50.045272+0000","last_peered":"2024-05-24T11:36:50.045272+0000","last_clean":"2024-05-24T11:36:50.045272+0000","last_became_active":"2024-05-24T11:36:48.526608+0000","last_became_peered":"2024-05-24T11:36:48.526608+0000","last_unstale":"2024-05-24T11:36:50.045272+0000","last_undegraded":"2024-05-24T11:36:50.045272+0000","last_fullsized":"2024-05-24T11:36:50.045272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:32:04.831995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518371+0000","last_change":"2024-05-24T11:36:48.523597+0000","last_active":"2024-05-24T11:36:49.518371+0000","last_peered":"2024-05-24T11:36:49.518371+0000","last_clean":"2024-05-24T11:36:49.518371+0000","last_became_active":"2024-05-24T11:36:48.523279+0000","last_became_peered":"2024-05-24T11:36:48.523279+0000","last_unstale":"2024-05-24T11:36:49.518371+0000","last_undegraded":"2024-05-24T11:36:49.518371+0000","last_fullsized":"2024-05-24T11:36:49.518371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:49:46.063341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.544125+0000","last_change":"2024-05-24T11:36:48.526836+0000","last_active":"2024-05-24T11:36:49.544125+0000","last_peered":"2024-05-24T11:36:49.544125+0000","last_clean":"2024-05-24T11:36:49.544125+0000","last_became_active":"2024-05-24T11:36:48.523838+0000","last_became_peered":"2024-05-24T11:36:48.523838+0000","last_unstale":"2024-05-24T11:36:49.544125+0000","last_undegraded":"2024-05-24T11:36:49.544125+0000","last_fullsized":"2024-05-24T11:36:49.544125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:01:13.223240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044611+0000","last_change":"2024-05-24T11:36:48.532826+0000","last_active":"2024-05-24T11:36:50.044611+0000","last_peered":"2024-05-24T11:36:50.044611+0000","last_clean":"2024-05-24T11:36:50.044611+0000","last_became_active":"2024-05-24T11:36:48.532526+0000","last_became_peered":"2024-05-24T11:36:48.532526+0000","last_unstale":"2024-05-24T11:36:50.044611+0000","last_undegraded":"2024-05-24T11:36:50.044611+0000","last_fullsized":"2024-05-24T11:36:50.044611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:15:10.133493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044873+0000","last_change":"2024-05-24T11:36:48.527805+0000","last_active":"2024-05-24T11:36:50.044873+0000","last_peered":"2024-05-24T11:36:50.044873+0000","last_clean":"2024-05-24T11:36:50.044873+0000","last_became_active":"2024-05-24T11:36:48.526047+0000","last_became_peered":"2024-05-24T11:36:48.526047+0000","last_unstale":"2024-05-24T11:36:50.044873+0000","last_undegraded":"2024-05-24T11:36:50.044873+0000","last_fullsized":"2024-05-24T11:36:50.044873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:20:56.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044374+0000","last_change":"2024-05-24T11:36:48.519295+0000","last_active":"2024-05-24T11:36:50.044374+0000","last_peered":"2024-05-24T11:36:50.044374+0000","last_clean":"2024-05-24T11:36:50.044374+0000","last_became_active":"2024-05-24T11:36:48.518955+0000","last_became_peered":"2024-05-24T11:36:48.518955+0000","last_unstale":"2024-05-24T11:36:50.044374+0000","last_undegraded":"2024-05-24T11:36:50.044374+0000","last_fullsized":"2024-05-24T11:36:50.044374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:39:14.437903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518558+0000","last_change":"2024-05-24T11:36:48.525324+0000","last_active":"2024-05-24T11:36:49.518558+0000","last_peered":"2024-05-24T11:36:49.518558+0000","last_clean":"2024-05-24T11:36:49.518558+0000","last_became_active":"2024-05-24T11:36:48.525045+0000","last_became_peered":"2024-05-24T11:36:48.525045+0000","last_unstale":"2024-05-24T11:36:49.518558+0000","last_undegraded":"2024-05-24T11:36:49.518558+0000","last_fullsized":"2024-05-24T11:36:49.518558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:19:52.341136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045101+0000","last_change":"2024-05-24T11:36:48.528095+0000","last_active":"2024-05-24T11:36:50.045101+0000","last_peered":"2024-05-24T11:36:50.045101+0000","last_clean":"2024-05-24T11:36:50.045101+0000","last_became_active":"2024-05-24T11:36:48.527814+0000","last_became_peered":"2024-05-24T11:36:48.527814+0000","last_unstale":"2024-05-24T11:36:50.045101+0000","last_undegraded":"2024-05-24T11:36:50.045101+0000","last_fullsized":"2024-05-24T11:36:50.045101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:12:42.591269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518210+0000","last_change":"2024-05-24T11:36:48.528775+0000","last_active":"2024-05-24T11:36:49.518210+0000","last_peered":"2024-05-24T11:36:49.518210+0000","last_clean":"2024-05-24T11:36:49.518210+0000","last_became_active":"2024-05-24T11:36:48.528467+0000","last_became_peered":"2024-05-24T11:36:48.528467+0000","last_unstale":"2024-05-24T11:36:49.518210+0000","last_undegraded":"2024-05-24T11:36:49.518210+0000","last_fullsized":"2024-05-24T11:36:49.518210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:10:59.444142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5806,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":17,"num_read_kb":16,"num_write":156,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":17165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645322,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":835584,"data_stored":399105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":10,"up_from":65,"seq":279172874254,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40732,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704420,"statfs":{"total":95995035648,"available":95953326080,"internally_reserved":0,"allocated":1462272,"data_stored":991030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":40237505},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":9,"up_from":61,"seq":261993005074,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":860160,"data_stored":398690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.499}]},{"osd":3,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.39000000000000001}]},{"osd":4,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.622}]},{"osd":8,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.66600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":8,"up_from":54,"seq":231928234005,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":399068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61799999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68600000000000005}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66200000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":7,"up_from":47,"seq":201863462936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1425408,"data_stored":989028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.438}]},{"osd":1,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.5},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri May 24 11:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.76400000000000001}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.80500000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":6,"up_from":43,"seq":184683593756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31924,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713228,"statfs":{"total":95995035648,"available":95962345472,"internally_reserved":0,"allocated":823296,"data_stored":397568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.32900000000000001}]},{"osd":1,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.312},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.35999999999999999}]},{"osd":4,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.495}]},{"osd":7,"last update":"Fri May 24 11:36:07 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri May 24 11:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61599999999999999}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.47299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":1,"up_from":12,"seq":51539607598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40128,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93705024,"statfs":{"total":95995035648,"available":95953944576,"internally_reserved":0,"allocated":835584,"data_stored":399132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.33700000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.747,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri May 24 11:36:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.81200000000000006}]},{"osd":3,"last update":"Fri May 24 11:36:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.74199999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri May 24 11:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47799999999999998,"15min":0.47299999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri May 24 11:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.87}]},{"osd":6,"last update":"Fri May 24 11:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73699999999999999}]},{"osd":7,"last update":"Fri May 24 11:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":1.095}]},{"osd":8,"last update":"Fri May 24 11:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.022},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.85999999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":0,"up_from":8,"seq":34359738418,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31944,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713208,"statfs":{"total":95995035648,"available":95962324992,"internally_reserved":0,"allocated":851968,"data_stored":399046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 24 11:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.33300000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri May 24 11:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.45000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri May 24 11:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.40600000000000003,"15min":0.39600000000000002},"min":{"1min":0.23799999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri May 24 11:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82699999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri May 24 11:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.47599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72199999999999998,"5min":0.755,"15min":0.755},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.46999999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri May 24 11:36:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.32800000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri May 24 11:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55000000000000004}]},{"osd":8,"last update":"Fri May 24 11:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.69899999999999995}]},{"osd":9,"last update":"Fri May 24 11:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":2,"up_from":17,"seq":73014444075,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":398848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.746,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.36399999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.85699999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri May 24 11:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77700000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri May 24 11:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72599999999999998,"5min":0.746,"15min":0.746},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri May 24 11:36:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.433,"15min":0.42399999999999999},"min":{"1min":0.313,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.436},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Fri May 24 11:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri May 24 11:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57599999999999996}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.62},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.76900000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182438,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31928,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713224,"statfs":{"total":95995035648,"available":95962341376,"internally_reserved":0,"allocated":827392,"data_stored":397341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35899999999999999},"min":{"1min":0.25800000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46999999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.36399999999999999,"15min":0.36199999999999999},"min":{"1min":0.28599999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.502,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri May 24 11:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri May 24 11:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri May 24 11:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.45000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri May 24 11:36:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84299999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.439}]},{"osd":6,"last update":"Fri May 24 11:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45400000000000001}]},{"osd":7,"last update":"Fri May 24 11:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.75600000000000001}]},{"osd":8,"last update":"Fri May 24 11:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.66900000000000004}]},{"osd":9,"last update":"Fri May 24 11:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.499},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51100000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953508,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":398186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.502,"15min":0.495},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.37,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.308,"5min":0.249,"15min":0.249},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40899999999999997,"15min":0.40300000000000002},"min":{"1min":0.28799999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.437}]},{"osd":2,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83299999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.41499999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89800000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri May 24 11:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.874,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri May 24 11:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61599999999999999,"15min":0.60299999999999998},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.433,"5min":0.378,"15min":0.378},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri May 24 11:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64200000000000002}]},{"osd":8,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.626}]},{"osd":9,"last update":"Fri May 24 11:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32504,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712648,"statfs":{"total":95995035648,"available":95961751552,"internally_reserved":0,"allocated":1421312,"data_stored":987514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76100000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.28499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.51300000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.40500000000000003,"15min":0.40899999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.621}]},{"osd":3,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.441,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri May 24 11:36:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri May 24 11:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.73099999999999998}]},{"osd":8,"last update":"Fri May 24 11:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.83099999999999996}]},{"osd":9,"last update":"Fri May 24 11:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-24T11:36:57.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-24T11:36:57.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.185+0000 7fd342ccd700 1 -- 172.21.15.39:0/2344125214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 msgr2=0x7fd33c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:57.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.185+0000 7fd342ccd700 1 --2- 172.21.15.39:0/2344125214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 0x7fd33c103d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd330009990 tx=0x7fd330009ca0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.185+0000 7fd342ccd700 1 -- 172.21.15.39:0/2344125214 shutdown_connections 2024-05-24T11:36:57.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.185+0000 7fd342ccd700 1 --2- 172.21.15.39:0/2344125214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd33c106b80 0x7fd33c108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.185+0000 7fd342ccd700 1 --2- 172.21.15.39:0/2344125214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 0x7fd33c103d10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.185+0000 7fd342ccd700 1 --2- 172.21.15.39:0/2344125214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c104250 0x7fd33c106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.185+0000 7fd342ccd700 1 -- 172.21.15.39:0/2344125214 >> 172.21.15.39:0/2344125214 conn(0x7fd33c0fb2f0 msgr2=0x7fd33c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:57.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.185+0000 7fd342ccd700 1 -- 172.21.15.39:0/2344125214 shutdown_connections 2024-05-24T11:36:57.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.185+0000 7fd342ccd700 1 -- 172.21.15.39:0/2344125214 wait complete. 2024-05-24T11:36:57.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd342ccd700 1 Processor -- start 2024-05-24T11:36:57.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd342ccd700 1 -- start start 2024-05-24T11:36:57.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd342ccd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 0x7fd33c196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:57.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd342ccd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd33c104250 0x7fd33c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:57.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd342ccd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106b80 0x7fd33c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:57.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd342ccd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c19e8f0 con 0x7fd33c101920 2024-05-24T11:36:57.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd342ccd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c19ea60 con 0x7fd33c104250 2024-05-24T11:36:57.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd342ccd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c19ec20 con 0x7fd33c106b80 2024-05-24T11:36:57.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd340a69700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 0x7fd33c196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:57.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd340a69700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 0x7fd33c196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54444/0 (socket says 172.21.15.39:54444) 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd340a69700 1 -- 172.21.15.39:0/645821267 learned_addr learned my addr 172.21.15.39:0/645821267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd33bfff700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd33c104250 0x7fd33c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.186+0000 7fd340a69700 1 -- 172.21.15.39:0/645821267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106b80 msgr2=0x7fd33c19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd340a69700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106b80 0x7fd33c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd340a69700 1 -- 172.21.15.39:0/645821267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd33c104250 msgr2=0x7fd33c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd340a69700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd33c104250 0x7fd33c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd340a69700 1 -- 172.21.15.39:0/645821267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300095f0 con 0x7fd33c101920 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd340a69700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 0x7fd33c196e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd330000c00 tx=0x7fd3300038d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd339ffb700 1 -- 172.21.15.39:0/645821267 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33001d070 con 0x7fd33c101920 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd339ffb700 1 -- 172.21.15.39:0/645821267 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd330022470 con 0x7fd33c101920 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c19eea0 con 0x7fd33c101920 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd339ffb700 1 -- 172.21.15.39:0/645821267 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33000f6c0 con 0x7fd33c101920 2024-05-24T11:36:57.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.187+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c19f290 con 0x7fd33c101920 2024-05-24T11:36:57.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.188+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33c0fce70 con 0x7fd33c101920 2024-05-24T11:36:57.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.189+0000 7fd339ffb700 1 -- 172.21.15.39:0/645821267 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd3300225e0 con 0x7fd33c101920 2024-05-24T11:36:57.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.190+0000 7fd339ffb700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd32c0a1230 0x7fd32c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:57.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.190+0000 7fd33bfff700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd32c0a1230 0x7fd32c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:57.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.190+0000 7fd339ffb700 1 -- 172.21.15.39:0/645821267 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd3300cde60 con 0x7fd33c101920 2024-05-24T11:36:57.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.190+0000 7fd33bfff700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd32c0a1230 0x7fd32c0a36f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd328006fd0 tx=0x7fd328008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:57.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.194+0000 7fd339ffb700 1 -- 172.21.15.39:0/645821267 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd3300976d0 con 0x7fd33c101920 2024-05-24T11:36:57.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.395+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd33c02ceb0 con 0x7fd32c0a1230 2024-05-24T11:36:57.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.405+0000 7fd339ffb700 1 -- 172.21.15.39:0/645821267 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328439 (secure 0 0 0) 0x7fd33c02ceb0 con 0x7fd32c0a1230 2024-05-24T11:36:57.414 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:36:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd32c0a1230 msgr2=0x7fd32c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd32c0a1230 0x7fd32c0a36f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd328006fd0 tx=0x7fd328008040 comp rx=0 tx=0).stop 2024-05-24T11:36:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 msgr2=0x7fd33c196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 0x7fd33c196e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd330000c00 tx=0x7fd3300038d0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 shutdown_connections 2024-05-24T11:36:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd32c0a1230 0x7fd32c0a36f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd33c106b80 0x7fd33c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd33c101920 0x7fd33c196e00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 --2- 172.21.15.39:0/645821267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd33c104250 0x7fd33c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 >> 172.21.15.39:0/645821267 conn(0x7fd33c0fb2f0 msgr2=0x7fd33c1023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 shutdown_connections 2024-05-24T11:36:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:57.409+0000 7fd342ccd700 1 -- 172.21.15.39:0/645821267 wait complete. 2024-05-24T11:36:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-24T11:36:57.427 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-05-24T11:36:55.876816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596174,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":153,"num_read_kb":135,"num_write":337,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380876,"kb_used_data":11600,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124560948,"statfs":{"total":1151940427776,"available":1151550410752,"internally_reserved":0,"allocated":11878400,"data_stored":6554556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":377992517},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5806,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":17,"num_read_kb":16,"num_write":156,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.805385"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518423+0000","last_change":"2024-05-24T11:36:48.521973+0000","last_active":"2024-05-24T11:36:49.518423+0000","last_peered":"2024-05-24T11:36:49.518423+0000","last_clean":"2024-05-24T11:36:49.518423+0000","last_became_active":"2024-05-24T11:36:48.521587+0000","last_became_peered":"2024-05-24T11:36:48.521587+0000","last_unstale":"2024-05-24T11:36:49.518423+0000","last_undegraded":"2024-05-24T11:36:49.518423+0000","last_fullsized":"2024-05-24T11:36:49.518423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:25:29.234714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811230+0000","last_change":"2024-05-24T11:36:48.536290+0000","last_active":"2024-05-24T11:36:49.811230+0000","last_peered":"2024-05-24T11:36:49.811230+0000","last_clean":"2024-05-24T11:36:49.811230+0000","last_became_active":"2024-05-24T11:36:48.535961+0000","last_became_peered":"2024-05-24T11:36:48.535961+0000","last_unstale":"2024-05-24T11:36:49.811230+0000","last_undegraded":"2024-05-24T11:36:49.811230+0000","last_fullsized":"2024-05-24T11:36:49.811230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:52:37.951507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811216+0000","last_change":"2024-05-24T11:36:48.520039+0000","last_active":"2024-05-24T11:36:49.811216+0000","last_peered":"2024-05-24T11:36:49.811216+0000","last_clean":"2024-05-24T11:36:49.811216+0000","last_became_active":"2024-05-24T11:36:48.519809+0000","last_became_peered":"2024-05-24T11:36:48.519809+0000","last_unstale":"2024-05-24T11:36:49.811216+0000","last_undegraded":"2024-05-24T11:36:49.811216+0000","last_fullsized":"2024-05-24T11:36:49.811216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:44:20.133334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045745+0000","last_change":"2024-05-24T11:36:48.528100+0000","last_active":"2024-05-24T11:36:50.045745+0000","last_peered":"2024-05-24T11:36:50.045745+0000","last_clean":"2024-05-24T11:36:50.045745+0000","last_became_active":"2024-05-24T11:36:48.524918+0000","last_became_peered":"2024-05-24T11:36:48.524918+0000","last_unstale":"2024-05-24T11:36:50.045745+0000","last_undegraded":"2024-05-24T11:36:50.045745+0000","last_fullsized":"2024-05-24T11:36:50.045745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:20:34.628252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044676+0000","last_change":"2024-05-24T11:36:48.522735+0000","last_active":"2024-05-24T11:36:50.044676+0000","last_peered":"2024-05-24T11:36:50.044676+0000","last_clean":"2024-05-24T11:36:50.044676+0000","last_became_active":"2024-05-24T11:36:48.521657+0000","last_became_peered":"2024-05-24T11:36:48.521657+0000","last_unstale":"2024-05-24T11:36:50.044676+0000","last_undegraded":"2024-05-24T11:36:50.044676+0000","last_fullsized":"2024-05-24T11:36:50.044676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:03:32.056035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.835970+0000","last_change":"2024-05-24T11:36:48.534898+0000","last_active":"2024-05-24T11:36:48.835970+0000","last_peered":"2024-05-24T11:36:48.835970+0000","last_clean":"2024-05-24T11:36:48.835970+0000","last_became_active":"2024-05-24T11:36:48.534557+0000","last_became_peered":"2024-05-24T11:36:48.534557+0000","last_unstale":"2024-05-24T11:36:48.835970+0000","last_undegraded":"2024-05-24T11:36:48.835970+0000","last_fullsized":"2024-05-24T11:36:48.835970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:59:23.847279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.533394+0000","last_change":"2024-05-24T11:36:48.532435+0000","last_active":"2024-05-24T11:36:48.533394+0000","last_peered":"2024-05-24T11:36:48.533394+0000","last_clean":"2024-05-24T11:36:48.533394+0000","last_became_active":"2024-05-24T11:36:48.532058+0000","last_became_peered":"2024-05-24T11:36:48.532058+0000","last_unstale":"2024-05-24T11:36:48.533394+0000","last_undegraded":"2024-05-24T11:36:48.533394+0000","last_fullsized":"2024-05-24T11:36:48.533394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:33:26.178124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044447+0000","last_change":"2024-05-24T11:36:48.519054+0000","last_active":"2024-05-24T11:36:50.044447+0000","last_peered":"2024-05-24T11:36:50.044447+0000","last_clean":"2024-05-24T11:36:50.044447+0000","last_became_active":"2024-05-24T11:36:48.518756+0000","last_became_peered":"2024-05-24T11:36:48.518756+0000","last_unstale":"2024-05-24T11:36:50.044447+0000","last_undegraded":"2024-05-24T11:36:50.044447+0000","last_fullsized":"2024-05-24T11:36:50.044447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:56:19.599232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518761+0000","last_change":"2024-05-24T11:36:48.522937+0000","last_active":"2024-05-24T11:36:49.518761+0000","last_peered":"2024-05-24T11:36:49.518761+0000","last_clean":"2024-05-24T11:36:49.518761+0000","last_became_active":"2024-05-24T11:36:48.521531+0000","last_became_peered":"2024-05-24T11:36:48.521531+0000","last_unstale":"2024-05-24T11:36:49.518761+0000","last_undegraded":"2024-05-24T11:36:49.518761+0000","last_fullsized":"2024-05-24T11:36:49.518761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:16:54.392477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.836050+0000","last_change":"2024-05-24T11:36:48.536738+0000","last_active":"2024-05-24T11:36:48.836050+0000","last_peered":"2024-05-24T11:36:48.836050+0000","last_clean":"2024-05-24T11:36:48.836050+0000","last_became_active":"2024-05-24T11:36:48.536010+0000","last_became_peered":"2024-05-24T11:36:48.536010+0000","last_unstale":"2024-05-24T11:36:48.836050+0000","last_undegraded":"2024-05-24T11:36:48.836050+0000","last_fullsized":"2024-05-24T11:36:48.836050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:21:24.078676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.536403+0000","last_change":"2024-05-24T11:36:48.535036+0000","last_active":"2024-05-24T11:36:48.536403+0000","last_peered":"2024-05-24T11:36:48.536403+0000","last_clean":"2024-05-24T11:36:48.536403+0000","last_became_active":"2024-05-24T11:36:48.534232+0000","last_became_peered":"2024-05-24T11:36:48.534232+0000","last_unstale":"2024-05-24T11:36:48.536403+0000","last_undegraded":"2024-05-24T11:36:48.536403+0000","last_fullsized":"2024-05-24T11:36:48.536403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:56:28.111680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045685+0000","last_change":"2024-05-24T11:36:48.537837+0000","last_active":"2024-05-24T11:36:50.045685+0000","last_peered":"2024-05-24T11:36:50.045685+0000","last_clean":"2024-05-24T11:36:50.045685+0000","last_became_active":"2024-05-24T11:36:48.537570+0000","last_became_peered":"2024-05-24T11:36:48.537570+0000","last_unstale":"2024-05-24T11:36:50.045685+0000","last_undegraded":"2024-05-24T11:36:50.045685+0000","last_fullsized":"2024-05-24T11:36:50.045685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:47:14.769120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045004+0000","last_change":"2024-05-24T11:36:48.525333+0000","last_active":"2024-05-24T11:36:50.045004+0000","last_peered":"2024-05-24T11:36:50.045004+0000","last_clean":"2024-05-24T11:36:50.045004+0000","last_became_active":"2024-05-24T11:36:48.525146+0000","last_became_peered":"2024-05-24T11:36:48.525146+0000","last_unstale":"2024-05-24T11:36:50.045004+0000","last_undegraded":"2024-05-24T11:36:50.045004+0000","last_fullsized":"2024-05-24T11:36:50.045004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:22:58.950175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811013+0000","last_change":"2024-05-24T11:36:48.525438+0000","last_active":"2024-05-24T11:36:49.811013+0000","last_peered":"2024-05-24T11:36:49.811013+0000","last_clean":"2024-05-24T11:36:49.811013+0000","last_became_active":"2024-05-24T11:36:48.524866+0000","last_became_peered":"2024-05-24T11:36:48.524866+0000","last_unstale":"2024-05-24T11:36:49.811013+0000","last_undegraded":"2024-05-24T11:36:49.811013+0000","last_fullsized":"2024-05-24T11:36:49.811013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:39:11.826673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.810917+0000","last_change":"2024-05-24T11:36:48.529993+0000","last_active":"2024-05-24T11:36:49.810917+0000","last_peered":"2024-05-24T11:36:49.810917+0000","last_clean":"2024-05-24T11:36:49.810917+0000","last_became_active":"2024-05-24T11:36:48.529644+0000","last_became_peered":"2024-05-24T11:36:48.529644+0000","last_unstale":"2024-05-24T11:36:49.810917+0000","last_undegraded":"2024-05-24T11:36:49.810917+0000","last_fullsized":"2024-05-24T11:36:49.810917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:00:02.580642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518315+0000","last_change":"2024-05-24T11:36:48.521904+0000","last_active":"2024-05-24T11:36:49.518315+0000","last_peered":"2024-05-24T11:36:49.518315+0000","last_clean":"2024-05-24T11:36:49.518315+0000","last_became_active":"2024-05-24T11:36:48.521281+0000","last_became_peered":"2024-05-24T11:36:48.521281+0000","last_unstale":"2024-05-24T11:36:49.518315+0000","last_undegraded":"2024-05-24T11:36:49.518315+0000","last_fullsized":"2024-05-24T11:36:49.518315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:16:32.578404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518556+0000","last_change":"2024-05-24T11:36:48.526737+0000","last_active":"2024-05-24T11:36:49.518556+0000","last_peered":"2024-05-24T11:36:49.518556+0000","last_clean":"2024-05-24T11:36:49.518556+0000","last_became_active":"2024-05-24T11:36:48.526379+0000","last_became_peered":"2024-05-24T11:36:48.526379+0000","last_unstale":"2024-05-24T11:36:49.518556+0000","last_undegraded":"2024-05-24T11:36:49.518556+0000","last_fullsized":"2024-05-24T11:36:49.518556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:32:19.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.522182+0000","last_change":"2024-05-24T11:36:48.529315+0000","last_active":"2024-05-24T11:36:49.522182+0000","last_peered":"2024-05-24T11:36:49.522182+0000","last_clean":"2024-05-24T11:36:49.522182+0000","last_became_active":"2024-05-24T11:36:48.528963+0000","last_became_peered":"2024-05-24T11:36:48.528963+0000","last_unstale":"2024-05-24T11:36:49.522182+0000","last_undegraded":"2024-05-24T11:36:49.522182+0000","last_fullsized":"2024-05-24T11:36:49.522182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:54:23.922811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.544120+0000","last_change":"2024-05-24T11:36:48.520925+0000","last_active":"2024-05-24T11:36:49.544120+0000","last_peered":"2024-05-24T11:36:49.544120+0000","last_clean":"2024-05-24T11:36:49.544120+0000","last_became_active":"2024-05-24T11:36:48.520592+0000","last_became_peered":"2024-05-24T11:36:48.520592+0000","last_unstale":"2024-05-24T11:36:49.544120+0000","last_undegraded":"2024-05-24T11:36:49.544120+0000","last_fullsized":"2024-05-24T11:36:49.544120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:39:09.918746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.590609+0000","last_change":"2024-05-24T11:36:48.090815+0000","last_active":"2024-05-24T11:36:54.590609+0000","last_peered":"2024-05-24T11:36:54.590609+0000","last_clean":"2024-05-24T11:36:54.590609+0000","last_became_active":"2024-05-24T11:36:48.090481+0000","last_became_peered":"2024-05-24T11:36:48.090481+0000","last_unstale":"2024-05-24T11:36:54.590609+0000","last_undegraded":"2024-05-24T11:36:54.590609+0000","last_fullsized":"2024-05-24T11:36:54.590609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:32:40.850290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044588+0000","last_change":"2024-05-24T11:36:48.519300+0000","last_active":"2024-05-24T11:36:50.044588+0000","last_peered":"2024-05-24T11:36:50.044588+0000","last_clean":"2024-05-24T11:36:50.044588+0000","last_became_active":"2024-05-24T11:36:48.519004+0000","last_became_peered":"2024-05-24T11:36:48.519004+0000","last_unstale":"2024-05-24T11:36:50.044588+0000","last_undegraded":"2024-05-24T11:36:50.044588+0000","last_fullsized":"2024-05-24T11:36:50.044588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:15:58.098701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044840+0000","last_change":"2024-05-24T11:36:48.091695+0000","last_active":"2024-05-24T11:36:50.044840+0000","last_peered":"2024-05-24T11:36:50.044840+0000","last_clean":"2024-05-24T11:36:50.044840+0000","last_became_active":"2024-05-24T11:36:48.091379+0000","last_became_peered":"2024-05-24T11:36:48.091379+0000","last_unstale":"2024-05-24T11:36:50.044840+0000","last_undegraded":"2024-05-24T11:36:50.044840+0000","last_fullsized":"2024-05-24T11:36:50.044840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:07:55.039339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811338+0000","last_change":"2024-05-24T11:36:48.519313+0000","last_active":"2024-05-24T11:36:49.811338+0000","last_peered":"2024-05-24T11:36:49.811338+0000","last_clean":"2024-05-24T11:36:49.811338+0000","last_became_active":"2024-05-24T11:36:48.518968+0000","last_became_peered":"2024-05-24T11:36:48.518968+0000","last_unstale":"2024-05-24T11:36:49.811338+0000","last_undegraded":"2024-05-24T11:36:49.811338+0000","last_fullsized":"2024-05-24T11:36:49.811338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:51:23.712853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'11","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.590653+0000","last_change":"2024-05-24T11:36:48.096014+0000","last_active":"2024-05-24T11:36:54.590653+0000","last_peered":"2024-05-24T11:36:54.590653+0000","last_clean":"2024-05-24T11:36:54.590653+0000","last_became_active":"2024-05-24T11:36:48.095261+0000","last_became_peered":"2024-05-24T11:36:48.095261+0000","last_unstale":"2024-05-24T11:36:54.590653+0000","last_undegraded":"2024-05-24T11:36:54.590653+0000","last_fullsized":"2024-05-24T11:36:54.590653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:19:32.675770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518671+0000","last_change":"2024-05-24T11:36:48.524464+0000","last_active":"2024-05-24T11:36:49.518671+0000","last_peered":"2024-05-24T11:36:49.518671+0000","last_clean":"2024-05-24T11:36:49.518671+0000","last_became_active":"2024-05-24T11:36:48.524128+0000","last_became_peered":"2024-05-24T11:36:48.524128+0000","last_unstale":"2024-05-24T11:36:49.518671+0000","last_undegraded":"2024-05-24T11:36:49.518671+0000","last_fullsized":"2024-05-24T11:36:49.518671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:57:21.920169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.590552+0000","last_change":"2024-05-24T11:36:48.095739+0000","last_active":"2024-05-24T11:36:54.590552+0000","last_peered":"2024-05-24T11:36:54.590552+0000","last_clean":"2024-05-24T11:36:54.590552+0000","last_became_active":"2024-05-24T11:36:48.095313+0000","last_became_peered":"2024-05-24T11:36:48.095313+0000","last_unstale":"2024-05-24T11:36:54.590552+0000","last_undegraded":"2024-05-24T11:36:54.590552+0000","last_fullsized":"2024-05-24T11:36:54.590552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:59:55.283423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045238+0000","last_change":"2024-05-24T11:36:48.528788+0000","last_active":"2024-05-24T11:36:50.045238+0000","last_peered":"2024-05-24T11:36:50.045238+0000","last_clean":"2024-05-24T11:36:50.045238+0000","last_became_active":"2024-05-24T11:36:48.528358+0000","last_became_peered":"2024-05-24T11:36:48.528358+0000","last_unstale":"2024-05-24T11:36:50.045238+0000","last_undegraded":"2024-05-24T11:36:50.045238+0000","last_fullsized":"2024-05-24T11:36:50.045238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:19:37.870208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"77'5","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.601922+0000","last_change":"2024-05-24T11:36:48.093252+0000","last_active":"2024-05-24T11:36:51.601922+0000","last_peered":"2024-05-24T11:36:51.601922+0000","last_clean":"2024-05-24T11:36:51.601922+0000","last_became_active":"2024-05-24T11:36:48.092920+0000","last_became_peered":"2024-05-24T11:36:48.092920+0000","last_unstale":"2024-05-24T11:36:51.601922+0000","last_undegraded":"2024-05-24T11:36:51.601922+0000","last_fullsized":"2024-05-24T11:36:51.601922+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:51:39.032589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044926+0000","last_change":"2024-05-24T11:36:48.518108+0000","last_active":"2024-05-24T11:36:50.044926+0000","last_peered":"2024-05-24T11:36:50.044926+0000","last_clean":"2024-05-24T11:36:50.044926+0000","last_became_active":"2024-05-24T11:36:48.517786+0000","last_became_peered":"2024-05-24T11:36:48.517786+0000","last_unstale":"2024-05-24T11:36:50.044926+0000","last_undegraded":"2024-05-24T11:36:50.044926+0000","last_fullsized":"2024-05-24T11:36:50.044926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:41:44.779354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:53.618566+0000","last_change":"2024-05-24T11:36:48.082348+0000","last_active":"2024-05-24T11:36:53.618566+0000","last_peered":"2024-05-24T11:36:53.618566+0000","last_clean":"2024-05-24T11:36:53.618566+0000","last_became_active":"2024-05-24T11:36:48.081964+0000","last_became_peered":"2024-05-24T11:36:48.081964+0000","last_unstale":"2024-05-24T11:36:53.618566+0000","last_undegraded":"2024-05-24T11:36:53.618566+0000","last_fullsized":"2024-05-24T11:36:53.618566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:45:54.028437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045904+0000","last_change":"2024-05-24T11:36:48.528990+0000","last_active":"2024-05-24T11:36:50.045904+0000","last_peered":"2024-05-24T11:36:50.045904+0000","last_clean":"2024-05-24T11:36:50.045904+0000","last_became_active":"2024-05-24T11:36:48.528444+0000","last_became_peered":"2024-05-24T11:36:48.528444+0000","last_unstale":"2024-05-24T11:36:50.045904+0000","last_undegraded":"2024-05-24T11:36:50.045904+0000","last_fullsized":"2024-05-24T11:36:50.045904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:35:56.108817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045030+0000","last_change":"2024-05-24T11:36:48.079471+0000","last_active":"2024-05-24T11:36:50.045030+0000","last_peered":"2024-05-24T11:36:50.045030+0000","last_clean":"2024-05-24T11:36:50.045030+0000","last_became_active":"2024-05-24T11:36:48.079118+0000","last_became_peered":"2024-05-24T11:36:48.079118+0000","last_unstale":"2024-05-24T11:36:50.045030+0000","last_undegraded":"2024-05-24T11:36:50.045030+0000","last_fullsized":"2024-05-24T11:36:50.045030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:01:35.897437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044314+0000","last_change":"2024-05-24T11:36:48.517960+0000","last_active":"2024-05-24T11:36:50.044314+0000","last_peered":"2024-05-24T11:36:50.044314+0000","last_clean":"2024-05-24T11:36:50.044314+0000","last_became_active":"2024-05-24T11:36:48.517675+0000","last_became_peered":"2024-05-24T11:36:48.517675+0000","last_unstale":"2024-05-24T11:36:50.044314+0000","last_undegraded":"2024-05-24T11:36:50.044314+0000","last_fullsized":"2024-05-24T11:36:50.044314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:08:21.209080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599284+0000","last_change":"2024-05-24T11:36:48.091922+0000","last_active":"2024-05-24T11:36:51.599284+0000","last_peered":"2024-05-24T11:36:51.599284+0000","last_clean":"2024-05-24T11:36:51.599284+0000","last_became_active":"2024-05-24T11:36:48.091576+0000","last_became_peered":"2024-05-24T11:36:48.091576+0000","last_unstale":"2024-05-24T11:36:51.599284+0000","last_undegraded":"2024-05-24T11:36:51.599284+0000","last_fullsized":"2024-05-24T11:36:51.599284+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:01:39.386649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.521890+0000","last_change":"2024-05-24T11:36:48.523769+0000","last_active":"2024-05-24T11:36:49.521890+0000","last_peered":"2024-05-24T11:36:49.521890+0000","last_clean":"2024-05-24T11:36:49.521890+0000","last_became_active":"2024-05-24T11:36:48.523394+0000","last_became_peered":"2024-05-24T11:36:48.523394+0000","last_unstale":"2024-05-24T11:36:49.521890+0000","last_undegraded":"2024-05-24T11:36:49.521890+0000","last_fullsized":"2024-05-24T11:36:49.521890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:03:49.209246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.600064+0000","last_change":"2024-05-24T11:36:48.089651+0000","last_active":"2024-05-24T11:36:51.600064+0000","last_peered":"2024-05-24T11:36:51.600064+0000","last_clean":"2024-05-24T11:36:51.600064+0000","last_became_active":"2024-05-24T11:36:48.089195+0000","last_became_peered":"2024-05-24T11:36:48.089195+0000","last_unstale":"2024-05-24T11:36:51.600064+0000","last_undegraded":"2024-05-24T11:36:51.600064+0000","last_fullsized":"2024-05-24T11:36:51.600064+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:30:47.730594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518732+0000","last_change":"2024-05-24T11:36:48.531529+0000","last_active":"2024-05-24T11:36:49.518732+0000","last_peered":"2024-05-24T11:36:49.518732+0000","last_clean":"2024-05-24T11:36:49.518732+0000","last_became_active":"2024-05-24T11:36:48.530969+0000","last_became_peered":"2024-05-24T11:36:48.530969+0000","last_unstale":"2024-05-24T11:36:49.518732+0000","last_undegraded":"2024-05-24T11:36:49.518732+0000","last_fullsized":"2024-05-24T11:36:49.518732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:05:24.929278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599141+0000","last_change":"2024-05-24T11:36:48.094249+0000","last_active":"2024-05-24T11:36:51.599141+0000","last_peered":"2024-05-24T11:36:51.599141+0000","last_clean":"2024-05-24T11:36:51.599141+0000","last_became_active":"2024-05-24T11:36:48.093835+0000","last_became_peered":"2024-05-24T11:36:48.093835+0000","last_unstale":"2024-05-24T11:36:51.599141+0000","last_undegraded":"2024-05-24T11:36:51.599141+0000","last_fullsized":"2024-05-24T11:36:51.599141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:41:47.660513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811338+0000","last_change":"2024-05-24T11:36:48.531745+0000","last_active":"2024-05-24T11:36:49.811338+0000","last_peered":"2024-05-24T11:36:49.811338+0000","last_clean":"2024-05-24T11:36:49.811338+0000","last_became_active":"2024-05-24T11:36:48.531459+0000","last_became_peered":"2024-05-24T11:36:48.531459+0000","last_unstale":"2024-05-24T11:36:49.811338+0000","last_undegraded":"2024-05-24T11:36:49.811338+0000","last_fullsized":"2024-05-24T11:36:49.811338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:06:41.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.598752+0000","last_change":"2024-05-24T11:36:48.089567+0000","last_active":"2024-05-24T11:36:51.598752+0000","last_peered":"2024-05-24T11:36:51.598752+0000","last_clean":"2024-05-24T11:36:51.598752+0000","last_became_active":"2024-05-24T11:36:48.089248+0000","last_became_peered":"2024-05-24T11:36:48.089248+0000","last_unstale":"2024-05-24T11:36:51.598752+0000","last_undegraded":"2024-05-24T11:36:51.598752+0000","last_fullsized":"2024-05-24T11:36:51.598752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:49:06.307799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518786+0000","last_change":"2024-05-24T11:36:48.529531+0000","last_active":"2024-05-24T11:36:49.518786+0000","last_peered":"2024-05-24T11:36:49.518786+0000","last_clean":"2024-05-24T11:36:49.518786+0000","last_became_active":"2024-05-24T11:36:48.529222+0000","last_became_peered":"2024-05-24T11:36:48.529222+0000","last_unstale":"2024-05-24T11:36:49.518786+0000","last_undegraded":"2024-05-24T11:36:49.518786+0000","last_fullsized":"2024-05-24T11:36:49.518786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:18:49.571222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.600866+0000","last_change":"2024-05-24T11:36:48.086123+0000","last_active":"2024-05-24T11:36:51.600866+0000","last_peered":"2024-05-24T11:36:51.600866+0000","last_clean":"2024-05-24T11:36:51.600866+0000","last_became_active":"2024-05-24T11:36:48.085681+0000","last_became_peered":"2024-05-24T11:36:48.085681+0000","last_unstale":"2024-05-24T11:36:51.600866+0000","last_undegraded":"2024-05-24T11:36:51.600866+0000","last_fullsized":"2024-05-24T11:36:51.600866+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:17:12.646332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045057+0000","last_change":"2024-05-24T11:36:48.524883+0000","last_active":"2024-05-24T11:36:50.045057+0000","last_peered":"2024-05-24T11:36:50.045057+0000","last_clean":"2024-05-24T11:36:50.045057+0000","last_became_active":"2024-05-24T11:36:48.524578+0000","last_became_peered":"2024-05-24T11:36:48.524578+0000","last_unstale":"2024-05-24T11:36:50.045057+0000","last_undegraded":"2024-05-24T11:36:50.045057+0000","last_fullsized":"2024-05-24T11:36:50.045057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:54:15.419099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'6","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.610572+0000","last_change":"2024-05-24T11:36:48.090437+0000","last_active":"2024-05-24T11:36:52.610572+0000","last_peered":"2024-05-24T11:36:52.610572+0000","last_clean":"2024-05-24T11:36:52.610572+0000","last_became_active":"2024-05-24T11:36:48.088745+0000","last_became_peered":"2024-05-24T11:36:48.088745+0000","last_unstale":"2024-05-24T11:36:52.610572+0000","last_undegraded":"2024-05-24T11:36:52.610572+0000","last_fullsized":"2024-05-24T11:36:52.610572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:12:48.829945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044821+0000","last_change":"2024-05-24T11:36:48.525723+0000","last_active":"2024-05-24T11:36:50.044821+0000","last_peered":"2024-05-24T11:36:50.044821+0000","last_clean":"2024-05-24T11:36:50.044821+0000","last_became_active":"2024-05-24T11:36:48.525267+0000","last_became_peered":"2024-05-24T11:36:48.525267+0000","last_unstale":"2024-05-24T11:36:50.044821+0000","last_undegraded":"2024-05-24T11:36:50.044821+0000","last_fullsized":"2024-05-24T11:36:50.044821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:27:59.404214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.600200+0000","last_change":"2024-05-24T11:36:48.094426+0000","last_active":"2024-05-24T11:36:51.600200+0000","last_peered":"2024-05-24T11:36:51.600200+0000","last_clean":"2024-05-24T11:36:51.600200+0000","last_became_active":"2024-05-24T11:36:48.094169+0000","last_became_peered":"2024-05-24T11:36:48.094169+0000","last_unstale":"2024-05-24T11:36:51.600200+0000","last_undegraded":"2024-05-24T11:36:51.600200+0000","last_fullsized":"2024-05-24T11:36:51.600200+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:31:02.682818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.531031+0000","last_change":"2024-05-24T11:36:48.529851+0000","last_active":"2024-05-24T11:36:48.531031+0000","last_peered":"2024-05-24T11:36:48.531031+0000","last_clean":"2024-05-24T11:36:48.531031+0000","last_became_active":"2024-05-24T11:36:48.529540+0000","last_became_peered":"2024-05-24T11:36:48.529540+0000","last_unstale":"2024-05-24T11:36:48.531031+0000","last_undegraded":"2024-05-24T11:36:48.531031+0000","last_fullsized":"2024-05-24T11:36:48.531031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:22:12.101608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'5","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.583517+0000","last_change":"2024-05-24T11:36:48.090611+0000","last_active":"2024-05-24T11:36:54.583517+0000","last_peered":"2024-05-24T11:36:54.583517+0000","last_clean":"2024-05-24T11:36:54.583517+0000","last_became_active":"2024-05-24T11:36:48.090283+0000","last_became_peered":"2024-05-24T11:36:48.090283+0000","last_unstale":"2024-05-24T11:36:54.583517+0000","last_undegraded":"2024-05-24T11:36:54.583517+0000","last_fullsized":"2024-05-24T11:36:54.583517+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:24:57.272371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.537097+0000","last_change":"2024-05-24T11:36:48.536763+0000","last_active":"2024-05-24T11:36:48.537097+0000","last_peered":"2024-05-24T11:36:48.537097+0000","last_clean":"2024-05-24T11:36:48.537097+0000","last_became_active":"2024-05-24T11:36:48.536398+0000","last_became_peered":"2024-05-24T11:36:48.536398+0000","last_unstale":"2024-05-24T11:36:48.537097+0000","last_undegraded":"2024-05-24T11:36:48.537097+0000","last_fullsized":"2024-05-24T11:36:48.537097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:13:58.446126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.581083+0000","last_change":"2024-05-24T11:36:48.080590+0000","last_active":"2024-05-24T11:36:54.581083+0000","last_peered":"2024-05-24T11:36:54.581083+0000","last_clean":"2024-05-24T11:36:54.581083+0000","last_became_active":"2024-05-24T11:36:48.080192+0000","last_became_peered":"2024-05-24T11:36:48.080192+0000","last_unstale":"2024-05-24T11:36:54.581083+0000","last_undegraded":"2024-05-24T11:36:54.581083+0000","last_fullsized":"2024-05-24T11:36:54.581083+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:13:35.452878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044986+0000","last_change":"2024-05-24T11:36:48.530444+0000","last_active":"2024-05-24T11:36:50.044986+0000","last_peered":"2024-05-24T11:36:50.044986+0000","last_clean":"2024-05-24T11:36:50.044986+0000","last_became_active":"2024-05-24T11:36:48.530063+0000","last_became_peered":"2024-05-24T11:36:48.530063+0000","last_unstale":"2024-05-24T11:36:50.044986+0000","last_undegraded":"2024-05-24T11:36:50.044986+0000","last_fullsized":"2024-05-24T11:36:50.044986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:23:12.380170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.581432+0000","last_change":"2024-05-24T11:36:48.086566+0000","last_active":"2024-05-24T11:36:54.581432+0000","last_peered":"2024-05-24T11:36:54.581432+0000","last_clean":"2024-05-24T11:36:54.581432+0000","last_became_active":"2024-05-24T11:36:48.086213+0000","last_became_peered":"2024-05-24T11:36:48.086213+0000","last_unstale":"2024-05-24T11:36:54.581432+0000","last_undegraded":"2024-05-24T11:36:54.581432+0000","last_fullsized":"2024-05-24T11:36:54.581432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:09:53.744478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518496+0000","last_change":"2024-05-24T11:36:48.528746+0000","last_active":"2024-05-24T11:36:49.518496+0000","last_peered":"2024-05-24T11:36:49.518496+0000","last_clean":"2024-05-24T11:36:49.518496+0000","last_became_active":"2024-05-24T11:36:48.528399+0000","last_became_peered":"2024-05-24T11:36:48.528399+0000","last_unstale":"2024-05-24T11:36:49.518496+0000","last_undegraded":"2024-05-24T11:36:49.518496+0000","last_fullsized":"2024-05-24T11:36:49.518496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:51:28.838280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.790552+0000","last_change":"2024-05-24T11:36:48.084268+0000","last_active":"2024-05-24T11:36:50.790552+0000","last_peered":"2024-05-24T11:36:50.790552+0000","last_clean":"2024-05-24T11:36:50.790552+0000","last_became_active":"2024-05-24T11:36:48.083968+0000","last_became_peered":"2024-05-24T11:36:48.083968+0000","last_unstale":"2024-05-24T11:36:50.790552+0000","last_undegraded":"2024-05-24T11:36:50.790552+0000","last_fullsized":"2024-05-24T11:36:50.790552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:02:00.989621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.836040+0000","last_change":"2024-05-24T11:36:48.523784+0000","last_active":"2024-05-24T11:36:48.836040+0000","last_peered":"2024-05-24T11:36:48.836040+0000","last_clean":"2024-05-24T11:36:48.836040+0000","last_became_active":"2024-05-24T11:36:48.523430+0000","last_became_peered":"2024-05-24T11:36:48.523430+0000","last_unstale":"2024-05-24T11:36:48.836040+0000","last_undegraded":"2024-05-24T11:36:48.836040+0000","last_fullsized":"2024-05-24T11:36:48.836040+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:02:33.070861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.048190+0000","last_change":"2024-05-24T11:36:48.092166+0000","last_active":"2024-05-24T11:36:50.048190+0000","last_peered":"2024-05-24T11:36:50.048190+0000","last_clean":"2024-05-24T11:36:50.048190+0000","last_became_active":"2024-05-24T11:36:48.091682+0000","last_became_peered":"2024-05-24T11:36:48.091682+0000","last_unstale":"2024-05-24T11:36:50.048190+0000","last_undegraded":"2024-05-24T11:36:50.048190+0000","last_fullsized":"2024-05-24T11:36:50.048190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:14:53.083387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.835997+0000","last_change":"2024-05-24T11:36:48.534618+0000","last_active":"2024-05-24T11:36:48.835997+0000","last_peered":"2024-05-24T11:36:48.835997+0000","last_clean":"2024-05-24T11:36:48.835997+0000","last_became_active":"2024-05-24T11:36:48.534251+0000","last_became_peered":"2024-05-24T11:36:48.534251+0000","last_unstale":"2024-05-24T11:36:48.835997+0000","last_undegraded":"2024-05-24T11:36:48.835997+0000","last_fullsized":"2024-05-24T11:36:48.835997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:14:46.889921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045023+0000","last_change":"2024-05-24T11:36:48.528214+0000","last_active":"2024-05-24T11:36:50.045023+0000","last_peered":"2024-05-24T11:36:50.045023+0000","last_clean":"2024-05-24T11:36:50.045023+0000","last_became_active":"2024-05-24T11:36:48.526711+0000","last_became_peered":"2024-05-24T11:36:48.526711+0000","last_unstale":"2024-05-24T11:36:50.045023+0000","last_undegraded":"2024-05-24T11:36:50.045023+0000","last_fullsized":"2024-05-24T11:36:50.045023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:06:27.935139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045003+0000","last_change":"2024-05-24T11:36:48.089263+0000","last_active":"2024-05-24T11:36:50.045003+0000","last_peered":"2024-05-24T11:36:50.045003+0000","last_clean":"2024-05-24T11:36:50.045003+0000","last_became_active":"2024-05-24T11:36:48.087905+0000","last_became_peered":"2024-05-24T11:36:48.087905+0000","last_unstale":"2024-05-24T11:36:50.045003+0000","last_undegraded":"2024-05-24T11:36:50.045003+0000","last_fullsized":"2024-05-24T11:36:50.045003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:06:37.941039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044877+0000","last_change":"2024-05-24T11:36:48.517569+0000","last_active":"2024-05-24T11:36:50.044877+0000","last_peered":"2024-05-24T11:36:50.044877+0000","last_clean":"2024-05-24T11:36:50.044877+0000","last_became_active":"2024-05-24T11:36:48.517202+0000","last_became_peered":"2024-05-24T11:36:48.517202+0000","last_unstale":"2024-05-24T11:36:50.044877+0000","last_undegraded":"2024-05-24T11:36:50.044877+0000","last_fullsized":"2024-05-24T11:36:50.044877+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:38:50.778284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.582717+0000","last_change":"2024-05-24T11:36:48.077543+0000","last_active":"2024-05-24T11:36:54.582717+0000","last_peered":"2024-05-24T11:36:54.582717+0000","last_clean":"2024-05-24T11:36:54.582717+0000","last_became_active":"2024-05-24T11:36:48.077058+0000","last_became_peered":"2024-05-24T11:36:48.077058+0000","last_unstale":"2024-05-24T11:36:54.582717+0000","last_undegraded":"2024-05-24T11:36:54.582717+0000","last_fullsized":"2024-05-24T11:36:54.582717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:19:49.486922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":327,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.515699+0000","last_change":"2024-05-24T11:36:00.577887+0000","last_active":"2024-05-24T11:36:48.515699+0000","last_peered":"2024-05-24T11:36:48.515699+0000","last_clean":"2024-05-24T11:36:48.515699+0000","last_became_active":"2024-05-24T11:36:00.577291+0000","last_became_peered":"2024-05-24T11:36:00.577291+0000","last_unstale":"2024-05-24T11:36:48.515699+0000","last_undegraded":"2024-05-24T11:36:48.515699+0000","last_fullsized":"2024-05-24T11:36:48.515699+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:33:34.896223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:33:34.896223+0000","last_clean_scrub_stamp":"2024-05-24T11:33:34.896223+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:11:36.393700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044963+0000","last_change":"2024-05-24T11:36:48.527294+0000","last_active":"2024-05-24T11:36:50.044963+0000","last_peered":"2024-05-24T11:36:50.044963+0000","last_clean":"2024-05-24T11:36:50.044963+0000","last_became_active":"2024-05-24T11:36:48.526983+0000","last_became_peered":"2024-05-24T11:36:48.526983+0000","last_unstale":"2024-05-24T11:36:50.044963+0000","last_undegraded":"2024-05-24T11:36:50.044963+0000","last_fullsized":"2024-05-24T11:36:50.044963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:23:26.492502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.610703+0000","last_change":"2024-05-24T11:36:48.097290+0000","last_active":"2024-05-24T11:36:52.610703+0000","last_peered":"2024-05-24T11:36:52.610703+0000","last_clean":"2024-05-24T11:36:52.610703+0000","last_became_active":"2024-05-24T11:36:48.096942+0000","last_became_peered":"2024-05-24T11:36:48.096942+0000","last_unstale":"2024-05-24T11:36:52.610703+0000","last_undegraded":"2024-05-24T11:36:52.610703+0000","last_fullsized":"2024-05-24T11:36:52.610703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:24:20.221023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.579815+0000","last_change":"2024-05-24T11:36:48.079294+0000","last_active":"2024-05-24T11:36:54.579815+0000","last_peered":"2024-05-24T11:36:54.579815+0000","last_clean":"2024-05-24T11:36:54.579815+0000","last_became_active":"2024-05-24T11:36:48.078919+0000","last_became_peered":"2024-05-24T11:36:48.078919+0000","last_unstale":"2024-05-24T11:36:54.579815+0000","last_undegraded":"2024-05-24T11:36:54.579815+0000","last_fullsized":"2024-05-24T11:36:54.579815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:22:32.262674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045222+0000","last_change":"2024-05-24T11:36:48.524666+0000","last_active":"2024-05-24T11:36:50.045222+0000","last_peered":"2024-05-24T11:36:50.045222+0000","last_clean":"2024-05-24T11:36:50.045222+0000","last_became_active":"2024-05-24T11:36:48.524347+0000","last_became_peered":"2024-05-24T11:36:48.524347+0000","last_unstale":"2024-05-24T11:36:50.045222+0000","last_undegraded":"2024-05-24T11:36:50.045222+0000","last_fullsized":"2024-05-24T11:36:50.045222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:14:13.716982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:53.617908+0000","last_change":"2024-05-24T11:36:48.085665+0000","last_active":"2024-05-24T11:36:53.617908+0000","last_peered":"2024-05-24T11:36:53.617908+0000","last_clean":"2024-05-24T11:36:53.617908+0000","last_became_active":"2024-05-24T11:36:48.085344+0000","last_became_peered":"2024-05-24T11:36:48.085344+0000","last_unstale":"2024-05-24T11:36:53.617908+0000","last_undegraded":"2024-05-24T11:36:53.617908+0000","last_fullsized":"2024-05-24T11:36:53.617908+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:11:29.930656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044736+0000","last_change":"2024-05-24T11:36:48.523060+0000","last_active":"2024-05-24T11:36:50.044736+0000","last_peered":"2024-05-24T11:36:50.044736+0000","last_clean":"2024-05-24T11:36:50.044736+0000","last_became_active":"2024-05-24T11:36:48.522712+0000","last_became_peered":"2024-05-24T11:36:48.522712+0000","last_unstale":"2024-05-24T11:36:50.044736+0000","last_undegraded":"2024-05-24T11:36:50.044736+0000","last_fullsized":"2024-05-24T11:36:50.044736+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:07:54.311695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.589339+0000","last_change":"2024-05-24T11:36:48.092809+0000","last_active":"2024-05-24T11:36:54.589339+0000","last_peered":"2024-05-24T11:36:54.589339+0000","last_clean":"2024-05-24T11:36:54.589339+0000","last_became_active":"2024-05-24T11:36:48.092447+0000","last_became_peered":"2024-05-24T11:36:48.092447+0000","last_unstale":"2024-05-24T11:36:54.589339+0000","last_undegraded":"2024-05-24T11:36:54.589339+0000","last_fullsized":"2024-05-24T11:36:54.589339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:27:12.916761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811262+0000","last_change":"2024-05-24T11:36:48.528514+0000","last_active":"2024-05-24T11:36:49.811262+0000","last_peered":"2024-05-24T11:36:49.811262+0000","last_clean":"2024-05-24T11:36:49.811262+0000","last_became_active":"2024-05-24T11:36:48.528244+0000","last_became_peered":"2024-05-24T11:36:48.528244+0000","last_unstale":"2024-05-24T11:36:49.811262+0000","last_undegraded":"2024-05-24T11:36:49.811262+0000","last_fullsized":"2024-05-24T11:36:49.811262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:53:05.993805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518367+0000","last_change":"2024-05-24T11:36:48.094084+0000","last_active":"2024-05-24T11:36:49.518367+0000","last_peered":"2024-05-24T11:36:49.518367+0000","last_clean":"2024-05-24T11:36:49.518367+0000","last_became_active":"2024-05-24T11:36:48.093254+0000","last_became_peered":"2024-05-24T11:36:48.093254+0000","last_unstale":"2024-05-24T11:36:49.518367+0000","last_undegraded":"2024-05-24T11:36:49.518367+0000","last_fullsized":"2024-05-24T11:36:49.518367+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:56:18.896200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044843+0000","last_change":"2024-05-24T11:36:48.522292+0000","last_active":"2024-05-24T11:36:50.044843+0000","last_peered":"2024-05-24T11:36:50.044843+0000","last_clean":"2024-05-24T11:36:50.044843+0000","last_became_active":"2024-05-24T11:36:48.521356+0000","last_became_peered":"2024-05-24T11:36:48.521356+0000","last_unstale":"2024-05-24T11:36:50.044843+0000","last_undegraded":"2024-05-24T11:36:50.044843+0000","last_fullsized":"2024-05-24T11:36:50.044843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:51:44.805455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044445+0000","last_change":"2024-05-24T11:36:48.086209+0000","last_active":"2024-05-24T11:36:50.044445+0000","last_peered":"2024-05-24T11:36:50.044445+0000","last_clean":"2024-05-24T11:36:50.044445+0000","last_became_active":"2024-05-24T11:36:48.084978+0000","last_became_peered":"2024-05-24T11:36:48.084978+0000","last_unstale":"2024-05-24T11:36:50.044445+0000","last_undegraded":"2024-05-24T11:36:50.044445+0000","last_fullsized":"2024-05-24T11:36:50.044445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:26:39.181832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518178+0000","last_change":"2024-05-24T11:36:48.520890+0000","last_active":"2024-05-24T11:36:49.518178+0000","last_peered":"2024-05-24T11:36:49.518178+0000","last_clean":"2024-05-24T11:36:49.518178+0000","last_became_active":"2024-05-24T11:36:48.519860+0000","last_became_peered":"2024-05-24T11:36:48.519860+0000","last_unstale":"2024-05-24T11:36:49.518178+0000","last_undegraded":"2024-05-24T11:36:49.518178+0000","last_fullsized":"2024-05-24T11:36:49.518178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:54:35.691695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.512436+0000","last_change":"2024-05-24T11:36:48.089884+0000","last_active":"2024-05-24T11:36:48.512436+0000","last_peered":"2024-05-24T11:36:48.512436+0000","last_clean":"2024-05-24T11:36:48.512436+0000","last_became_active":"2024-05-24T11:36:48.089386+0000","last_became_peered":"2024-05-24T11:36:48.089386+0000","last_unstale":"2024-05-24T11:36:48.512436+0000","last_undegraded":"2024-05-24T11:36:48.512436+0000","last_fullsized":"2024-05-24T11:36:48.512436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:56:14.463210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044730+0000","last_change":"2024-05-24T11:36:48.515178+0000","last_active":"2024-05-24T11:36:50.044730+0000","last_peered":"2024-05-24T11:36:50.044730+0000","last_clean":"2024-05-24T11:36:50.044730+0000","last_became_active":"2024-05-24T11:36:48.514277+0000","last_became_peered":"2024-05-24T11:36:48.514277+0000","last_unstale":"2024-05-24T11:36:50.044730+0000","last_undegraded":"2024-05-24T11:36:50.044730+0000","last_fullsized":"2024-05-24T11:36:50.044730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:46:38.526893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599557+0000","last_change":"2024-05-24T11:36:48.095493+0000","last_active":"2024-05-24T11:36:51.599557+0000","last_peered":"2024-05-24T11:36:51.599557+0000","last_clean":"2024-05-24T11:36:51.599557+0000","last_became_active":"2024-05-24T11:36:48.094863+0000","last_became_peered":"2024-05-24T11:36:48.094863+0000","last_unstale":"2024-05-24T11:36:51.599557+0000","last_undegraded":"2024-05-24T11:36:51.599557+0000","last_fullsized":"2024-05-24T11:36:51.599557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:45:06.827169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.530486+0000","last_change":"2024-05-24T11:36:48.528024+0000","last_active":"2024-05-24T11:36:48.530486+0000","last_peered":"2024-05-24T11:36:48.530486+0000","last_clean":"2024-05-24T11:36:48.530486+0000","last_became_active":"2024-05-24T11:36:48.527689+0000","last_became_peered":"2024-05-24T11:36:48.527689+0000","last_unstale":"2024-05-24T11:36:48.530486+0000","last_undegraded":"2024-05-24T11:36:48.530486+0000","last_fullsized":"2024-05-24T11:36:48.530486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:07:50.186661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.612656+0000","last_change":"2024-05-24T11:36:48.090660+0000","last_active":"2024-05-24T11:36:52.612656+0000","last_peered":"2024-05-24T11:36:52.612656+0000","last_clean":"2024-05-24T11:36:52.612656+0000","last_became_active":"2024-05-24T11:36:48.090298+0000","last_became_peered":"2024-05-24T11:36:48.090298+0000","last_unstale":"2024-05-24T11:36:52.612656+0000","last_undegraded":"2024-05-24T11:36:52.612656+0000","last_fullsized":"2024-05-24T11:36:52.612656+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:07:30.366506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.528160+0000","last_change":"2024-05-24T11:36:48.526287+0000","last_active":"2024-05-24T11:36:48.528160+0000","last_peered":"2024-05-24T11:36:48.528160+0000","last_clean":"2024-05-24T11:36:48.528160+0000","last_became_active":"2024-05-24T11:36:48.525951+0000","last_became_peered":"2024-05-24T11:36:48.525951+0000","last_unstale":"2024-05-24T11:36:48.528160+0000","last_undegraded":"2024-05-24T11:36:48.528160+0000","last_fullsized":"2024-05-24T11:36:48.528160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:38:11.562370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045141+0000","last_change":"2024-05-24T11:36:48.092808+0000","last_active":"2024-05-24T11:36:50.045141+0000","last_peered":"2024-05-24T11:36:50.045141+0000","last_clean":"2024-05-24T11:36:50.045141+0000","last_became_active":"2024-05-24T11:36:48.092466+0000","last_became_peered":"2024-05-24T11:36:48.092466+0000","last_unstale":"2024-05-24T11:36:50.045141+0000","last_undegraded":"2024-05-24T11:36:50.045141+0000","last_fullsized":"2024-05-24T11:36:50.045141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:27:08.423449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.535337+0000","last_change":"2024-05-24T11:36:48.533175+0000","last_active":"2024-05-24T11:36:48.535337+0000","last_peered":"2024-05-24T11:36:48.535337+0000","last_clean":"2024-05-24T11:36:48.535337+0000","last_became_active":"2024-05-24T11:36:48.532779+0000","last_became_peered":"2024-05-24T11:36:48.532779+0000","last_unstale":"2024-05-24T11:36:48.535337+0000","last_undegraded":"2024-05-24T11:36:48.535337+0000","last_fullsized":"2024-05-24T11:36:48.535337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:39:40.041747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045161+0000","last_change":"2024-05-24T11:36:48.080825+0000","last_active":"2024-05-24T11:36:50.045161+0000","last_peered":"2024-05-24T11:36:50.045161+0000","last_clean":"2024-05-24T11:36:50.045161+0000","last_became_active":"2024-05-24T11:36:48.080462+0000","last_became_peered":"2024-05-24T11:36:48.080462+0000","last_unstale":"2024-05-24T11:36:50.045161+0000","last_undegraded":"2024-05-24T11:36:50.045161+0000","last_fullsized":"2024-05-24T11:36:50.045161+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:05:25.665222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.536673+0000","last_change":"2024-05-24T11:36:48.535094+0000","last_active":"2024-05-24T11:36:48.536673+0000","last_peered":"2024-05-24T11:36:48.536673+0000","last_clean":"2024-05-24T11:36:48.536673+0000","last_became_active":"2024-05-24T11:36:48.534670+0000","last_became_peered":"2024-05-24T11:36:48.534670+0000","last_unstale":"2024-05-24T11:36:48.536673+0000","last_undegraded":"2024-05-24T11:36:48.536673+0000","last_fullsized":"2024-05-24T11:36:48.536673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:24:49.492069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518418+0000","last_change":"2024-05-24T11:36:48.521113+0000","last_active":"2024-05-24T11:36:49.518418+0000","last_peered":"2024-05-24T11:36:49.518418+0000","last_clean":"2024-05-24T11:36:49.518418+0000","last_became_active":"2024-05-24T11:36:48.520379+0000","last_became_peered":"2024-05-24T11:36:48.520379+0000","last_unstale":"2024-05-24T11:36:49.518418+0000","last_undegraded":"2024-05-24T11:36:49.518418+0000","last_fullsized":"2024-05-24T11:36:49.518418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:07:17.112794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.543923+0000","last_change":"2024-05-24T11:36:48.529329+0000","last_active":"2024-05-24T11:36:49.543923+0000","last_peered":"2024-05-24T11:36:49.543923+0000","last_clean":"2024-05-24T11:36:49.543923+0000","last_became_active":"2024-05-24T11:36:48.528994+0000","last_became_peered":"2024-05-24T11:36:48.528994+0000","last_unstale":"2024-05-24T11:36:49.543923+0000","last_undegraded":"2024-05-24T11:36:49.543923+0000","last_fullsized":"2024-05-24T11:36:49.543923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:23:37.166318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.543981+0000","last_change":"2024-05-24T11:36:48.519429+0000","last_active":"2024-05-24T11:36:49.543981+0000","last_peered":"2024-05-24T11:36:49.543981+0000","last_clean":"2024-05-24T11:36:49.543981+0000","last_became_active":"2024-05-24T11:36:48.519085+0000","last_became_peered":"2024-05-24T11:36:48.519085+0000","last_unstale":"2024-05-24T11:36:49.543981+0000","last_undegraded":"2024-05-24T11:36:49.543981+0000","last_fullsized":"2024-05-24T11:36:49.543981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:41:55.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811057+0000","last_change":"2024-05-24T11:36:48.528932+0000","last_active":"2024-05-24T11:36:49.811057+0000","last_peered":"2024-05-24T11:36:49.811057+0000","last_clean":"2024-05-24T11:36:49.811057+0000","last_became_active":"2024-05-24T11:36:48.528413+0000","last_became_peered":"2024-05-24T11:36:48.528413+0000","last_unstale":"2024-05-24T11:36:49.811057+0000","last_undegraded":"2024-05-24T11:36:49.811057+0000","last_fullsized":"2024-05-24T11:36:49.811057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:55:47.051779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045272+0000","last_change":"2024-05-24T11:36:48.526905+0000","last_active":"2024-05-24T11:36:50.045272+0000","last_peered":"2024-05-24T11:36:50.045272+0000","last_clean":"2024-05-24T11:36:50.045272+0000","last_became_active":"2024-05-24T11:36:48.526608+0000","last_became_peered":"2024-05-24T11:36:48.526608+0000","last_unstale":"2024-05-24T11:36:50.045272+0000","last_undegraded":"2024-05-24T11:36:50.045272+0000","last_fullsized":"2024-05-24T11:36:50.045272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:32:04.831995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518371+0000","last_change":"2024-05-24T11:36:48.523597+0000","last_active":"2024-05-24T11:36:49.518371+0000","last_peered":"2024-05-24T11:36:49.518371+0000","last_clean":"2024-05-24T11:36:49.518371+0000","last_became_active":"2024-05-24T11:36:48.523279+0000","last_became_peered":"2024-05-24T11:36:48.523279+0000","last_unstale":"2024-05-24T11:36:49.518371+0000","last_undegraded":"2024-05-24T11:36:49.518371+0000","last_fullsized":"2024-05-24T11:36:49.518371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:49:46.063341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.544125+0000","last_change":"2024-05-24T11:36:48.526836+0000","last_active":"2024-05-24T11:36:49.544125+0000","last_peered":"2024-05-24T11:36:49.544125+0000","last_clean":"2024-05-24T11:36:49.544125+0000","last_became_active":"2024-05-24T11:36:48.523838+0000","last_became_peered":"2024-05-24T11:36:48.523838+0000","last_unstale":"2024-05-24T11:36:49.544125+0000","last_undegraded":"2024-05-24T11:36:49.544125+0000","last_fullsized":"2024-05-24T11:36:49.544125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:01:13.223240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044611+0000","last_change":"2024-05-24T11:36:48.532826+0000","last_active":"2024-05-24T11:36:50.044611+0000","last_peered":"2024-05-24T11:36:50.044611+0000","last_clean":"2024-05-24T11:36:50.044611+0000","last_became_active":"2024-05-24T11:36:48.532526+0000","last_became_peered":"2024-05-24T11:36:48.532526+0000","last_unstale":"2024-05-24T11:36:50.044611+0000","last_undegraded":"2024-05-24T11:36:50.044611+0000","last_fullsized":"2024-05-24T11:36:50.044611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:15:10.133493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044873+0000","last_change":"2024-05-24T11:36:48.527805+0000","last_active":"2024-05-24T11:36:50.044873+0000","last_peered":"2024-05-24T11:36:50.044873+0000","last_clean":"2024-05-24T11:36:50.044873+0000","last_became_active":"2024-05-24T11:36:48.526047+0000","last_became_peered":"2024-05-24T11:36:48.526047+0000","last_unstale":"2024-05-24T11:36:50.044873+0000","last_undegraded":"2024-05-24T11:36:50.044873+0000","last_fullsized":"2024-05-24T11:36:50.044873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:20:56.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044374+0000","last_change":"2024-05-24T11:36:48.519295+0000","last_active":"2024-05-24T11:36:50.044374+0000","last_peered":"2024-05-24T11:36:50.044374+0000","last_clean":"2024-05-24T11:36:50.044374+0000","last_became_active":"2024-05-24T11:36:48.518955+0000","last_became_peered":"2024-05-24T11:36:48.518955+0000","last_unstale":"2024-05-24T11:36:50.044374+0000","last_undegraded":"2024-05-24T11:36:50.044374+0000","last_fullsized":"2024-05-24T11:36:50.044374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:39:14.437903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518558+0000","last_change":"2024-05-24T11:36:48.525324+0000","last_active":"2024-05-24T11:36:49.518558+0000","last_peered":"2024-05-24T11:36:49.518558+0000","last_clean":"2024-05-24T11:36:49.518558+0000","last_became_active":"2024-05-24T11:36:48.525045+0000","last_became_peered":"2024-05-24T11:36:48.525045+0000","last_unstale":"2024-05-24T11:36:49.518558+0000","last_undegraded":"2024-05-24T11:36:49.518558+0000","last_fullsized":"2024-05-24T11:36:49.518558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:19:52.341136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045101+0000","last_change":"2024-05-24T11:36:48.528095+0000","last_active":"2024-05-24T11:36:50.045101+0000","last_peered":"2024-05-24T11:36:50.045101+0000","last_clean":"2024-05-24T11:36:50.045101+0000","last_became_active":"2024-05-24T11:36:48.527814+0000","last_became_peered":"2024-05-24T11:36:48.527814+0000","last_unstale":"2024-05-24T11:36:50.045101+0000","last_undegraded":"2024-05-24T11:36:50.045101+0000","last_fullsized":"2024-05-24T11:36:50.045101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:12:42.591269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518210+0000","last_change":"2024-05-24T11:36:48.528775+0000","last_active":"2024-05-24T11:36:49.518210+0000","last_peered":"2024-05-24T11:36:49.518210+0000","last_clean":"2024-05-24T11:36:49.518210+0000","last_became_active":"2024-05-24T11:36:48.528467+0000","last_became_peered":"2024-05-24T11:36:48.528467+0000","last_unstale":"2024-05-24T11:36:49.518210+0000","last_undegraded":"2024-05-24T11:36:49.518210+0000","last_fullsized":"2024-05-24T11:36:49.518210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:10:59.444142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5806,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":17,"num_read_kb":16,"num_write":156,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":17165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645322,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":835584,"data_stored":399105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":10,"up_from":65,"seq":279172874254,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40732,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704420,"statfs":{"total":95995035648,"available":95953326080,"internally_reserved":0,"allocated":1462272,"data_stored":991030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":40237505},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":9,"up_from":61,"seq":261993005074,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":860160,"data_stored":398690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.499}]},{"osd":3,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.39000000000000001}]},{"osd":4,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.622}]},{"osd":8,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.66600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":8,"up_from":54,"seq":231928234005,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":399068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61799999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68600000000000005}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66200000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":7,"up_from":47,"seq":201863462936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1425408,"data_stored":989028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.438}]},{"osd":1,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.5},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri May 24 11:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.76400000000000001}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.80500000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]},{"osd":6,"up_from":43,"seq":184683593756,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31924,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713228,"statfs":{"total":95995035648,"available":95962345472,"internally_reserved":0,"allocated":823296,"data_stored":397568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.32900000000000001}]},{"osd":1,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.312},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.35999999999999999}]},{"osd":4,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri May 24 11:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.495}]},{"osd":7,"last update":"Fri May 24 11:36:07 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68999999999999995}]},{"osd":8,"last update":"Fri May 24 11:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61599999999999999}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.47299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":1,"up_from":12,"seq":51539607598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40128,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93705024,"statfs":{"total":95995035648,"available":95953944576,"internally_reserved":0,"allocated":835584,"data_stored":399132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.33700000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.747,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri May 24 11:36:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.81200000000000006}]},{"osd":3,"last update":"Fri May 24 11:36:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.74199999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri May 24 11:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.47799999999999998,"15min":0.47299999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri May 24 11:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.87}]},{"osd":6,"last update":"Fri May 24 11:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73699999999999999}]},{"osd":7,"last update":"Fri May 24 11:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":1.095}]},{"osd":8,"last update":"Fri May 24 11:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.022},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.85999999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":0,"up_from":8,"seq":34359738418,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31944,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713208,"statfs":{"total":95995035648,"available":95962324992,"internally_reserved":0,"allocated":851968,"data_stored":399046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 24 11:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.33300000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri May 24 11:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.45000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri May 24 11:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.40600000000000003,"15min":0.39600000000000002},"min":{"1min":0.23799999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri May 24 11:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82699999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri May 24 11:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.47599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72199999999999998,"5min":0.755,"15min":0.755},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.46999999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri May 24 11:36:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.32800000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri May 24 11:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55000000000000004}]},{"osd":8,"last update":"Fri May 24 11:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.69899999999999995}]},{"osd":9,"last update":"Fri May 24 11:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":2,"up_from":17,"seq":73014444075,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":398848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.746,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.36399999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.85699999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri May 24 11:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77700000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri May 24 11:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72599999999999998,"5min":0.746,"15min":0.746},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri May 24 11:36:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.433,"15min":0.42399999999999999},"min":{"1min":0.313,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.436},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Fri May 24 11:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri May 24 11:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57599999999999996}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.62},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.76900000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182438,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31928,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713224,"statfs":{"total":95995035648,"available":95962341376,"internally_reserved":0,"allocated":827392,"data_stored":397341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35999999999999999,"15min":0.35899999999999999},"min":{"1min":0.25800000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46999999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.36399999999999999,"15min":0.36199999999999999},"min":{"1min":0.28599999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.502,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri May 24 11:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri May 24 11:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri May 24 11:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.45000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri May 24 11:36:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84299999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.439}]},{"osd":6,"last update":"Fri May 24 11:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45400000000000001}]},{"osd":7,"last update":"Fri May 24 11:36:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.75600000000000001}]},{"osd":8,"last update":"Fri May 24 11:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.66900000000000004}]},{"osd":9,"last update":"Fri May 24 11:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.499},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51100000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953508,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":398186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.502,"15min":0.495},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.37,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.308,"5min":0.249,"15min":0.249},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40899999999999997,"15min":0.40300000000000002},"min":{"1min":0.28799999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.437}]},{"osd":2,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83299999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.41499999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89800000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri May 24 11:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.874,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri May 24 11:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61599999999999999,"15min":0.60299999999999998},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.433,"5min":0.378,"15min":0.378},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri May 24 11:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64200000000000002}]},{"osd":8,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.626}]},{"osd":9,"last update":"Fri May 24 11:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32504,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712648,"statfs":{"total":95995035648,"available":95961751552,"internally_reserved":0,"allocated":1421312,"data_stored":987514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76100000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.28499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.51300000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.40500000000000003,"15min":0.40899999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.621}]},{"osd":3,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.441,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri May 24 11:36:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri May 24 11:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.73099999999999998}]},{"osd":8,"last update":"Fri May 24 11:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.83099999999999996}]},{"osd":9,"last update":"Fri May 24 11:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-24T11:36:57.432 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-24T11:36:57.433 INFO:tasks.cephadm:Setting up client nodes... 2024-05-24T11:36:57.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-24T11:36:57.694 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:36:57.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:57 smithi039 ceph-mon[42919]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:57.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:57 smithi039 ceph-mon[42919]: pgmap v224: 97 pgs: 97 active+clean; 582 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.4 KiB/s wr, 22 op/s 2024-05-24T11:36:57.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:57 smithi039 ceph-mon[42919]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-24T11:36:57.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:57 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1033092850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:57 smithi053 ceph-mon[47448]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:57 smithi053 ceph-mon[47448]: pgmap v224: 97 pgs: 97 active+clean; 582 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.4 KiB/s wr, 22 op/s 2024-05-24T11:36:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:57 smithi053 ceph-mon[47448]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-24T11:36:57.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:57 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1033092850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:57.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:57 smithi137 ceph-mon[47907]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:36:57.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:57 smithi137 ceph-mon[47907]: pgmap v224: 97 pgs: 97 active+clean; 582 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.4 KiB/s wr, 22 op/s 2024-05-24T11:36:57.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:57 smithi137 ceph-mon[47907]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-24T11:36:57.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:57 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1033092850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.780+0000 7f99a6148700 1 -- 172.21.15.39:0/133067221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 msgr2=0x7f99a0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.780+0000 7f99a6148700 1 --2- 172.21.15.39:0/133067221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 0x7f99a0101410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f99940099c0 tx=0x7f9994009cd0 comp rx=0 tx=0).stop 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.780+0000 7f99a6148700 1 -- 172.21.15.39:0/133067221 shutdown_connections 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.780+0000 7f99a6148700 1 --2- 172.21.15.39:0/133067221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99a0102b90 0x7f99a0109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.780+0000 7f99a6148700 1 --2- 172.21.15.39:0/133067221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 0x7f99a0101410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.780+0000 7f99a6148700 1 --2- 172.21.15.39:0/133067221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99a01021f0 0x7f99a0102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.780+0000 7f99a6148700 1 -- 172.21.15.39:0/133067221 >> 172.21.15.39:0/133067221 conn(0x7f99a00fc5b0 msgr2=0x7f99a00fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.780+0000 7f99a6148700 1 -- 172.21.15.39:0/133067221 shutdown_connections 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.780+0000 7f99a6148700 1 -- 172.21.15.39:0/133067221 wait complete. 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f99a6148700 1 Processor -- start 2024-05-24T11:36:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f99a6148700 1 -- start start 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f99a6148700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 0x7f99a0196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f99a6148700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99a01021f0 0x7f99a0197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f99a6148700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99a0102b90 0x7f99a019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f99a6148700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a019e8e0 con 0x7f99a0100ff0 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f99a6148700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a019ea20 con 0x7f99a01021f0 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f99a6148700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a019ebe0 con 0x7f99a0102b90 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f999f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 0x7f99a0196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f999f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 0x7f99a0196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36838/0 (socket says 172.21.15.39:36838) 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.781+0000 7f999f7fe700 1 -- 172.21.15.39:0/1806569492 learned_addr learned my addr 172.21.15.39:0/1806569492 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999effd700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99a01021f0 0x7f99a0197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999ffff700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99a0102b90 0x7f99a019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999f7fe700 1 -- 172.21.15.39:0/1806569492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99a0102b90 msgr2=0x7f99a019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999f7fe700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99a0102b90 0x7f99a019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999f7fe700 1 -- 172.21.15.39:0/1806569492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99a01021f0 msgr2=0x7f99a0197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999f7fe700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99a01021f0 0x7f99a0197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999f7fe700 1 -- 172.21.15.39:0/1806569492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940095f0 con 0x7f99a0100ff0 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999ffff700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99a0102b90 0x7f99a019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999f7fe700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 0x7f99a0196e30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9994000c00 tx=0x7f9994009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999cff9700 1 -- 172.21.15.39:0/1806569492 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999401d070 con 0x7f99a0100ff0 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999cff9700 1 -- 172.21.15.39:0/1806569492 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9994003fe0 con 0x7f99a0100ff0 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f999cff9700 1 -- 172.21.15.39:0/1806569492 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999400fd40 con 0x7f99a0100ff0 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a019ee60 con 0x7f99a0100ff0 2024-05-24T11:36:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.782+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a019f1c0 con 0x7f99a0100ff0 2024-05-24T11:36:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.783+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a01084d0 con 0x7f99a0100ff0 2024-05-24T11:36:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.784+0000 7f999cff9700 1 -- 172.21.15.39:0/1806569492 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f999400f460 con 0x7f99a0100ff0 2024-05-24T11:36:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.785+0000 7f999cff9700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99880a1230 0x7f99880a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:36:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.785+0000 7f999effd700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99880a1230 0x7f99880a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:36:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.785+0000 7f999cff9700 1 -- 172.21.15.39:0/1806569492 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f99940cd6f0 con 0x7f99a0100ff0 2024-05-24T11:36:58.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.785+0000 7f999effd700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99880a1230 0x7f99880a36f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9990005950 tx=0x7f999000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:36:58.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:58.789+0000 7f999cff9700 1 -- 172.21.15.39:0/1806569492 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f999405cd60 con 0x7f99a0100ff0 2024-05-24T11:36:58.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:58 smithi053 ceph-mon[47448]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:36:58.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:58 smithi053 ceph-mon[47448]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:36:58.989 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:58 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-b[84735]: 2024-05-24T11:36:58.717+0000 7f0de1fce700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-24T11:36:58.990 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:36:58 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-b[84735]: 2024-05-24T11:36:58.717+0000 7f0de1fce700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-24T11:36:58.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:58 smithi137 ceph-mon[47907]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:36:58.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:58 smithi137 ceph-mon[47907]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:36:59.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:58 smithi039 ceph-mon[42919]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:36:59.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:58 smithi039 ceph-mon[42919]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:36:59.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.077+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f99a010c6f0 con 0x7f99a0100ff0 2024-05-24T11:36:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.086+0000 7f999cff9700 1 -- 172.21.15.39:0/1806569492 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f9994096f60 con 0x7f99a0100ff0 2024-05-24T11:36:59.091 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-05-24T11:36:59.091 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDbe1Bm8IS+BBAApZtjq2WjZdMjDeUxw8uJYg== 2024-05-24T11:36:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99880a1230 msgr2=0x7f99880a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99880a1230 0x7f99880a36f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9990005950 tx=0x7f999000b410 comp rx=0 tx=0).stop 2024-05-24T11:36:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 msgr2=0x7f99a0196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:36:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 0x7f99a0196e30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9994000c00 tx=0x7f9994009580 comp rx=0 tx=0).stop 2024-05-24T11:36:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 shutdown_connections 2024-05-24T11:36:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99880a1230 0x7f99880a36f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99a0102b90 0x7f99a019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99a0100ff0 0x7f99a0196e30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 --2- 172.21.15.39:0/1806569492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99a01021f0 0x7f99a0197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:36:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 >> 172.21.15.39:0/1806569492 conn(0x7f99a00fc5b0 msgr2=0x7f99a0105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:36:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 shutdown_connections 2024-05-24T11:36:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:36:59.088+0000 7f99a6148700 1 -- 172.21.15.39:0/1806569492 wait complete. 2024-05-24T11:36:59.460 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:36:59.461 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-24T11:36:59.461 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-24T11:36:59.503 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-24T11:36:59.503 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-24T11:36:59.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-24T11:36:59.772 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:36:59.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:59 smithi039 ceph-mon[42919]: pgmap v225: 97 pgs: 97 active+clean; 583 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-05-24T11:36:59.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:59 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1806569492' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-24T11:36:59.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:36:59 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1806569492' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-24T11:36:59.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:59 smithi053 ceph-mon[47448]: pgmap v225: 97 pgs: 97 active+clean; 583 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-05-24T11:36:59.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:59 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1806569492' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-24T11:36:59.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:36:59 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1806569492' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-24T11:36:59.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:59 smithi137 ceph-mon[47907]: pgmap v225: 97 pgs: 97 active+clean; 583 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-05-24T11:36:59.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:59 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1806569492' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-24T11:36:59.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:36:59 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1806569492' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-24T11:37:00.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.802+0000 7f81982ae700 1 -- 172.21.15.39:0/4008691952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8190105f00 msgr2=0x7f81901062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:00.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.802+0000 7f81982ae700 1 --2- 172.21.15.39:0/4008691952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8190105f00 0x7f81901062e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f818c009990 tx=0x7f818c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:00.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.802+0000 7f81982ae700 1 -- 172.21.15.39:0/4008691952 shutdown_connections 2024-05-24T11:37:00.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.802+0000 7f81982ae700 1 --2- 172.21.15.39:0/4008691952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8190100890 0x7f819010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:00.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.802+0000 7f81982ae700 1 --2- 172.21.15.39:0/4008691952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8190105f00 0x7f81901062e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:00.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.802+0000 7f81982ae700 1 --2- 172.21.15.39:0/4008691952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81900ffed0 0x7f8190100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:00.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.802+0000 7f81982ae700 1 -- 172.21.15.39:0/4008691952 >> 172.21.15.39:0/4008691952 conn(0x7f8190075f80 msgr2=0x7f8190076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:00.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 -- 172.21.15.39:0/4008691952 shutdown_connections 2024-05-24T11:37:00.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 -- 172.21.15.39:0/4008691952 wait complete. 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 Processor -- start 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 -- start start 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81900ffed0 0x7f81901971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8190100890 0x7f8190197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8190105f00 0x7f819019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819019c0a0 con 0x7f81900ffed0 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819019c210 con 0x7f8190105f00 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.803+0000 7f81982ae700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819019c3d0 con 0x7f8190100890 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819604a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81900ffed0 0x7f81901971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819604a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81900ffed0 0x7f81901971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36868/0 (socket says 172.21.15.39:36868) 2024-05-24T11:37:00.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819604a700 1 -- 172.21.15.39:0/3612839340 learned_addr learned my addr 172.21.15.39:0/3612839340 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f8195849700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8190100890 0x7f8190197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819684b700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8190105f00 0x7f819019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819604a700 1 -- 172.21.15.39:0/3612839340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8190100890 msgr2=0x7f8190197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819604a700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8190100890 0x7f8190197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819604a700 1 -- 172.21.15.39:0/3612839340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8190105f00 msgr2=0x7f819019ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819604a700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8190105f00 0x7f819019ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819604a700 1 -- 172.21.15.39:0/3612839340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0095f0 con 0x7f81900ffed0 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f819604a700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81900ffed0 0x7f81901971f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f818c006010 tx=0x7f818c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f81877fe700 1 -- 172.21.15.39:0/3612839340 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c01d070 con 0x7f81900ffed0 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f81877fe700 1 -- 172.21.15.39:0/3612839340 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f818c00fd70 con 0x7f81900ffed0 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819019c650 con 0x7f81900ffed0 2024-05-24T11:37:00.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f81877fe700 1 -- 172.21.15.39:0/3612839340 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c022dd0 con 0x7f81900ffed0 2024-05-24T11:37:00.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.804+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81901a3f10 con 0x7f81900ffed0 2024-05-24T11:37:00.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.805+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81901015b0 con 0x7f81900ffed0 2024-05-24T11:37:00.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.806+0000 7f81877fe700 1 -- 172.21.15.39:0/3612839340 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f818c022570 con 0x7f81900ffed0 2024-05-24T11:37:00.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.807+0000 7f81877fe700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f817c0a11b0 0x7f817c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:00.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.807+0000 7f8195849700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f817c0a11b0 0x7f817c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:00.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.807+0000 7f81877fe700 1 -- 172.21.15.39:0/3612839340 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f818c0ce1c0 con 0x7f81900ffed0 2024-05-24T11:37:00.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.807+0000 7f8195849700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f817c0a11b0 0x7f817c0a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8190198700 tx=0x7f818000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:00.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:00.810+0000 7f81877fe700 1 -- 172.21.15.39:0/3612839340 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f818c097930 con 0x7f81900ffed0 2024-05-24T11:37:01.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.069+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f81901984b0 con 0x7f81900ffed0 2024-05-24T11:37:01.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.074+0000 7f81877fe700 1 -- 172.21.15.39:0/3612839340 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260283 (secure 0 0 0) 0x7f818c09af50 con 0x7f81900ffed0 2024-05-24T11:37:01.083 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:01.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f817c0a11b0 msgr2=0x7f817c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f817c0a11b0 0x7f817c0a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8190198700 tx=0x7f818000a380 comp rx=0 tx=0).stop 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81900ffed0 msgr2=0x7f81901971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81900ffed0 0x7f81901971f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f818c006010 tx=0x7f818c004730 comp rx=0 tx=0).stop 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 shutdown_connections 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f817c0a11b0 0x7f817c0a3670 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8190105f00 0x7f819019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81900ffed0 0x7f81901971f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 --2- 172.21.15.39:0/3612839340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8190100890 0x7f8190197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 >> 172.21.15.39:0/3612839340 conn(0x7f8190075f80 msgr2=0x7f81900ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.077+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 shutdown_connections 2024-05-24T11:37:01.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:01.078+0000 7f81982ae700 1 -- 172.21.15.39:0/3612839340 wait complete. 2024-05-24T11:37:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":2090515453},{"type":"v1","addr":"172.21.15.39:6801","nonce":2090515453}]},"active_addr":"172.21.15.39:6801/2090515453","active_change":"2024-05-24T11:31:41.814809+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24121,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.39:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3248772673}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":870992183}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":333059399}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":338214439}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3290494740}]}]} 2024-05-24T11:37:01.472 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-24T11:37:01.472 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-24T11:37:01.472 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-24T11:37:01.703 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:01.820 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:01 smithi039 ceph-mon[42919]: pgmap v226: 97 pgs: 97 active+clean; 624 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.9 KiB/s wr, 26 op/s 2024-05-24T11:37:01.820 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:01 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3612839340' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-24T11:37:01.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:01 smithi053 ceph-mon[47448]: pgmap v226: 97 pgs: 97 active+clean; 624 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.9 KiB/s wr, 26 op/s 2024-05-24T11:37:01.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:01 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3612839340' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-24T11:37:01.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:01 smithi137 ceph-mon[47907]: pgmap v226: 97 pgs: 97 active+clean; 624 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.9 KiB/s wr, 26 op/s 2024-05-24T11:37:01.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:01 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3612839340' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-24T11:37:02.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 -- 172.21.15.39:0/3875527899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8898105f00 msgr2=0x7f88981062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:02.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 --2- 172.21.15.39:0/3875527899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8898105f00 0x7f88981062e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f888c009990 tx=0x7f888c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 -- 172.21.15.39:0/3875527899 shutdown_connections 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 --2- 172.21.15.39:0/3875527899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100890 0x7f889810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 --2- 172.21.15.39:0/3875527899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8898105f00 0x7f88981062e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 --2- 172.21.15.39:0/3875527899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88980ffed0 0x7f8898100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 -- 172.21.15.39:0/3875527899 >> 172.21.15.39:0/3875527899 conn(0x7f8898075f80 msgr2=0x7f8898076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 -- 172.21.15.39:0/3875527899 shutdown_connections 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 -- 172.21.15.39:0/3875527899 wait complete. 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 Processor -- start 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.727+0000 7f889f23f700 1 -- start start 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889f23f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88980ffed0 0x7f8898197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889f23f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100890 0x7f88981978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889f23f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8898105f00 0x7f889819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889f23f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889819c220 con 0x7f8898105f00 2024-05-24T11:37:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889f23f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889819c390 con 0x7f8898100890 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889f23f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889819c550 con 0x7f88980ffed0 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889cfdb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88980ffed0 0x7f8898197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889d7dc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8898105f00 0x7f889819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889cfdb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88980ffed0 0x7f8898197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:60958/0 (socket says 172.21.15.39:60958) 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889cfdb700 1 -- 172.21.15.39:0/1048136620 learned_addr learned my addr 172.21.15.39:0/1048136620 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f8897fff700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100890 0x7f88981978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889d7dc700 1 -- 172.21.15.39:0/1048136620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88980ffed0 msgr2=0x7f8898197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889d7dc700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88980ffed0 0x7f8898197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889d7dc700 1 -- 172.21.15.39:0/1048136620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100890 msgr2=0x7f88981978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889d7dc700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100890 0x7f88981978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889d7dc700 1 -- 172.21.15.39:0/1048136620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0095f0 con 0x7f8898105f00 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.728+0000 7f889d7dc700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8898105f00 0x7f889819bc10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8890007f10 tx=0x7f889000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.729+0000 7f8895ffb700 1 -- 172.21.15.39:0/1048136620 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890009ea0 con 0x7f8898105f00 2024-05-24T11:37:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.729+0000 7f8895ffb700 1 -- 172.21.15.39:0/1048136620 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f889000f040 con 0x7f8898105f00 2024-05-24T11:37:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.729+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889819c830 con 0x7f8898105f00 2024-05-24T11:37:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.729+0000 7f8895ffb700 1 -- 172.21.15.39:0/1048136620 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890004280 con 0x7f8898105f00 2024-05-24T11:37:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.729+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88981a4090 con 0x7f8898105f00 2024-05-24T11:37:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.730+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88981015b0 con 0x7f8898105f00 2024-05-24T11:37:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.731+0000 7f8895ffb700 1 -- 172.21.15.39:0/1048136620 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f889000db20 con 0x7f8898105f00 2024-05-24T11:37:02.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.731+0000 7f8895ffb700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f88880a11b0 0x7f88880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:02.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.731+0000 7f889cfdb700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f88880a11b0 0x7f88880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:02.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.731+0000 7f8895ffb700 1 -- 172.21.15.39:0/1048136620 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f88900cc180 con 0x7f8898105f00 2024-05-24T11:37:02.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.732+0000 7f889cfdb700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f88880a11b0 0x7f88880a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f888c0098e0 tx=0x7f888c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:02.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.736+0000 7f8895ffb700 1 -- 172.21.15.39:0/1048136620 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8890095960 con 0x7f8898105f00 2024-05-24T11:37:02.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.939+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8898198630 con 0x7f8898105f00 2024-05-24T11:37:02.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.941+0000 7f8895ffb700 1 -- 172.21.15.39:0/1048136620 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+22061 (secure 0 0 0) 0x7f8890098f80 con 0x7f8898105f00 2024-05-24T11:37:02.947 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:02.947 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:49.502598+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:37:02.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f88880a11b0 msgr2=0x7f88880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f88880a11b0 0x7f88880a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f888c0098e0 tx=0x7f888c005960 comp rx=0 tx=0).stop 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8898105f00 msgr2=0x7f889819bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8898105f00 0x7f889819bc10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8890007f10 tx=0x7f889000d3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 shutdown_connections 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f88880a11b0 0x7f88880a3670 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88980ffed0 0x7f8898197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8898105f00 0x7f889819bc10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 --2- 172.21.15.39:0/1048136620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100890 0x7f88981978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 >> 172.21.15.39:0/1048136620 conn(0x7f8898075f80 msgr2=0x7f8898068c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 shutdown_connections 2024-05-24T11:37:02.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:02.943+0000 7f889f23f700 1 -- 172.21.15.39:0/1048136620 wait complete. 2024-05-24T11:37:03.341 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-24T11:37:03.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-24T11:37:03.571 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:03.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:03 smithi053 ceph-mon[47448]: pgmap v227: 97 pgs: 97 active+clean; 624 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-05-24T11:37:03.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:03 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1048136620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:03.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:03 smithi137 ceph-mon[47907]: pgmap v227: 97 pgs: 97 active+clean; 624 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-05-24T11:37:03.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:03 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1048136620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:04.036 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:03 smithi039 ceph-mon[42919]: pgmap v227: 97 pgs: 97 active+clean; 624 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-05-24T11:37:04.037 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:03 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1048136620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:04.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.621+0000 7f6a68aef700 1 -- 172.21.15.39:0/2751093155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6406a310 msgr2=0x7f6a64105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:04.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.621+0000 7f6a68aef700 1 --2- 172.21.15.39:0/2751093155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6406a310 0x7f6a64105be0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009990 tx=0x7f6a58009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.621+0000 7f6a68aef700 1 -- 172.21.15.39:0/2751093155 shutdown_connections 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.621+0000 7f6a68aef700 1 --2- 172.21.15.39:0/2751093155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a64069960 0x7f6a64069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.621+0000 7f6a68aef700 1 --2- 172.21.15.39:0/2751093155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a641062c0 0x7f6a6410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.621+0000 7f6a68aef700 1 --2- 172.21.15.39:0/2751093155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6406a310 0x7f6a64105be0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.621+0000 7f6a68aef700 1 -- 172.21.15.39:0/2751093155 >> 172.21.15.39:0/2751093155 conn(0x7f6a64076f40 msgr2=0x7f6a64077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.621+0000 7f6a68aef700 1 -- 172.21.15.39:0/2751093155 shutdown_connections 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.622+0000 7f6a68aef700 1 -- 172.21.15.39:0/2751093155 wait complete. 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.622+0000 7f6a68aef700 1 Processor -- start 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.622+0000 7f6a68aef700 1 -- start start 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a68aef700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a64069960 0x7f6a64197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a68aef700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6406a310 0x7f6a641977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a68aef700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641062c0 0x7f6a6419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a68aef700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6419c190 con 0x7f6a641062c0 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a68aef700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6419c300 con 0x7f6a64069960 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a68aef700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6419c4c0 con 0x7f6a6406a310 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a62d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641062c0 0x7f6a6419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a62d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641062c0 0x7f6a6419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36898/0 (socket says 172.21.15.39:36898) 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a62d9d700 1 -- 172.21.15.39:0/1085573433 learned_addr learned my addr 172.21.15.39:0/1085573433 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a6259c700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a64069960 0x7f6a64197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a61d9b700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6406a310 0x7f6a641977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a62d9d700 1 -- 172.21.15.39:0/1085573433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6406a310 msgr2=0x7f6a641977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a62d9d700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6406a310 0x7f6a641977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a62d9d700 1 -- 172.21.15.39:0/1085573433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a64069960 msgr2=0x7f6a64197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:04.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a62d9d700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a64069960 0x7f6a64197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a62d9d700 1 -- 172.21.15.39:0/1085573433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580095f0 con 0x7f6a641062c0 2024-05-24T11:37:04.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a62d9d700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641062c0 0x7f6a6419bb80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007d70 tx=0x7f6a5c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:04.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a537fe700 1 -- 172.21.15.39:0/1085573433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00ac90 con 0x7f6a641062c0 2024-05-24T11:37:04.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6419c7a0 con 0x7f6a641062c0 2024-05-24T11:37:04.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a537fe700 1 -- 172.21.15.39:0/1085573433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a5c00adf0 con 0x7f6a641062c0 2024-05-24T11:37:04.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a537fe700 1 -- 172.21.15.39:0/1085573433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0115e0 con 0x7f6a641062c0 2024-05-24T11:37:04.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.623+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a641a4000 con 0x7f6a641062c0 2024-05-24T11:37:04.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.624+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a641095e0 con 0x7f6a641062c0 2024-05-24T11:37:04.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.625+0000 7f6a537fe700 1 -- 172.21.15.39:0/1085573433 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6a5c012420 con 0x7f6a641062c0 2024-05-24T11:37:04.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.626+0000 7f6a537fe700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6a4c0a11b0 0x7f6a4c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:04.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.626+0000 7f6a6259c700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6a4c0a11b0 0x7f6a4c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:04.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.626+0000 7f6a537fe700 1 -- 172.21.15.39:0/1085573433 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6a5c0cc930 con 0x7f6a641062c0 2024-05-24T11:37:04.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.626+0000 7f6a6259c700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6a4c0a11b0 0x7f6a4c0a3670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400ac50 tx=0x7f6a5400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:04.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.630+0000 7f6a537fe700 1 -- 172.21.15.39:0/1085573433 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6a5c096220 con 0x7f6a641062c0 2024-05-24T11:37:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.835+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a641026c0 con 0x7f6a641062c0 2024-05-24T11:37:04.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.836+0000 7f6a537fe700 1 -- 172.21.15.39:0/1085573433 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+22061 (secure 0 0 0) 0x7f6a5c099840 con 0x7f6a641062c0 2024-05-24T11:37:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:04.842 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:49.502598+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6a4c0a11b0 msgr2=0x7f6a4c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6a4c0a11b0 0x7f6a4c0a3670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400ac50 tx=0x7f6a5400a380 comp rx=0 tx=0).stop 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641062c0 msgr2=0x7f6a6419bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641062c0 0x7f6a6419bb80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007d70 tx=0x7f6a5c00a340 comp rx=0 tx=0).stop 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 shutdown_connections 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6a4c0a11b0 0x7f6a4c0a3670 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641062c0 0x7f6a6419bb80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a64069960 0x7f6a64197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 --2- 172.21.15.39:0/1085573433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a6406a310 0x7f6a641977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 >> 172.21.15.39:0/1085573433 conn(0x7f6a64076f40 msgr2=0x7f6a640ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:04.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 shutdown_connections 2024-05-24T11:37:04.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:04.838+0000 7f6a68aef700 1 -- 172.21.15.39:0/1085573433 wait complete. 2024-05-24T11:37:05.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-24T11:37:05.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-24T11:37:05.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-24T11:37:05.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-24T11:37:05.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-24T11:37:05.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-24T11:37:05.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-24T11:37:05.221 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-24T11:37:05.221 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-24T11:37:05.221 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-24T11:37:05.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-24T11:37:05.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-24T11:37:05.666 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:05 smithi039 ceph-mon[42919]: pgmap v228: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 24 op/s 2024-05-24T11:37:05.666 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:05 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1085573433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:05.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:05 smithi053 ceph-mon[47448]: pgmap v228: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 24 op/s 2024-05-24T11:37:05.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:05 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1085573433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:05.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:05 smithi137 ceph-mon[47907]: pgmap v228: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 24 op/s 2024-05-24T11:37:05.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:05 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1085573433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:06.847 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:07.745 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:07.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.789+0000 7f7dca799700 1 -- 172.21.15.39:0/3140852557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc40692f0 msgr2=0x7f7dc410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:07.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.789+0000 7f7dca799700 1 --2- 172.21.15.39:0/3140852557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc40692f0 0x7f7dc410de60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00b0a0 tx=0x7f7dbc00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.790+0000 7f7dca799700 1 -- 172.21.15.39:0/3140852557 shutdown_connections 2024-05-24T11:37:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.790+0000 7f7dca799700 1 --2- 172.21.15.39:0/3140852557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dc4107020 0x7f7dc4107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.790+0000 7f7dca799700 1 --2- 172.21.15.39:0/3140852557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc40692f0 0x7f7dc410de60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.790+0000 7f7dca799700 1 --2- 172.21.15.39:0/3140852557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dc4068950 0x7f7dc4068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.790+0000 7f7dca799700 1 -- 172.21.15.39:0/3140852557 >> 172.21.15.39:0/3140852557 conn(0x7f7dc4075f80 msgr2=0x7f7dc4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.790+0000 7f7dca799700 1 -- 172.21.15.39:0/3140852557 shutdown_connections 2024-05-24T11:37:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.790+0000 7f7dca799700 1 -- 172.21.15.39:0/3140852557 wait complete. 2024-05-24T11:37:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.790+0000 7f7dca799700 1 Processor -- start 2024-05-24T11:37:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.790+0000 7f7dca799700 1 -- start start 2024-05-24T11:37:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dca799700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dc4068950 0x7f7dc40fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dca799700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dc40692f0 0x7f7dc41004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dca799700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc4107020 0x7f7dc4100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dca799700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc41a5800 con 0x7f7dc4107020 2024-05-24T11:37:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dca799700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc41a5970 con 0x7f7dc4068950 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dca799700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc41a5b30 con 0x7f7dc40692f0 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc8d36700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc4107020 0x7f7dc4100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc8d36700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc4107020 0x7f7dc4100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43490/0 (socket says 172.21.15.39:43490) 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc8d36700 1 -- 172.21.15.39:0/4123875553 learned_addr learned my addr 172.21.15.39:0/4123875553 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc3fff700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dc4068950 0x7f7dc40fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc37fe700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dc40692f0 0x7f7dc41004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc8d36700 1 -- 172.21.15.39:0/4123875553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dc40692f0 msgr2=0x7f7dc41004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc8d36700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dc40692f0 0x7f7dc41004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc8d36700 1 -- 172.21.15.39:0/4123875553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dc4068950 msgr2=0x7f7dc40fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc8d36700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7dc4068950 0x7f7dc40fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc8d36700 1 -- 172.21.15.39:0/4123875553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc009d00 con 0x7f7dc4107020 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc8d36700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc4107020 0x7f7dc4100a20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00b070 tx=0x7f7dbc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.791+0000 7f7dc17fa700 1 -- 172.21.15.39:0/4123875553 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc00e040 con 0x7f7dc4107020 2024-05-24T11:37:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.792+0000 7f7dc17fa700 1 -- 172.21.15.39:0/4123875553 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7dbc009460 con 0x7f7dc4107020 2024-05-24T11:37:07.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.792+0000 7f7dc17fa700 1 -- 172.21.15.39:0/4123875553 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc01b450 con 0x7f7dc4107020 2024-05-24T11:37:07.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.792+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc41a5db0 con 0x7f7dc4107020 2024-05-24T11:37:07.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.792+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc41a61d0 con 0x7f7dc4107020 2024-05-24T11:37:07.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.792+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7dc41026d0 con 0x7f7dc4107020 2024-05-24T11:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.794+0000 7f7dc17fa700 1 -- 172.21.15.39:0/4123875553 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7dbc01d480 con 0x7f7dc4107020 2024-05-24T11:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.794+0000 7f7dc17fa700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7db00a11b0 0x7f7db00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.795+0000 7f7dc3fff700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7db00a11b0 0x7f7db00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.795+0000 7f7dc17fa700 1 -- 172.21.15.39:0/4123875553 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7dbc0ce800 con 0x7f7dc4107020 2024-05-24T11:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.795+0000 7f7dc17fa700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] conn(0x7f7db00aa990 0x7f7db00acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.795+0000 7f7dc17fa700 1 -- 172.21.15.39:0/4123875553 --> [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7db00ad460 con 0x7f7db00aa990 2024-05-24T11:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.795+0000 7f7dc17fa700 1 -- 172.21.15.39:0/4123875553 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dbc0ceca0 con 0x7f7dc4107020 2024-05-24T11:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.795+0000 7f7dc3fff700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7db00a11b0 0x7f7db00a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7db4005950 tx=0x7f7db400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:07.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.795+0000 7f7dc37fe700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] conn(0x7f7db00aa990 0x7f7db00acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:07.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.796+0000 7f7dc37fe700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] conn(0x7f7db00aa990 0x7f7db00acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:07.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.796+0000 7f7dc17fa700 1 -- 172.21.15.39:0/4123875553 <== osd.1 v2:172.21.15.53:6800/3467330431 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7db00ad460 con 0x7f7db00aa990 2024-05-24T11:37:07.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.812+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 --> [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7dc4110fb0 con 0x7f7db00aa990 2024-05-24T11:37:07.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dc17fa700 1 -- 172.21.15.39:0/4123875553 <== osd.1 v2:172.21.15.53:6800/3467330431 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7dc4110fb0 con 0x7f7db00aa990 2024-05-24T11:37:07.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 >> [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] conn(0x7f7db00aa990 msgr2=0x7f7db00acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:07.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] conn(0x7f7db00aa990 0x7f7db00acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:07.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7db00a11b0 msgr2=0x7f7db00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:07.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7db00a11b0 0x7f7db00a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7db4005950 tx=0x7f7db400b410 comp rx=0 tx=0).stop 2024-05-24T11:37:07.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc4107020 msgr2=0x7f7dc4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:07.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 --2- 172.21.15.39:0/4123875553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc4107020 0x7f7dc4100a20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00b070 tx=0x7f7dbc00cd70 comp rx=0 tx=0).stop 2024-05-24T11:37:07.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dc8d36700 1 -- 172.21.15.39:0/4123875553 reap_dead start 2024-05-24T11:37:07.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 shutdown_connections 2024-05-24T11:37:07.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 >> 172.21.15.39:0/4123875553 conn(0x7f7dc4075f80 msgr2=0x7f7dc40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:07.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 shutdown_connections 2024-05-24T11:37:07.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:07.813+0000 7f7dca799700 1 -- 172.21.15.39:0/4123875553 wait complete. 2024-05-24T11:37:07.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:07 smithi053 ceph-mon[47448]: pgmap v229: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 22 op/s 2024-05-24T11:37:07.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:07 smithi137 ceph-mon[47907]: pgmap v229: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 22 op/s 2024-05-24T11:37:08.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:07 smithi039 ceph-mon[42919]: pgmap v229: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 22 op/s 2024-05-24T11:37:08.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.268+0000 7fea19812700 1 -- 172.21.15.39:0/1745606157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea14105f00 msgr2=0x7fea141062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:08.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.268+0000 7fea19812700 1 --2- 172.21.15.39:0/1745606157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea14105f00 0x7fea141062e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fea04009990 tx=0x7fea04009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:08.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.268+0000 7fea19812700 1 -- 172.21.15.39:0/1745606157 shutdown_connections 2024-05-24T11:37:08.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.268+0000 7fea19812700 1 --2- 172.21.15.39:0/1745606157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea14100890 0x7fea1410cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:08.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.268+0000 7fea19812700 1 --2- 172.21.15.39:0/1745606157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea14105f00 0x7fea141062e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:08.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.268+0000 7fea19812700 1 --2- 172.21.15.39:0/1745606157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140ffed0 0x7fea14100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:08.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.268+0000 7fea19812700 1 -- 172.21.15.39:0/1745606157 >> 172.21.15.39:0/1745606157 conn(0x7fea14075f80 msgr2=0x7fea14076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:08.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.268+0000 7fea19812700 1 -- 172.21.15.39:0/1745606157 shutdown_connections 2024-05-24T11:37:08.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.268+0000 7fea19812700 1 -- 172.21.15.39:0/1745606157 wait complete. 2024-05-24T11:37:08.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea19812700 1 Processor -- start 2024-05-24T11:37:08.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea19812700 1 -- start start 2024-05-24T11:37:08.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea19812700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140ffed0 0x7fea14101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea19812700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea14100890 0x7fea141020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea19812700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea14105f00 0x7fea141025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea19812700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea141a5820 con 0x7fea14100890 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea19812700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea141a5990 con 0x7fea140ffed0 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea19812700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea141a5b50 con 0x7fea14105f00 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea127fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea14100890 0x7fea141020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea127fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea14100890 0x7fea141020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43508/0 (socket says 172.21.15.39:43508) 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.269+0000 7fea127fc700 1 -- 172.21.15.39:0/2902407747 learned_addr learned my addr 172.21.15.39:0/2902407747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea12ffd700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140ffed0 0x7fea14101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea137fe700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea14105f00 0x7fea141025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea127fc700 1 -- 172.21.15.39:0/2902407747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea14105f00 msgr2=0x7fea141025f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea127fc700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea14105f00 0x7fea141025f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea127fc700 1 -- 172.21.15.39:0/2902407747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140ffed0 msgr2=0x7fea14101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea127fc700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea140ffed0 0x7fea14101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea127fc700 1 -- 172.21.15.39:0/2902407747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040095f0 con 0x7fea14100890 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea137fe700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea14105f00 0x7fea141025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea127fc700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea14100890 0x7fea141020b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fea0800b530 tx=0x7fea0800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea18810700 1 -- 172.21.15.39:0/2902407747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08011830 con 0x7fea14100890 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea18810700 1 -- 172.21.15.39:0/2902407747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea08011e20 con 0x7fea14100890 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea141a5e30 con 0x7fea14100890 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea18810700 1 -- 172.21.15.39:0/2902407747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0800f2e0 con 0x7fea14100890 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.270+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea141a62a0 con 0x7fea14100890 2024-05-24T11:37:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.271+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea1410ffe0 con 0x7fea14100890 2024-05-24T11:37:08.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.272+0000 7fea18810700 1 -- 172.21.15.39:0/2902407747 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fea08011990 con 0x7fea14100890 2024-05-24T11:37:08.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.273+0000 7fea18810700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe9fc0a11b0 0x7fe9fc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.273+0000 7fea12ffd700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe9fc0a11b0 0x7fe9fc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.273+0000 7fea18810700 1 -- 172.21.15.39:0/2902407747 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fea080cc2f0 con 0x7fea14100890 2024-05-24T11:37:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.273+0000 7fea18810700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] conn(0x7fe9fc0aa990 0x7fe9fc0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.273+0000 7fea18810700 1 -- 172.21.15.39:0/2902407747 --> [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9fc0ad460 con 0x7fe9fc0aa990 2024-05-24T11:37:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.273+0000 7fea18810700 1 -- 172.21.15.39:0/2902407747 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea080cea00 con 0x7fea14100890 2024-05-24T11:37:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.273+0000 7fea137fe700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] conn(0x7fe9fc0aa990 0x7fe9fc0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.273+0000 7fea12ffd700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe9fc0a11b0 0x7fe9fc0a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea0400b5c0 tx=0x7fea04005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.274+0000 7fea137fe700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] conn(0x7fe9fc0aa990 0x7fe9fc0acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.274+0000 7fea18810700 1 -- 172.21.15.39:0/2902407747 <== osd.3 v2:172.21.15.39:6810/445163923 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe9fc0ad460 con 0x7fe9fc0aa990 2024-05-24T11:37:08.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.290+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 --> [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea14104f00 con 0x7fe9fc0aa990 2024-05-24T11:37:08.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea18810700 1 -- 172.21.15.39:0/2902407747 <== osd.3 v2:172.21.15.39:6810/445163923 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fea14104f00 con 0x7fe9fc0aa990 2024-05-24T11:37:08.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] conn(0x7fe9fc0aa990 msgr2=0x7fe9fc0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:08.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] conn(0x7fe9fc0aa990 0x7fe9fc0acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:08.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe9fc0a11b0 msgr2=0x7fe9fc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:08.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe9fc0a11b0 0x7fe9fc0a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea0400b5c0 tx=0x7fea04005fb0 comp rx=0 tx=0).stop 2024-05-24T11:37:08.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea14100890 msgr2=0x7fea141020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:08.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 --2- 172.21.15.39:0/2902407747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea14100890 0x7fea141020b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fea0800b530 tx=0x7fea0800b840 comp rx=0 tx=0).stop 2024-05-24T11:37:08.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea137fe700 1 -- 172.21.15.39:0/2902407747 reap_dead start 2024-05-24T11:37:08.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 shutdown_connections 2024-05-24T11:37:08.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 >> 172.21.15.39:0/2902407747 conn(0x7fea14075f80 msgr2=0x7fea14068c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:08.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 shutdown_connections 2024-05-24T11:37:08.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:08.291+0000 7fea19812700 1 -- 172.21.15.39:0/2902407747 wait complete. 2024-05-24T11:37:09.031 INFO:teuthology.orchestra.run.smithi039.stdout:51539607602 2024-05-24T11:37:09.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-24T11:37:09.214 INFO:teuthology.orchestra.run.smithi039.stdout:107374182442 2024-05-24T11:37:09.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-24T11:37:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.141+0000 7f728a274700 1 -- 172.21.15.39:0/4169284065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7284107020 msgr2=0x7f7284107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.141+0000 7f728a274700 1 --2- 172.21.15.39:0/4169284065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7284107020 0x7f7284107400 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7274009990 tx=0x7f7274009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.141+0000 7f728a274700 1 -- 172.21.15.39:0/4169284065 shutdown_connections 2024-05-24T11:37:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.142+0000 7f728a274700 1 --2- 172.21.15.39:0/4169284065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72840692f0 0x7f728410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.142+0000 7f728a274700 1 --2- 172.21.15.39:0/4169284065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7284107020 0x7f7284107400 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.142+0000 7f728a274700 1 --2- 172.21.15.39:0/4169284065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7284068950 0x7f7284068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.142+0000 7f728a274700 1 -- 172.21.15.39:0/4169284065 >> 172.21.15.39:0/4169284065 conn(0x7f7284075f80 msgr2=0x7f7284076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.142+0000 7f728a274700 1 -- 172.21.15.39:0/4169284065 shutdown_connections 2024-05-24T11:37:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.142+0000 7f728a274700 1 -- 172.21.15.39:0/4169284065 wait complete. 2024-05-24T11:37:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.142+0000 7f728a274700 1 Processor -- start 2024-05-24T11:37:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.142+0000 7f728a274700 1 -- start start 2024-05-24T11:37:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f728a274700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7284068950 0x7f72840fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f728a274700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72840692f0 0x7f72841004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f728a274700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7284107020 0x7f7284100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f728a274700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72841a56c0 con 0x7f7284068950 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f728a274700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72841a5830 con 0x7f72840692f0 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f728a274700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72841a59f0 con 0x7f7284107020 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7283fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7284068950 0x7f72840fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7283fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7284068950 0x7f72840fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43516/0 (socket says 172.21.15.39:43516) 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7283fff700 1 -- 172.21.15.39:0/455481886 learned_addr learned my addr 172.21.15.39:0/455481886 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7288811700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7284107020 0x7f7284100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f72837fe700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72840692f0 0x7f72841004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7283fff700 1 -- 172.21.15.39:0/455481886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7284107020 msgr2=0x7f7284100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7283fff700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7284107020 0x7f7284100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7283fff700 1 -- 172.21.15.39:0/455481886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72840692f0 msgr2=0x7f72841004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7283fff700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72840692f0 0x7f72841004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7283fff700 1 -- 172.21.15.39:0/455481886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72740095f0 con 0x7f7284068950 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.143+0000 7f7283fff700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7284068950 0x7f72840fffa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7274006010 tx=0x7f7274004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.144+0000 7f72817fa700 1 -- 172.21.15.39:0/455481886 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727401d070 con 0x7f7284068950 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.144+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72841a5c70 con 0x7f7284068950 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.144+0000 7f72817fa700 1 -- 172.21.15.39:0/455481886 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f727400fb80 con 0x7f7284068950 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.144+0000 7f72817fa700 1 -- 172.21.15.39:0/455481886 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274022b90 con 0x7f7284068950 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.144+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72841a6090 con 0x7f7284068950 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.144+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72841026d0 con 0x7f7284068950 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.146+0000 7f72817fa700 1 -- 172.21.15.39:0/455481886 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f727400f6f0 con 0x7f7284068950 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.147+0000 7f72817fa700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f726c0a11b0 0x7f726c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.147+0000 7f72837fe700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f726c0a11b0 0x7f726c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.147+0000 7f72817fa700 1 -- 172.21.15.39:0/455481886 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72740ce690 con 0x7f7284068950 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.147+0000 7f72817fa700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] conn(0x7f726c0aa990 0x7f726c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.147+0000 7f72817fa700 1 -- 172.21.15.39:0/455481886 --> [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f726c0ad460 con 0x7f726c0aa990 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.147+0000 7f72817fa700 1 -- 172.21.15.39:0/455481886 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72740ceb30 con 0x7f7284068950 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.147+0000 7f72837fe700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f726c0a11b0 0x7f726c0a3670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7284101680 tx=0x7f727800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.147+0000 7f7288811700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] conn(0x7f726c0aa990 0x7f726c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.148+0000 7f7288811700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] conn(0x7f726c0aa990 0x7f726c0acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.149+0000 7f72817fa700 1 -- 172.21.15.39:0/455481886 <== osd.10 v2:172.21.15.53:6824/3257679839 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f726c0ad460 con 0x7f726c0aa990 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.165+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 --> [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7284111010 con 0x7f726c0aa990 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f72817fa700 1 -- 172.21.15.39:0/455481886 <== osd.10 v2:172.21.15.53:6824/3257679839 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7284111010 con 0x7f726c0aa990 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 >> [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] conn(0x7f726c0aa990 msgr2=0x7f726c0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] conn(0x7f726c0aa990 0x7f726c0acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f726c0a11b0 msgr2=0x7f726c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f726c0a11b0 0x7f726c0a3670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7284101680 tx=0x7f727800b410 comp rx=0 tx=0).stop 2024-05-24T11:37:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7284068950 msgr2=0x7f72840fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 --2- 172.21.15.39:0/455481886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7284068950 0x7f72840fffa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7274006010 tx=0x7f7274004730 comp rx=0 tx=0).stop 2024-05-24T11:37:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f7288811700 1 -- 172.21.15.39:0/455481886 reap_dead start 2024-05-24T11:37:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 shutdown_connections 2024-05-24T11:37:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 >> 172.21.15.39:0/455481886 conn(0x7f7284075f80 msgr2=0x7f72840fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 shutdown_connections 2024-05-24T11:37:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:09.166+0000 7f728a274700 1 -- 172.21.15.39:0/455481886 wait complete. 2024-05-24T11:37:09.405 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:09.963 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:09 smithi039 ceph-mon[42919]: pgmap v230: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-05-24T11:37:09.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:09 smithi053 ceph-mon[47448]: pgmap v230: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-05-24T11:37:09.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:09 smithi137 ceph-mon[47907]: pgmap v230: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-05-24T11:37:10.211 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:10.765 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.907+0000 7fb2dcae9700 1 -- 172.21.15.39:0/2720155070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8105f00 msgr2=0x7fb2d81062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.907+0000 7fb2dcae9700 1 --2- 172.21.15.39:0/2720155070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8105f00 0x7fb2d81062e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800b0a0 tx=0x7fb2c800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.907+0000 7fb2dcae9700 1 -- 172.21.15.39:0/2720155070 shutdown_connections 2024-05-24T11:37:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.907+0000 7fb2dcae9700 1 --2- 172.21.15.39:0/2720155070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d8100890 0x7fb2d810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.907+0000 7fb2dcae9700 1 --2- 172.21.15.39:0/2720155070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8105f00 0x7fb2d81062e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.907+0000 7fb2dcae9700 1 --2- 172.21.15.39:0/2720155070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d80ffed0 0x7fb2d8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:10.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.907+0000 7fb2dcae9700 1 -- 172.21.15.39:0/2720155070 >> 172.21.15.39:0/2720155070 conn(0x7fb2d8075f80 msgr2=0x7fb2d8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.907+0000 7fb2dcae9700 1 -- 172.21.15.39:0/2720155070 shutdown_connections 2024-05-24T11:37:10.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.907+0000 7fb2dcae9700 1 -- 172.21.15.39:0/2720155070 wait complete. 2024-05-24T11:37:10.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.908+0000 7fb2dcae9700 1 Processor -- start 2024-05-24T11:37:10.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.908+0000 7fb2dcae9700 1 -- start start 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.908+0000 7fb2dcae9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ffed0 0x7fb2d81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.908+0000 7fb2dcae9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8100890 0x7fb2d8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.908+0000 7fb2dcae9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8105f00 0x7fb2d819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.908+0000 7fb2dcae9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d819c4c0 con 0x7fb2d8100890 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.908+0000 7fb2dcae9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d819c630 con 0x7fb2d80ffed0 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.908+0000 7fb2dcae9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d819c7f0 con 0x7fb2d8105f00 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8100890 0x7fb2d8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8100890 0x7fb2d8197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43522/0 (socket says 172.21.15.39:43522) 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d5d9b700 1 -- 172.21.15.39:0/1238223016 learned_addr learned my addr 172.21.15.39:0/1238223016 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d659c700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ffed0 0x7fb2d81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d6d9d700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8105f00 0x7fb2d819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:10.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d5d9b700 1 -- 172.21.15.39:0/1238223016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8105f00 msgr2=0x7fb2d819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d5d9b700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8105f00 0x7fb2d819beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d5d9b700 1 -- 172.21.15.39:0/1238223016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ffed0 msgr2=0x7fb2d81975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d5d9b700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ffed0 0x7fb2d81975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d5d9b700 1 -- 172.21.15.39:0/1238223016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8009d00 con 0x7fb2d8100890 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2d5d9b700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8100890 0x7fb2d8197b00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00b530 tx=0x7fb2cc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2c77fe700 1 -- 172.21.15.39:0/1238223016 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc011830 con 0x7fb2d8100890 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2c77fe700 1 -- 172.21.15.39:0/1238223016 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb2cc011e20 con 0x7fb2d8100890 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d819cad0 con 0x7fb2d8100890 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.909+0000 7fb2c77fe700 1 -- 172.21.15.39:0/1238223016 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc010dd0 con 0x7fb2d8100890 2024-05-24T11:37:10.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.910+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d81a4330 con 0x7fb2d8100890 2024-05-24T11:37:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.911+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2d8110020 con 0x7fb2d8100890 2024-05-24T11:37:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.911+0000 7fb2c77fe700 1 -- 172.21.15.39:0/1238223016 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb2cc011990 con 0x7fb2d8100890 2024-05-24T11:37:10.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.912+0000 7fb2c77fe700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb2c00a11b0 0x7fb2c00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.912+0000 7fb2d659c700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb2c00a11b0 0x7fb2c00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.912+0000 7fb2c77fe700 1 -- 172.21.15.39:0/1238223016 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb2cc0cc2f0 con 0x7fb2d8100890 2024-05-24T11:37:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.912+0000 7fb2c77fe700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] conn(0x7fb2c00aa990 0x7fb2c00acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.912+0000 7fb2c77fe700 1 -- 172.21.15.39:0/1238223016 --> [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2c00ad460 con 0x7fb2c00aa990 2024-05-24T11:37:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.912+0000 7fb2c77fe700 1 -- 172.21.15.39:0/1238223016 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2cc0cea00 con 0x7fb2d8100890 2024-05-24T11:37:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.912+0000 7fb2d6d9d700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] conn(0x7fb2c00aa990 0x7fb2c00acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.913+0000 7fb2d659c700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb2c00a11b0 0x7fb2c00a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006130 tx=0x7fb2c8006080 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.913+0000 7fb2d6d9d700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] conn(0x7fb2c00aa990 0x7fb2c00acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:10.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.914+0000 7fb2c77fe700 1 -- 172.21.15.39:0/1238223016 <== osd.9 v2:172.21.15.39:6826/1424013773 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb2c00ad460 con 0x7fb2c00aa990 2024-05-24T11:37:10.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.929+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 --> [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2d8104f80 con 0x7fb2c00aa990 2024-05-24T11:37:10.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2c77fe700 1 -- 172.21.15.39:0/1238223016 <== osd.9 v2:172.21.15.39:6826/1424013773 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb2d8104f80 con 0x7fb2c00aa990 2024-05-24T11:37:10.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] conn(0x7fb2c00aa990 msgr2=0x7fb2c00acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:10.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] conn(0x7fb2c00aa990 0x7fb2c00acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:10.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb2c00a11b0 msgr2=0x7fb2c00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:10.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb2c00a11b0 0x7fb2c00a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006130 tx=0x7fb2c8006080 comp rx=0 tx=0).stop 2024-05-24T11:37:10.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8100890 msgr2=0x7fb2d8197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:10.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 --2- 172.21.15.39:0/1238223016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8100890 0x7fb2d8197b00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00b530 tx=0x7fb2cc00b8f0 comp rx=0 tx=0).stop 2024-05-24T11:37:10.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2d6d9d700 1 -- 172.21.15.39:0/1238223016 reap_dead start 2024-05-24T11:37:10.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 shutdown_connections 2024-05-24T11:37:10.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 >> 172.21.15.39:0/1238223016 conn(0x7fb2d8075f80 msgr2=0x7fb2d8068cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:10.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 shutdown_connections 2024-05-24T11:37:10.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:10.930+0000 7fb2dcae9700 1 -- 172.21.15.39:0/1238223016 wait complete. 2024-05-24T11:37:11.824 INFO:teuthology.orchestra.run.smithi039.stdout:261993005078 2024-05-24T11:37:11.824 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-24T11:37:11.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:11 smithi039 ceph-mon[42919]: pgmap v231: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-05-24T11:37:11.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:11 smithi053 ceph-mon[47448]: pgmap v231: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-05-24T11:37:12.028 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:11 smithi137 ceph-mon[47907]: pgmap v231: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-05-24T11:37:12.423 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:12.597 INFO:teuthology.orchestra.run.smithi039.stdout:279172874258 2024-05-24T11:37:12.598 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-24T11:37:12.727 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.064+0000 7f1abee9a700 1 -- 172.21.15.39:0/1183897949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab810da80 msgr2=0x7f1ab810fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.064+0000 7f1abee9a700 1 --2- 172.21.15.39:0/1183897949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab810da80 0x7f1ab810fe70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009f90 tx=0x7f1ab000b330 comp rx=0 tx=0).stop 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.064+0000 7f1abee9a700 1 -- 172.21.15.39:0/1183897949 shutdown_connections 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.064+0000 7f1abee9a700 1 --2- 172.21.15.39:0/1183897949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ab8068950 0x7f1ab8068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.064+0000 7f1abee9a700 1 --2- 172.21.15.39:0/1183897949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab810da80 0x7f1ab810fe70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.064+0000 7f1abee9a700 1 --2- 172.21.15.39:0/1183897949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab8069270 0x7f1ab810d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.064+0000 7f1abee9a700 1 -- 172.21.15.39:0/1183897949 >> 172.21.15.39:0/1183897949 conn(0x7f1ab8075f80 msgr2=0x7f1ab8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.064+0000 7f1abee9a700 1 -- 172.21.15.39:0/1183897949 shutdown_connections 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.064+0000 7f1abee9a700 1 -- 172.21.15.39:0/1183897949 wait complete. 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abee9a700 1 Processor -- start 2024-05-24T11:37:13.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abee9a700 1 -- start start 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abee9a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab8068950 0x7f1ab8106b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abee9a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab8069270 0x7f1ab81070e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abee9a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ab810da80 0x7f1ab81020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abee9a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab81028b0 con 0x7f1ab8068950 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abee9a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8102a20 con 0x7f1ab8069270 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abee9a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8102be0 con 0x7f1ab810da80 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abcc36700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab8068950 0x7f1ab8106b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abcc36700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab8068950 0x7f1ab8106b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43540/0 (socket says 172.21.15.39:43540) 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abcc36700 1 -- 172.21.15.39:0/1956243286 learned_addr learned my addr 172.21.15.39:0/1956243286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1ab7fff700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab8069270 0x7f1ab81070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.065+0000 7f1abd437700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ab810da80 0x7f1ab81020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1abcc36700 1 -- 172.21.15.39:0/1956243286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ab810da80 msgr2=0x7f1ab81020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1abcc36700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ab810da80 0x7f1ab81020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1abcc36700 1 -- 172.21.15.39:0/1956243286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab8069270 msgr2=0x7f1ab81070e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1abcc36700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab8069270 0x7f1ab81070e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1abcc36700 1 -- 172.21.15.39:0/1956243286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab0009c40 con 0x7f1ab8068950 2024-05-24T11:37:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1abcc36700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab8068950 0x7f1ab8106b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e9a0 tx=0x7f1aac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1ab5ffb700 1 -- 172.21.15.39:0/1956243286 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac009790 con 0x7f1ab8068950 2024-05-24T11:37:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1ab5ffb700 1 -- 172.21.15.39:0/1956243286 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1aac00ce50 con 0x7f1ab8068950 2024-05-24T11:37:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab8102ec0 con 0x7f1ab8068950 2024-05-24T11:37:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1ab5ffb700 1 -- 172.21.15.39:0/1956243286 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac018420 con 0x7f1ab8068950 2024-05-24T11:37:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.066+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab810cbf0 con 0x7f1ab8068950 2024-05-24T11:37:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.067+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ab8113290 con 0x7f1ab8068950 2024-05-24T11:37:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.068+0000 7f1ab5ffb700 1 -- 172.21.15.39:0/1956243286 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1aac010430 con 0x7f1ab8068950 2024-05-24T11:37:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.069+0000 7f1ab5ffb700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1aa80a11b0 0x7f1aa80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.070+0000 7f1ab7fff700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1aa80a11b0 0x7f1aa80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.070+0000 7f1ab5ffb700 1 -- 172.21.15.39:0/1956243286 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1aac014070 con 0x7f1ab8068950 2024-05-24T11:37:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.070+0000 7f1ab5ffb700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] conn(0x7f1aa80aa990 0x7f1aa80acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.070+0000 7f1ab5ffb700 1 -- 172.21.15.39:0/1956243286 --> [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa80ad460 con 0x7f1aa80aa990 2024-05-24T11:37:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.070+0000 7f1ab5ffb700 1 -- 172.21.15.39:0/1956243286 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aac0cd520 con 0x7f1ab8068950 2024-05-24T11:37:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.070+0000 7f1ab7fff700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1aa80a11b0 0x7f1aa80a3670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8103420 tx=0x7f1aa4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.070+0000 7f1abd437700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] conn(0x7f1aa80aa990 0x7f1aa80acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.071+0000 7f1abd437700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] conn(0x7f1aa80aa990 0x7f1aa80acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.072+0000 7f1ab5ffb700 1 -- 172.21.15.39:0/1956243286 <== osd.8 v2:172.21.15.137:6816/753285325 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1aa80ad460 con 0x7f1aa80aa990 2024-05-24T11:37:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.088+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 --> [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1ab81004b0 con 0x7f1aa80aa990 2024-05-24T11:37:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1ab5ffb700 1 -- 172.21.15.39:0/1956243286 <== osd.8 v2:172.21.15.137:6816/753285325 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1ab81004b0 con 0x7f1aa80aa990 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 >> [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] conn(0x7f1aa80aa990 msgr2=0x7f1aa80acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] conn(0x7f1aa80aa990 0x7f1aa80acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1aa80a11b0 msgr2=0x7f1aa80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f1aa80a11b0 0x7f1aa80a3670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8103420 tx=0x7f1aa4009310 comp rx=0 tx=0).stop 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab8068950 msgr2=0x7f1ab8106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 --2- 172.21.15.39:0/1956243286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab8068950 0x7f1ab8106b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e9a0 tx=0x7f1aac00ecb0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abd437700 1 -- 172.21.15.39:0/1956243286 reap_dead start 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 shutdown_connections 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 >> 172.21.15.39:0/1956243286 conn(0x7f1ab8075f80 msgr2=0x7f1ab810f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 shutdown_connections 2024-05-24T11:37:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.089+0000 7f1abee9a700 1 -- 172.21.15.39:0/1956243286 wait complete. 2024-05-24T11:37:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 -- 172.21.15.39:0/911094073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a4100ff0 msgr2=0x7ff5a4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 --2- 172.21.15.39:0/911094073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a4100ff0 0x7ff5a4101450 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff598009990 tx=0x7ff598009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 -- 172.21.15.39:0/911094073 shutdown_connections 2024-05-24T11:37:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 --2- 172.21.15.39:0/911094073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5a4101990 0x7ff5a410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 --2- 172.21.15.39:0/911094073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a4107030 0x7ff5a4107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 --2- 172.21.15.39:0/911094073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a4100ff0 0x7ff5a4101450 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 -- 172.21.15.39:0/911094073 >> 172.21.15.39:0/911094073 conn(0x7ff5a4078190 msgr2=0x7ff5a40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:13.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 -- 172.21.15.39:0/911094073 shutdown_connections 2024-05-24T11:37:13.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 -- 172.21.15.39:0/911094073 wait complete. 2024-05-24T11:37:13.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 Processor -- start 2024-05-24T11:37:13.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.284+0000 7ff5aaa35700 1 -- start start 2024-05-24T11:37:13.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5aaa35700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a4100ff0 0x7ff5a4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5aaa35700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a4101990 0x7ff5a4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5aaa35700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5a4107030 0x7ff5a419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5aaa35700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a419c2c0 con 0x7ff5a4101990 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5aaa35700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a419c430 con 0x7ff5a4100ff0 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5aaa35700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a419c5f0 con 0x7ff5a4107030 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a4101990 0x7ff5a4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a4101990 0x7ff5a4197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43562/0 (socket says 172.21.15.39:43562) 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a37fe700 1 -- 172.21.15.39:0/1624100943 learned_addr learned my addr 172.21.15.39:0/1624100943 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a8fd2700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5a4107030 0x7ff5a419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a3fff700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a4100ff0 0x7ff5a4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a37fe700 1 -- 172.21.15.39:0/1624100943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5a4107030 msgr2=0x7ff5a419bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a37fe700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5a4107030 0x7ff5a419bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a37fe700 1 -- 172.21.15.39:0/1624100943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a4100ff0 msgr2=0x7ff5a4197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a37fe700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5a4100ff0 0x7ff5a4197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.285+0000 7ff5a37fe700 1 -- 172.21.15.39:0/1624100943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980095f0 con 0x7ff5a4101990 2024-05-24T11:37:13.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.286+0000 7ff5a37fe700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a4101990 0x7ff5a4197950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff598006010 tx=0x7ff598005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.286+0000 7ff5a17fa700 1 -- 172.21.15.39:0/1624100943 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59801d070 con 0x7ff5a4101990 2024-05-24T11:37:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.286+0000 7ff5a17fa700 1 -- 172.21.15.39:0/1624100943 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff59800fbc0 con 0x7ff5a4101990 2024-05-24T11:37:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.286+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a419c870 con 0x7ff5a4101990 2024-05-24T11:37:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.286+0000 7ff5a17fa700 1 -- 172.21.15.39:0/1624100943 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598022a60 con 0x7ff5a4101990 2024-05-24T11:37:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.286+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a41a4130 con 0x7ff5a4101990 2024-05-24T11:37:13.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.287+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5a4111060 con 0x7ff5a4101990 2024-05-24T11:37:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.288+0000 7ff5a17fa700 1 -- 172.21.15.39:0/1624100943 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff59800f730 con 0x7ff5a4101990 2024-05-24T11:37:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.288+0000 7ff5a17fa700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff58c0a11b0 0x7ff58c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.289+0000 7ff5a17fa700 1 -- 172.21.15.39:0/1624100943 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff5980ce5e0 con 0x7ff5a4101990 2024-05-24T11:37:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.289+0000 7ff5a3fff700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff58c0a11b0 0x7ff58c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.289+0000 7ff5a17fa700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] conn(0x7ff58c0aa990 0x7ff58c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.289+0000 7ff5a17fa700 1 -- 172.21.15.39:0/1624100943 --> [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff58c0ad460 con 0x7ff58c0aa990 2024-05-24T11:37:13.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.289+0000 7ff5a17fa700 1 -- 172.21.15.39:0/1624100943 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5980cea80 con 0x7ff5a4101990 2024-05-24T11:37:13.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.289+0000 7ff5a8fd2700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] conn(0x7ff58c0aa990 0x7ff58c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:13.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.289+0000 7ff5a3fff700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff58c0a11b0 0x7ff58c0a3670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff594005950 tx=0x7ff59400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:13.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.289+0000 7ff5a8fd2700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] conn(0x7ff58c0aa990 0x7ff58c0acdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:13.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.290+0000 7ff5a17fa700 1 -- 172.21.15.39:0/1624100943 <== osd.6 v2:172.21.15.39:6818/2437170915 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff58c0ad460 con 0x7ff58c0aa990 2024-05-24T11:37:13.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.306+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 --> [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff5a4061620 con 0x7ff58c0aa990 2024-05-24T11:37:13.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5a17fa700 1 -- 172.21.15.39:0/1624100943 <== osd.6 v2:172.21.15.39:6818/2437170915 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff5a4061620 con 0x7ff58c0aa990 2024-05-24T11:37:13.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] conn(0x7ff58c0aa990 msgr2=0x7ff58c0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] conn(0x7ff58c0aa990 0x7ff58c0acdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:13.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff58c0a11b0 msgr2=0x7ff58c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff58c0a11b0 0x7ff58c0a3670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff594005950 tx=0x7ff59400b410 comp rx=0 tx=0).stop 2024-05-24T11:37:13.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a4101990 msgr2=0x7ff5a4197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:13.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 --2- 172.21.15.39:0/1624100943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a4101990 0x7ff5a4197950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff598006010 tx=0x7ff598005720 comp rx=0 tx=0).stop 2024-05-24T11:37:13.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5a8fd2700 1 -- 172.21.15.39:0/1624100943 reap_dead start 2024-05-24T11:37:13.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 shutdown_connections 2024-05-24T11:37:13.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 >> 172.21.15.39:0/1624100943 conn(0x7ff5a4078190 msgr2=0x7ff5a41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:13.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 shutdown_connections 2024-05-24T11:37:13.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:13.307+0000 7ff5aaa35700 1 -- 172.21.15.39:0/1624100943 wait complete. 2024-05-24T11:37:13.906 INFO:teuthology.orchestra.run.smithi039.stdout:184683593761 2024-05-24T11:37:13.907 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-24T11:37:13.974 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:13 smithi039 ceph-mon[42919]: pgmap v232: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-24T11:37:13.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:13 smithi053 ceph-mon[47448]: pgmap v232: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-24T11:37:13.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:13 smithi137 ceph-mon[47907]: pgmap v232: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-24T11:37:14.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.214+0000 7f47a131e700 1 -- 172.21.15.39:0/2962745041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479c107060 msgr2=0x7f479c107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.214+0000 7f47a131e700 1 --2- 172.21.15.39:0/2962745041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479c107060 0x7f479c107440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f478c009990 tx=0x7f478c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.214+0000 7f47a131e700 1 -- 172.21.15.39:0/2962745041 shutdown_connections 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.214+0000 7f47a131e700 1 --2- 172.21.15.39:0/2962745041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c1019e0 0x7f479c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.214+0000 7f47a131e700 1 --2- 172.21.15.39:0/2962745041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479c107060 0x7f479c107440 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.214+0000 7f47a131e700 1 --2- 172.21.15.39:0/2962745041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479c101020 0x7f479c1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.214+0000 7f47a131e700 1 -- 172.21.15.39:0/2962745041 >> 172.21.15.39:0/2962745041 conn(0x7f479c078100 msgr2=0x7f479c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.214+0000 7f47a131e700 1 -- 172.21.15.39:0/2962745041 shutdown_connections 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.214+0000 7f47a131e700 1 -- 172.21.15.39:0/2962745041 wait complete. 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f47a131e700 1 Processor -- start 2024-05-24T11:37:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f47a131e700 1 -- start start 2024-05-24T11:37:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f47a131e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c101020 0x7f479c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f47a131e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479c1019e0 0x7f479c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f47a131e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479c107060 0x7f479c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f47a131e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c19e430 con 0x7f479c1019e0 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f47a131e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c19e5a0 con 0x7f479c101020 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f47a131e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c19e760 con 0x7f479c107060 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f479a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479c1019e0 0x7f479c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f479a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479c1019e0 0x7f479c199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43572/0 (socket says 172.21.15.39:43572) 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f479a7fc700 1 -- 172.21.15.39:0/4289179131 learned_addr learned my addr 172.21.15.39:0/4289179131 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.215+0000 7f479affd700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c101020 0x7f479c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f479b7fe700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479c107060 0x7f479c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f479a7fc700 1 -- 172.21.15.39:0/4289179131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479c107060 msgr2=0x7f479c19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f479a7fc700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479c107060 0x7f479c19de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f479a7fc700 1 -- 172.21.15.39:0/4289179131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c101020 msgr2=0x7f479c199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f479a7fc700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c101020 0x7f479c199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f479a7fc700 1 -- 172.21.15.39:0/4289179131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c0095f0 con 0x7f479c1019e0 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f479b7fe700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479c107060 0x7f479c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f479a7fc700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479c1019e0 0x7f479c199a70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f479000b530 tx=0x7f479000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f4783fff700 1 -- 172.21.15.39:0/4289179131 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790011830 con 0x7f479c1019e0 2024-05-24T11:37:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c19ea40 con 0x7f479c1019e0 2024-05-24T11:37:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f4783fff700 1 -- 172.21.15.39:0/4289179131 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4790011e20 con 0x7f479c1019e0 2024-05-24T11:37:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f4783fff700 1 -- 172.21.15.39:0/4289179131 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479000f2e0 con 0x7f479c1019e0 2024-05-24T11:37:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.216+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479c1a62a0 con 0x7f479c1019e0 2024-05-24T11:37:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.217+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f479c1026a0 con 0x7f479c1019e0 2024-05-24T11:37:14.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.218+0000 7f4783fff700 1 -- 172.21.15.39:0/4289179131 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4790011990 con 0x7f479c1019e0 2024-05-24T11:37:14.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.219+0000 7f4783fff700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47840a11b0 0x7f47840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.219+0000 7f479affd700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47840a11b0 0x7f47840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.219+0000 7f4783fff700 1 -- 172.21.15.39:0/4289179131 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f47900cc8b0 con 0x7f479c1019e0 2024-05-24T11:37:14.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.219+0000 7f4783fff700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] conn(0x7f47840aa990 0x7f47840acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.219+0000 7f4783fff700 1 -- 172.21.15.39:0/4289179131 --> [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47840ad460 con 0x7f47840aa990 2024-05-24T11:37:14.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.219+0000 7f4783fff700 1 -- 172.21.15.39:0/4289179131 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47900ccd50 con 0x7f479c1019e0 2024-05-24T11:37:14.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.219+0000 7f479affd700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47840a11b0 0x7f47840a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f478c00b5c0 tx=0x7f478c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:14.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.219+0000 7f479b7fe700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] conn(0x7f47840aa990 0x7f47840acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.220+0000 7f479b7fe700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] conn(0x7f47840aa990 0x7f47840acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:14.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.221+0000 7f4783fff700 1 -- 172.21.15.39:0/4289179131 <== osd.7 v2:172.21.15.53:6816/1522585706 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f47840ad460 con 0x7f47840aa990 2024-05-24T11:37:14.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.241+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 --> [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f479c10ddc0 con 0x7f47840aa990 2024-05-24T11:37:14.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f4783fff700 1 -- 172.21.15.39:0/4289179131 <== osd.7 v2:172.21.15.53:6816/1522585706 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f479c10ddc0 con 0x7f47840aa990 2024-05-24T11:37:14.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 >> [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] conn(0x7f47840aa990 msgr2=0x7f47840acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f47a131e700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] conn(0x7f47840aa990 0x7f47840acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47840a11b0 msgr2=0x7f47840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f47a131e700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47840a11b0 0x7f47840a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f478c00b5c0 tx=0x7f478c005fb0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479c1019e0 msgr2=0x7f479c199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f47a131e700 1 --2- 172.21.15.39:0/4289179131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479c1019e0 0x7f479c199a70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f479000b530 tx=0x7f479000b840 comp rx=0 tx=0).stop 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f479b7fe700 1 -- 172.21.15.39:0/4289179131 reap_dead start 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 shutdown_connections 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.242+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 >> 172.21.15.39:0/4289179131 conn(0x7f479c078100 msgr2=0x7f479c10c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.243+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 shutdown_connections 2024-05-24T11:37:14.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.243+0000 7f47a131e700 1 -- 172.21.15.39:0/4289179131 wait complete. 2024-05-24T11:37:14.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.738+0000 7f31f707b700 1 -- 172.21.15.39:0/2913389068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0101990 msgr2=0x7f31f010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.738+0000 7f31f707b700 1 --2- 172.21.15.39:0/2913389068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0101990 0x7f31f010de70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f31e800b0a0 tx=0x7f31e800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.738+0000 7f31f707b700 1 -- 172.21.15.39:0/2913389068 shutdown_connections 2024-05-24T11:37:14.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.738+0000 7f31f707b700 1 --2- 172.21.15.39:0/2913389068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0101990 0x7f31f010de70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.738+0000 7f31f707b700 1 --2- 172.21.15.39:0/2913389068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107030 0x7f31f0107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.738+0000 7f31f707b700 1 --2- 172.21.15.39:0/2913389068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31f0100ff0 0x7f31f0101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.738+0000 7f31f707b700 1 -- 172.21.15.39:0/2913389068 >> 172.21.15.39:0/2913389068 conn(0x7f31f0078190 msgr2=0x7f31f00785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.738+0000 7f31f707b700 1 -- 172.21.15.39:0/2913389068 shutdown_connections 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.739+0000 7f31f707b700 1 -- 172.21.15.39:0/2913389068 wait complete. 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f707b700 1 Processor -- start 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f707b700 1 -- start start 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f707b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0100ff0 0x7f31f0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f707b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31f0101990 0x7f31f0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f707b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107030 0x7f31f019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f707b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f019c330 con 0x7f31f0100ff0 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f707b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f019c4a0 con 0x7f31f0107030 2024-05-24T11:37:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f707b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f019c660 con 0x7f31f0101990 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f4e17700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0100ff0 0x7f31f0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f4e17700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0100ff0 0x7f31f0197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43580/0 (socket says 172.21.15.39:43580) 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.740+0000 7f31f4e17700 1 -- 172.21.15.39:0/3971930669 learned_addr learned my addr 172.21.15.39:0/3971930669 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31f5618700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107030 0x7f31f019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31effff700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31f0101990 0x7f31f0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31f4e17700 1 -- 172.21.15.39:0/3971930669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31f0101990 msgr2=0x7f31f0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31f4e17700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31f0101990 0x7f31f0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31f4e17700 1 -- 172.21.15.39:0/3971930669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107030 msgr2=0x7f31f019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31f4e17700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107030 0x7f31f019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31f4e17700 1 -- 172.21.15.39:0/3971930669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8009d00 con 0x7f31f0100ff0 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31effff700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31f0101990 0x7f31f0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31f4e17700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0100ff0 0x7f31f0197430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9a0 tx=0x7f31e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:14.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31edffb700 1 -- 172.21.15.39:0/3971930669 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4009790 con 0x7f31f0100ff0 2024-05-24T11:37:14.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31edffb700 1 -- 172.21.15.39:0/3971930669 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31e400cbd0 con 0x7f31f0100ff0 2024-05-24T11:37:14.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f019c940 con 0x7f31f0100ff0 2024-05-24T11:37:14.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.741+0000 7f31edffb700 1 -- 172.21.15.39:0/3971930669 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4018680 con 0x7f31f0100ff0 2024-05-24T11:37:14.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.742+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f01a41a0 con 0x7f31f0100ff0 2024-05-24T11:37:14.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.743+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31f01026e0 con 0x7f31f0100ff0 2024-05-24T11:37:14.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.744+0000 7f31edffb700 1 -- 172.21.15.39:0/3971930669 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f31e400cd40 con 0x7f31f0100ff0 2024-05-24T11:37:14.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.745+0000 7f31edffb700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f31e00a1230 0x7f31e00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.745+0000 7f31effff700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f31e00a1230 0x7f31e00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.745+0000 7f31edffb700 1 -- 172.21.15.39:0/3971930669 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f31e4014070 con 0x7f31f0100ff0 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.746+0000 7f31edffb700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] conn(0x7f31e00aaa10 0x7f31e00ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.746+0000 7f31edffb700 1 -- 172.21.15.39:0/3971930669 --> [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31e00ad4e0 con 0x7f31e00aaa10 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.746+0000 7f31edffb700 1 -- 172.21.15.39:0/3971930669 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e40cd370 con 0x7f31f0100ff0 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.746+0000 7f31effff700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f31e00a1230 0x7f31e00a36f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f31f0198990 tx=0x7f31dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.746+0000 7f31f5618700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] conn(0x7f31e00aaa10 0x7f31e00ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.746+0000 7f31f5618700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] conn(0x7f31e00aaa10 0x7f31e00ace30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.747+0000 7f31edffb700 1 -- 172.21.15.39:0/3971930669 <== osd.11 v2:172.21.15.137:6824/2768517457 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f31e00ad4e0 con 0x7f31e00aaa10 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.766+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 --> [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31f0110f80 con 0x7f31e00aaa10 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.767+0000 7f31edffb700 1 -- 172.21.15.39:0/3971930669 <== osd.11 v2:172.21.15.137:6824/2768517457 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f31f0110f80 con 0x7f31e00aaa10 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.767+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 >> [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] conn(0x7f31e00aaa10 msgr2=0x7f31e00ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.767+0000 7f31f707b700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] conn(0x7f31e00aaa10 0x7f31e00ace30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.767+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f31e00a1230 msgr2=0x7f31e00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.767+0000 7f31f707b700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f31e00a1230 0x7f31e00a36f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f31f0198990 tx=0x7f31dc009260 comp rx=0 tx=0).stop 2024-05-24T11:37:14.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.767+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0100ff0 msgr2=0x7f31f0197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:14.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.767+0000 7f31f707b700 1 --2- 172.21.15.39:0/3971930669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0100ff0 0x7f31f0197430 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9a0 tx=0x7f31e400ecb0 comp rx=0 tx=0).stop 2024-05-24T11:37:14.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.768+0000 7f31f5618700 1 -- 172.21.15.39:0/3971930669 reap_dead start 2024-05-24T11:37:14.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.768+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 shutdown_connections 2024-05-24T11:37:14.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.768+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 >> 172.21.15.39:0/3971930669 conn(0x7f31f0078190 msgr2=0x7f31f00ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:14.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.768+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 shutdown_connections 2024-05-24T11:37:14.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:14.768+0000 7f31f707b700 1 -- 172.21.15.39:0/3971930669 wait complete. 2024-05-24T11:37:15.157 INFO:teuthology.orchestra.run.smithi039.stdout:231928234009 2024-05-24T11:37:15.157 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-24T11:37:15.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.490+0000 7f9d7ad14700 1 -- 172.21.15.39:0/1290931824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d74068950 msgr2=0x7f9d74068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:15.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.490+0000 7f9d7ad14700 1 --2- 172.21.15.39:0/1290931824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d74068950 0x7f9d74068db0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009990 tx=0x7f9d60009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:15.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.490+0000 7f9d7ad14700 1 -- 172.21.15.39:0/1290931824 shutdown_connections 2024-05-24T11:37:15.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.490+0000 7f9d7ad14700 1 --2- 172.21.15.39:0/1290931824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d740692f0 0x7f9d7410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:15.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.490+0000 7f9d7ad14700 1 --2- 172.21.15.39:0/1290931824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d74107020 0x7f9d74107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:15.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.490+0000 7f9d7ad14700 1 --2- 172.21.15.39:0/1290931824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d74068950 0x7f9d74068db0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:15.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.490+0000 7f9d7ad14700 1 -- 172.21.15.39:0/1290931824 >> 172.21.15.39:0/1290931824 conn(0x7f9d74075f80 msgr2=0x7f9d74076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:15.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.490+0000 7f9d7ad14700 1 -- 172.21.15.39:0/1290931824 shutdown_connections 2024-05-24T11:37:15.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.490+0000 7f9d7ad14700 1 -- 172.21.15.39:0/1290931824 wait complete. 2024-05-24T11:37:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d7ad14700 1 Processor -- start 2024-05-24T11:37:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d7ad14700 1 -- start start 2024-05-24T11:37:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d7ad14700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d74068950 0x7f9d74197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d7ad14700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d740692f0 0x7f9d741978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:15.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d7ad14700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d74107020 0x7f9d7419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d7ad14700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7419c280 con 0x7f9d74107020 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d7ad14700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7419c3f0 con 0x7f9d74068950 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d7ad14700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7419c5b0 con 0x7f9d740692f0 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d792b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d74107020 0x7f9d7419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d792b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d74107020 0x7f9d7419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43594/0 (socket says 172.21.15.39:43594) 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d78ab0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d74068950 0x7f9d74197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d792b1700 1 -- 172.21.15.39:0/2646258802 learned_addr learned my addr 172.21.15.39:0/2646258802 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d73fff700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d740692f0 0x7f9d741978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.491+0000 7f9d792b1700 1 -- 172.21.15.39:0/2646258802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d740692f0 msgr2=0x7f9d741978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d792b1700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d740692f0 0x7f9d741978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d792b1700 1 -- 172.21.15.39:0/2646258802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d74068950 msgr2=0x7f9d74197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:15.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d792b1700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d74068950 0x7f9d74197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:15.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d792b1700 1 -- 172.21.15.39:0/2646258802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d600095f0 con 0x7f9d74107020 2024-05-24T11:37:15.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d78ab0700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d74068950 0x7f9d74197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:37:15.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d792b1700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d74107020 0x7f9d7419bc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c007ee0 tx=0x7f9d6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:15.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d71ffb700 1 -- 172.21.15.39:0/2646258802 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c017070 con 0x7f9d74107020 2024-05-24T11:37:15.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d71ffb700 1 -- 172.21.15.39:0/2646258802 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d6c00f040 con 0x7f9d74107020 2024-05-24T11:37:15.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d71ffb700 1 -- 172.21.15.39:0/2646258802 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c005570 con 0x7f9d74107020 2024-05-24T11:37:15.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.492+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7419c890 con 0x7f9d74107020 2024-05-24T11:37:15.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.493+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d741a40f0 con 0x7f9d74107020 2024-05-24T11:37:15.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.493+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d741026d0 con 0x7f9d74107020 2024-05-24T11:37:15.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.494+0000 7f9d71ffb700 1 -- 172.21.15.39:0/2646258802 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9d6c00db20 con 0x7f9d74107020 2024-05-24T11:37:15.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.495+0000 7f9d71ffb700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9d640a1230 0x7f9d640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:15.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.495+0000 7f9d78ab0700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9d640a1230 0x7f9d640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:15.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.495+0000 7f9d71ffb700 1 -- 172.21.15.39:0/2646258802 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9d6c0cc1f0 con 0x7f9d74107020 2024-05-24T11:37:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.495+0000 7f9d71ffb700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] conn(0x7f9d640aaa10 0x7f9d640ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.495+0000 7f9d71ffb700 1 -- 172.21.15.39:0/2646258802 --> [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d640ad4e0 con 0x7f9d640aaa10 2024-05-24T11:37:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.495+0000 7f9d71ffb700 1 -- 172.21.15.39:0/2646258802 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d6c0cc560 con 0x7f9d74107020 2024-05-24T11:37:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.496+0000 7f9d78ab0700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9d640a1230 0x7f9d640a36f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d68005950 tx=0x7f9d6800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.496+0000 7f9d73fff700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] conn(0x7f9d640aaa10 0x7f9d640ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:15.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.496+0000 7f9d73fff700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] conn(0x7f9d640aaa10 0x7f9d640ace30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:15.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.497+0000 7f9d71ffb700 1 -- 172.21.15.39:0/2646258802 <== osd.5 v2:172.21.15.137:6808/173299621 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9d640ad4e0 con 0x7f9d640aaa10 2024-05-24T11:37:15.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.513+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 --> [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d74110fd0 con 0x7f9d640aaa10 2024-05-24T11:37:15.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.514+0000 7f9d71ffb700 1 -- 172.21.15.39:0/2646258802 <== osd.5 v2:172.21.15.137:6808/173299621 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9d74110fd0 con 0x7f9d640aaa10 2024-05-24T11:37:15.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.514+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 >> [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] conn(0x7f9d640aaa10 msgr2=0x7f9d640ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.514+0000 7f9d7ad14700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] conn(0x7f9d640aaa10 0x7f9d640ace30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.515+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9d640a1230 msgr2=0x7f9d640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.515+0000 7f9d7ad14700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9d640a1230 0x7f9d640a36f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9d68005950 tx=0x7f9d6800b410 comp rx=0 tx=0).stop 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.515+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d74107020 msgr2=0x7f9d7419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.515+0000 7f9d7ad14700 1 --2- 172.21.15.39:0/2646258802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d74107020 0x7f9d7419bc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c007ee0 tx=0x7f9d6c00d3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.515+0000 7f9d792b1700 1 -- 172.21.15.39:0/2646258802 reap_dead start 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.515+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 shutdown_connections 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.515+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 >> 172.21.15.39:0/2646258802 conn(0x7f9d74075f80 msgr2=0x7f9d740fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.515+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 shutdown_connections 2024-05-24T11:37:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:15.515+0000 7f9d7ad14700 1 -- 172.21.15.39:0/2646258802 wait complete. 2024-05-24T11:37:15.810 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:15 smithi039 ceph-mon[42919]: pgmap v233: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-24T11:37:15.987 INFO:teuthology.orchestra.run.smithi039.stdout:201863462941 2024-05-24T11:37:15.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-24T11:37:15.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:15 smithi053 ceph-mon[47448]: pgmap v233: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-24T11:37:15.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:15 smithi137 ceph-mon[47907]: pgmap v233: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-24T11:37:16.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.221+0000 7f498bb09700 1 -- 172.21.15.39:0/3243963149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49841000a0 msgr2=0x7f4984100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:16.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.221+0000 7f498bb09700 1 --2- 172.21.15.39:0/3243963149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49841000a0 0x7f4984100480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4980009990 tx=0x7f4980009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:16.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.221+0000 7f498bb09700 1 -- 172.21.15.39:0/3243963149 shutdown_connections 2024-05-24T11:37:16.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.221+0000 7f498bb09700 1 --2- 172.21.15.39:0/3243963149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984105180 0x7f498410cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:16.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.221+0000 7f498bb09700 1 --2- 172.21.15.39:0/3243963149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49841000a0 0x7f4984100480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:16.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.221+0000 7f498bb09700 1 --2- 172.21.15.39:0/3243963149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4984100a50 0x7f4984104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:16.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.221+0000 7f498bb09700 1 -- 172.21.15.39:0/3243963149 >> 172.21.15.39:0/3243963149 conn(0x7f4984076ea0 msgr2=0x7f49840772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:16.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.221+0000 7f498bb09700 1 -- 172.21.15.39:0/3243963149 shutdown_connections 2024-05-24T11:37:16.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.221+0000 7f498bb09700 1 -- 172.21.15.39:0/3243963149 wait complete. 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f498bb09700 1 Processor -- start 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f498bb09700 1 -- start start 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f498bb09700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49841000a0 0x7f4984197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f498bb09700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100a50 0x7f4984197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f498bb09700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4984105180 0x7f498419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f498bb09700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498419c160 con 0x7f49841000a0 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f498bb09700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498419c2a0 con 0x7f4984100a50 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f498bb09700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498419c460 con 0x7f4984105180 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f49898a5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49841000a0 0x7f4984197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f49898a5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49841000a0 0x7f4984197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43602/0 (socket says 172.21.15.39:43602) 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.222+0000 7f49898a5700 1 -- 172.21.15.39:0/3887987086 learned_addr learned my addr 172.21.15.39:0/3887987086 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f498a0a6700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4984105180 0x7f498419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f49890a4700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100a50 0x7f4984197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:16.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f49898a5700 1 -- 172.21.15.39:0/3887987086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4984105180 msgr2=0x7f498419bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f49898a5700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4984105180 0x7f498419bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f49898a5700 1 -- 172.21.15.39:0/3887987086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100a50 msgr2=0x7f4984197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f49898a5700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100a50 0x7f4984197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f49898a5700 1 -- 172.21.15.39:0/3887987086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49800095f0 con 0x7f49841000a0 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f49890a4700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4984100a50 0x7f4984197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f49898a5700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49841000a0 0x7f4984197670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4980004760 tx=0x7f4980009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f4976ffd700 1 -- 172.21.15.39:0/3887987086 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498001d070 con 0x7f49841000a0 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498419c6e0 con 0x7f49841000a0 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f4976ffd700 1 -- 172.21.15.39:0/3887987086 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4980005030 con 0x7f49841000a0 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f4976ffd700 1 -- 172.21.15.39:0/3887987086 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980004b10 con 0x7f49841000a0 2024-05-24T11:37:16.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.223+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49841a3e00 con 0x7f49841000a0 2024-05-24T11:37:16.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.224+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f498410fec0 con 0x7f49841000a0 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.225+0000 7f4976ffd700 1 -- 172.21.15.39:0/3887987086 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4980004c70 con 0x7f49841000a0 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.226+0000 7f4976ffd700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f49700a11b0 0x7f49700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.226+0000 7f49890a4700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f49700a11b0 0x7f49700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.226+0000 7f4976ffd700 1 -- 172.21.15.39:0/3887987086 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f49800ce2a0 con 0x7f49841000a0 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.226+0000 7f4976ffd700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] conn(0x7f49700aa990 0x7f49700acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.226+0000 7f4976ffd700 1 -- 172.21.15.39:0/3887987086 --> [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49700ad460 con 0x7f49700aa990 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.226+0000 7f4976ffd700 1 -- 172.21.15.39:0/3887987086 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49800ce610 con 0x7f49841000a0 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.226+0000 7f49890a4700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f49700a11b0 0x7f49700a3670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49841987c0 tx=0x7f4978009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.227+0000 7f498a0a6700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] conn(0x7f49700aa990 0x7f49700acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:16.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.227+0000 7f498a0a6700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] conn(0x7f49700aa990 0x7f49700acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:16.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.228+0000 7f4976ffd700 1 -- 172.21.15.39:0/3887987086 <== osd.4 v2:172.21.15.53:6808/2421884743 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f49700ad460 con 0x7f49700aa990 2024-05-24T11:37:16.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.244+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 --> [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4984101580 con 0x7f49700aa990 2024-05-24T11:37:16.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.244+0000 7f4976ffd700 1 -- 172.21.15.39:0/3887987086 <== osd.4 v2:172.21.15.53:6808/2421884743 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4984101580 con 0x7f49700aa990 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 >> [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] conn(0x7f49700aa990 msgr2=0x7f49700acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] conn(0x7f49700aa990 0x7f49700acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f49700a11b0 msgr2=0x7f49700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f49700a11b0 0x7f49700a3670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49841987c0 tx=0x7f4978009260 comp rx=0 tx=0).stop 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49841000a0 msgr2=0x7f4984197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 --2- 172.21.15.39:0/3887987086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49841000a0 0x7f4984197670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4980004760 tx=0x7f4980009570 comp rx=0 tx=0).stop 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498a0a6700 1 -- 172.21.15.39:0/3887987086 reap_dead start 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 shutdown_connections 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 >> 172.21.15.39:0/3887987086 conn(0x7f4984076ea0 msgr2=0x7f4984103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 shutdown_connections 2024-05-24T11:37:16.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:16.245+0000 7f498bb09700 1 -- 172.21.15.39:0/3887987086 wait complete. 2024-05-24T11:37:16.923 INFO:teuthology.orchestra.run.smithi039.stdout:309237645326 2024-05-24T11:37:16.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-24T11:37:16.967 INFO:teuthology.orchestra.run.smithi039.stdout:154618822693 2024-05-24T11:37:16.967 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-24T11:37:17.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.522+0000 7f9985582700 1 -- 172.21.15.39:0/1272647114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9980107020 msgr2=0x7f9980107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.522+0000 7f9985582700 1 --2- 172.21.15.39:0/1272647114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9980107020 0x7f9980107400 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9970009990 tx=0x7f9970009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.523+0000 7f9985582700 1 -- 172.21.15.39:0/1272647114 shutdown_connections 2024-05-24T11:37:17.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.523+0000 7f9985582700 1 --2- 172.21.15.39:0/1272647114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99800692f0 0x7f998010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.523+0000 7f9985582700 1 --2- 172.21.15.39:0/1272647114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9980107020 0x7f9980107400 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.523+0000 7f9985582700 1 --2- 172.21.15.39:0/1272647114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9980068950 0x7f9980068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.523+0000 7f9985582700 1 -- 172.21.15.39:0/1272647114 >> 172.21.15.39:0/1272647114 conn(0x7f9980075f80 msgr2=0x7f9980076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.523+0000 7f9985582700 1 -- 172.21.15.39:0/1272647114 shutdown_connections 2024-05-24T11:37:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.523+0000 7f9985582700 1 -- 172.21.15.39:0/1272647114 wait complete. 2024-05-24T11:37:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.523+0000 7f9985582700 1 Processor -- start 2024-05-24T11:37:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f9985582700 1 -- start start 2024-05-24T11:37:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f9985582700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9980068950 0x7f99801972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f9985582700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99800692f0 0x7f9980197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f9985582700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9980107020 0x7f998019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f9985582700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998019c1c0 con 0x7f9980107020 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f9985582700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998019c330 con 0x7f9980068950 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f9985582700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998019c4f0 con 0x7f99800692f0 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9980107020 0x7f998019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9980107020 0x7f998019bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45518/0 (socket says 172.21.15.39:45518) 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997f7fe700 1 -- 172.21.15.39:0/650452697 learned_addr learned my addr 172.21.15.39:0/650452697 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997effd700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9980068950 0x7f99801972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997e7fc700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99800692f0 0x7f9980197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997f7fe700 1 -- 172.21.15.39:0/650452697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99800692f0 msgr2=0x7f9980197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997f7fe700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99800692f0 0x7f9980197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997f7fe700 1 -- 172.21.15.39:0/650452697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9980068950 msgr2=0x7f99801972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997f7fe700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9980068950 0x7f99801972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997f7fe700 1 -- 172.21.15.39:0/650452697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99700095f0 con 0x7f9980107020 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.524+0000 7f997e7fc700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99800692f0 0x7f9980197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.525+0000 7f997f7fe700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9980107020 0x7f998019bbb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9978007ee0 tx=0x7f997800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.525+0000 7f9967fff700 1 -- 172.21.15.39:0/650452697 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978017070 con 0x7f9980107020 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.525+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998019c7d0 con 0x7f9980107020 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.525+0000 7f9967fff700 1 -- 172.21.15.39:0/650452697 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f997800f040 con 0x7f9980107020 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.525+0000 7f9967fff700 1 -- 172.21.15.39:0/650452697 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978005710 con 0x7f9980107020 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.525+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99801000a0 con 0x7f9980107020 2024-05-24T11:37:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.526+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9980111010 con 0x7f9980107020 2024-05-24T11:37:17.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.528+0000 7f9967fff700 1 -- 172.21.15.39:0/650452697 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9978005870 con 0x7f9980107020 2024-05-24T11:37:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.529+0000 7f9967fff700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99680a11b0 0x7f99680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.529+0000 7f997effd700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99680a11b0 0x7f99680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.529+0000 7f9967fff700 1 -- 172.21.15.39:0/650452697 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f99780cc780 con 0x7f9980107020 2024-05-24T11:37:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.529+0000 7f9967fff700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] conn(0x7f99680aa990 0x7f99680acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.529+0000 7f9967fff700 1 -- 172.21.15.39:0/650452697 --> [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99680ad460 con 0x7f99680aa990 2024-05-24T11:37:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.529+0000 7f9967fff700 1 -- 172.21.15.39:0/650452697 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99780ccba0 con 0x7f9980107020 2024-05-24T11:37:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.529+0000 7f997e7fc700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] conn(0x7f99680aa990 0x7f99680acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.530+0000 7f997effd700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99680a11b0 0x7f99680a3670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f997000b5c0 tx=0x7f9970005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.530+0000 7f997e7fc700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] conn(0x7f99680aa990 0x7f99680acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.530+0000 7f9967fff700 1 -- 172.21.15.39:0/650452697 <== osd.0 v2:172.21.15.39:6802/618291952 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f99680ad460 con 0x7f99680aa990 2024-05-24T11:37:17.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.548+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 --> [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99801026d0 con 0x7f99680aa990 2024-05-24T11:37:17.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9967fff700 1 -- 172.21.15.39:0/650452697 <== osd.0 v2:172.21.15.39:6802/618291952 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f99801026d0 con 0x7f99680aa990 2024-05-24T11:37:17.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] conn(0x7f99680aa990 msgr2=0x7f99680acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] conn(0x7f99680aa990 0x7f99680acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99680a11b0 msgr2=0x7f99680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f99680a11b0 0x7f99680a3670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f997000b5c0 tx=0x7f9970005fb0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9980107020 msgr2=0x7f998019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 --2- 172.21.15.39:0/650452697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9980107020 0x7f998019bbb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9978007ee0 tx=0x7f997800d3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f997f7fe700 1 -- 172.21.15.39:0/650452697 reap_dead start 2024-05-24T11:37:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 shutdown_connections 2024-05-24T11:37:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 >> 172.21.15.39:0/650452697 conn(0x7f9980075f80 msgr2=0x7f99800fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 shutdown_connections 2024-05-24T11:37:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.549+0000 7f9985582700 1 -- 172.21.15.39:0/650452697 wait complete. 2024-05-24T11:37:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.696+0000 7faf150b0700 1 -- 172.21.15.39:0/3770950128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf10106340 msgr2=0x7faf1010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.696+0000 7faf150b0700 1 --2- 172.21.15.39:0/3770950128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf10106340 0x7faf1010dec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b0a0 tx=0x7faf0800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.697+0000 7faf150b0700 1 -- 172.21.15.39:0/3770950128 shutdown_connections 2024-05-24T11:37:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.697+0000 7faf150b0700 1 --2- 172.21.15.39:0/3770950128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf10068bb0 0x7faf10068f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.697+0000 7faf150b0700 1 --2- 172.21.15.39:0/3770950128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf10106340 0x7faf1010dec0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.697+0000 7faf150b0700 1 --2- 172.21.15.39:0/3770950128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10069560 0x7faf10105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.697+0000 7faf150b0700 1 -- 172.21.15.39:0/3770950128 >> 172.21.15.39:0/3770950128 conn(0x7faf10075f80 msgr2=0x7faf10076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:17.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.697+0000 7faf150b0700 1 -- 172.21.15.39:0/3770950128 shutdown_connections 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.697+0000 7faf150b0700 1 -- 172.21.15.39:0/3770950128 wait complete. 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.697+0000 7faf150b0700 1 Processor -- start 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf150b0700 1 -- start start 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf150b0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10068bb0 0x7faf100fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf150b0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf10069560 0x7faf101004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf150b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf10106340 0x7faf10100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf150b0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf101a57e0 con 0x7faf10106340 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf150b0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf101a5950 con 0x7faf10068bb0 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf150b0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf101a5b10 con 0x7faf10069560 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0f59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf10106340 0x7faf10100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0f59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf10106340 0x7faf10100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45522/0 (socket says 172.21.15.39:45522) 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0f59e700 1 -- 172.21.15.39:0/2538381127 learned_addr learned my addr 172.21.15.39:0/2538381127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0ed9d700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10068bb0 0x7faf100fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0e59c700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf10069560 0x7faf101004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0f59e700 1 -- 172.21.15.39:0/2538381127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf10069560 msgr2=0x7faf101004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0f59e700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf10069560 0x7faf101004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0f59e700 1 -- 172.21.15.39:0/2538381127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10068bb0 msgr2=0x7faf100fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0f59e700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10068bb0 0x7faf100fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.698+0000 7faf0f59e700 1 -- 172.21.15.39:0/2538381127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf08009d00 con 0x7faf10106340 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.699+0000 7faf0f59e700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf10106340 0x7faf10100a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b070 tx=0x7faf0800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.699+0000 7faef7fff700 1 -- 172.21.15.39:0/2538381127 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800e040 con 0x7faf10106340 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.699+0000 7faef7fff700 1 -- 172.21.15.39:0/2538381127 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf08009460 con 0x7faf10106340 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.699+0000 7faef7fff700 1 -- 172.21.15.39:0/2538381127 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0801b450 con 0x7faf10106340 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.699+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf101a5d90 con 0x7faf10106340 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.699+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf101a61b0 con 0x7faf10106340 2024-05-24T11:37:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.700+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf101026f0 con 0x7faf10106340 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.701+0000 7faef7fff700 1 -- 172.21.15.39:0/2538381127 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faf0801d480 con 0x7faf10106340 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.701+0000 7faef7fff700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faef80a1230 0x7faef80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.702+0000 7faf0ed9d700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faef80a1230 0x7faef80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.702+0000 7faef7fff700 1 -- 172.21.15.39:0/2538381127 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf080ce730 con 0x7faf10106340 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.702+0000 7faef7fff700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] conn(0x7faef80aaa10 0x7faef80ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.702+0000 7faef7fff700 1 -- 172.21.15.39:0/2538381127 --> [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faef80ad4e0 con 0x7faef80aaa10 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.702+0000 7faef7fff700 1 -- 172.21.15.39:0/2538381127 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf080cebd0 con 0x7faf10106340 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.702+0000 7faf0ed9d700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faef80a1230 0x7faef80a36f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faf00009c60 tx=0x7faf00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.702+0000 7faf0e59c700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] conn(0x7faef80aaa10 0x7faef80ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.703+0000 7faf0e59c700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] conn(0x7faef80aaa10 0x7faef80ace30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:17.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.704+0000 7faef7fff700 1 -- 172.21.15.39:0/2538381127 <== osd.2 v2:172.21.15.137:6800/3821366387 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7faef80ad4e0 con 0x7faef80aaa10 2024-05-24T11:37:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.721+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 --> [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf10111090 con 0x7faef80aaa10 2024-05-24T11:37:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.722+0000 7faef7fff700 1 -- 172.21.15.39:0/2538381127 <== osd.2 v2:172.21.15.137:6800/3821366387 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faf0400b8c0 con 0x7faef80aaa10 2024-05-24T11:37:17.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.722+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 >> [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] conn(0x7faef80aaa10 msgr2=0x7faef80ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.722+0000 7faf150b0700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] conn(0x7faef80aaa10 0x7faef80ace30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:17.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.722+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faef80a1230 msgr2=0x7faef80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.722+0000 7faf150b0700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faef80a1230 0x7faef80a36f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faf00009c60 tx=0x7faf00009260 comp rx=0 tx=0).stop 2024-05-24T11:37:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.723+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf10106340 msgr2=0x7faf10100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.723+0000 7faf150b0700 1 --2- 172.21.15.39:0/2538381127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf10106340 0x7faf10100a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b070 tx=0x7faf0800cd70 comp rx=0 tx=0).stop 2024-05-24T11:37:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.723+0000 7faf0f59e700 1 -- 172.21.15.39:0/2538381127 reap_dead start 2024-05-24T11:37:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.723+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 shutdown_connections 2024-05-24T11:37:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.723+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 >> 172.21.15.39:0/2538381127 conn(0x7faf10075f80 msgr2=0x7faf100fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.723+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 shutdown_connections 2024-05-24T11:37:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:17.723+0000 7faf150b0700 1 -- 172.21.15.39:0/2538381127 wait complete. 2024-05-24T11:37:17.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:17 smithi039 ceph-mon[42919]: pgmap v234: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-24T11:37:17.989 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:17.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:17 smithi053 ceph-mon[47448]: pgmap v234: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-24T11:37:17.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:17 smithi137 ceph-mon[47907]: pgmap v234: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-24T11:37:18.013 INFO:teuthology.orchestra.run.smithi039.stdout:34359738423 2024-05-24T11:37:18.014 INFO:teuthology.orchestra.run.smithi039.stdout:137438953513 2024-05-24T11:37:18.015 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-24T11:37:18.015 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-24T11:37:18.139 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:19.045 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:19.222 INFO:teuthology.orchestra.run.smithi039.stdout:73014444080 2024-05-24T11:37:19.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-24T11:37:19.447 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:19.549 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:19.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:19 smithi053 ceph-mon[47448]: pgmap v235: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-24T11:37:19.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:19 smithi137 ceph-mon[47907]: pgmap v235: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-24T11:37:20.022 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:19 smithi039 ceph-mon[42919]: pgmap v235: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-24T11:37:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.059+0000 7f63391c7700 1 -- 172.21.15.39:0/3323830773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 msgr2=0x7f6334103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.059+0000 7f63391c7700 1 --2- 172.21.15.39:0/3323830773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 0x7f6334103e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b0a0 tx=0x7f632c00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.059+0000 7f63391c7700 1 -- 172.21.15.39:0/3323830773 shutdown_connections 2024-05-24T11:37:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.059+0000 7f63391c7700 1 --2- 172.21.15.39:0/3323830773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 0x7f6334103e40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.059+0000 7f63391c7700 1 --2- 172.21.15.39:0/3323830773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6334105fa0 0x7f63340691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.059+0000 7f63391c7700 1 --2- 172.21.15.39:0/3323830773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6334069720 0x7f6334069ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.059+0000 7f63391c7700 1 -- 172.21.15.39:0/3323830773 >> 172.21.15.39:0/3323830773 conn(0x7f6334076e70 msgr2=0x7f6334077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.059+0000 7f63391c7700 1 -- 172.21.15.39:0/3323830773 shutdown_connections 2024-05-24T11:37:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.059+0000 7f63391c7700 1 -- 172.21.15.39:0/3323830773 wait complete. 2024-05-24T11:37:20.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.060+0000 7f63391c7700 1 Processor -- start 2024-05-24T11:37:20.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.060+0000 7f63391c7700 1 -- start start 2024-05-24T11:37:20.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.060+0000 7f63391c7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6334069720 0x7f633419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f63391c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 0x7f633419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f63391c7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6334105fa0 0x7f63341a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f63391c7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341a25f0 con 0x7f6334103990 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f63391c7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341a2760 con 0x7f6334105fa0 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f63391c7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341a2920 con 0x7f6334069720 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 0x7f633419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 0x7f633419dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45544/0 (socket says 172.21.15.39:45544) 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633259c700 1 -- 172.21.15.39:0/1761691466 learned_addr learned my addr 172.21.15.39:0/1761691466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f6332d9d700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6334069720 0x7f633419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633359e700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6334105fa0 0x7f63341a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:20.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633259c700 1 -- 172.21.15.39:0/1761691466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6334069720 msgr2=0x7f633419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633259c700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6334069720 0x7f633419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633259c700 1 -- 172.21.15.39:0/1761691466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6334105fa0 msgr2=0x7f63341a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633259c700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6334105fa0 0x7f63341a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633259c700 1 -- 172.21.15.39:0/1761691466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009d00 con 0x7f6334103990 2024-05-24T11:37:20.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633359e700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6334105fa0 0x7f63341a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:20.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f633259c700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 0x7f633419dc80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f632800c9b0 tx=0x7f632800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:20.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.061+0000 7f631bfff700 1 -- 172.21.15.39:0/1761691466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328007b00 con 0x7f6334103990 2024-05-24T11:37:20.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.062+0000 7f631bfff700 1 -- 172.21.15.39:0/1761691466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6328007c60 con 0x7f6334103990 2024-05-24T11:37:20.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.062+0000 7f631bfff700 1 -- 172.21.15.39:0/1761691466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328018660 con 0x7f6334103990 2024-05-24T11:37:20.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.062+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63341a2c00 con 0x7f6334103990 2024-05-24T11:37:20.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.062+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633406ab90 con 0x7f6334103990 2024-05-24T11:37:20.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.063+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63341a2df0 con 0x7f6334103990 2024-05-24T11:37:20.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.065+0000 7f631bfff700 1 -- 172.21.15.39:0/1761691466 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f632800f450 con 0x7f6334103990 2024-05-24T11:37:20.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.065+0000 7f631bfff700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f631c0a11b0 0x7f631c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:20.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.066+0000 7f631bfff700 1 -- 172.21.15.39:0/1761691466 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f63280ccec0 con 0x7f6334103990 2024-05-24T11:37:20.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.066+0000 7f6332d9d700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f631c0a11b0 0x7f631c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:20.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.066+0000 7f6332d9d700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f631c0a11b0 0x7f631c0a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6324005fd0 tx=0x7f632400af20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:20.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.069+0000 7f631bfff700 1 -- 172.21.15.39:0/1761691466 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f63280965a0 con 0x7f6334103990 2024-05-24T11:37:20.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.291+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6334100470 con 0x7f6334103990 2024-05-24T11:37:20.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.291+0000 7f631bfff700 1 -- 172.21.15.39:0/1761691466 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6328099bc0 con 0x7f6334103990 2024-05-24T11:37:20.296 INFO:teuthology.orchestra.run.smithi039.stdout:51539607604 2024-05-24T11:37:20.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f631c0a11b0 msgr2=0x7f631c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f631c0a11b0 0x7f631c0a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6324005fd0 tx=0x7f632400af20 comp rx=0 tx=0).stop 2024-05-24T11:37:20.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 msgr2=0x7f633419dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 0x7f633419dc80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f632800c9b0 tx=0x7f632800cd70 comp rx=0 tx=0).stop 2024-05-24T11:37:20.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 shutdown_connections 2024-05-24T11:37:20.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f631c0a11b0 0x7f631c0a3670 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6334069720 0x7f633419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6334105fa0 0x7f63341a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 --2- 172.21.15.39:0/1761691466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6334103990 0x7f633419dc80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 >> 172.21.15.39:0/1761691466 conn(0x7f6334076e70 msgr2=0x7f63340fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:20.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 shutdown_connections 2024-05-24T11:37:20.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.294+0000 7f63391c7700 1 -- 172.21.15.39:0/1761691466 wait complete. 2024-05-24T11:37:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.468+0000 7faf5105e700 1 -- 172.21.15.39:0/636702543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 msgr2=0x7faf4c100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.468+0000 7faf5105e700 1 --2- 172.21.15.39:0/636702543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 0x7faf4c100350 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faf40009990 tx=0x7faf40009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.468+0000 7faf5105e700 1 -- 172.21.15.39:0/636702543 shutdown_connections 2024-05-24T11:37:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.468+0000 7faf5105e700 1 --2- 172.21.15.39:0/636702543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf4c105f00 0x7faf4c1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.468+0000 7faf5105e700 1 --2- 172.21.15.39:0/636702543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf4c100890 0x7faf4c10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.468+0000 7faf5105e700 1 --2- 172.21.15.39:0/636702543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 0x7faf4c100350 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.468+0000 7faf5105e700 1 -- 172.21.15.39:0/636702543 >> 172.21.15.39:0/636702543 conn(0x7faf4c075f80 msgr2=0x7faf4c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:20.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.468+0000 7faf5105e700 1 -- 172.21.15.39:0/636702543 shutdown_connections 2024-05-24T11:37:20.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.468+0000 7faf5105e700 1 -- 172.21.15.39:0/636702543 wait complete. 2024-05-24T11:37:20.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.469+0000 7faf5105e700 1 Processor -- start 2024-05-24T11:37:20.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.469+0000 7faf5105e700 1 -- start start 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.469+0000 7faf5105e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 0x7faf4c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.469+0000 7faf5105e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf4c100890 0x7faf4c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf5105e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf4c105f00 0x7faf4c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf5105e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c19c210 con 0x7faf4c0ffed0 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf5105e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c19c380 con 0x7faf4c100890 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf5105e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c19c540 con 0x7faf4c105f00 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 0x7faf4c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 0x7faf4c197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45562/0 (socket says 172.21.15.39:45562) 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4ad9d700 1 -- 172.21.15.39:0/1218562734 learned_addr learned my addr 172.21.15.39:0/1218562734 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4b59e700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf4c105f00 0x7faf4c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4a59c700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf4c100890 0x7faf4c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4ad9d700 1 -- 172.21.15.39:0/1218562734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf4c105f00 msgr2=0x7faf4c19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4ad9d700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf4c105f00 0x7faf4c19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4ad9d700 1 -- 172.21.15.39:0/1218562734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf4c100890 msgr2=0x7faf4c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4ad9d700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf4c100890 0x7faf4c197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4ad9d700 1 -- 172.21.15.39:0/1218562734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf400095f0 con 0x7faf4c0ffed0 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4b59e700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf4c105f00 0x7faf4c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf4ad9d700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 0x7faf4c197310 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00c920 tx=0x7faf3c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf33fff700 1 -- 172.21.15.39:0/1218562734 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c007b00 con 0x7faf4c0ffed0 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf33fff700 1 -- 172.21.15.39:0/1218562734 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf3c004cc0 con 0x7faf4c0ffed0 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c19c820 con 0x7faf4c0ffed0 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf33fff700 1 -- 172.21.15.39:0/1218562734 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c005590 con 0x7faf4c0ffed0 2024-05-24T11:37:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.470+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c1a4080 con 0x7faf4c0ffed0 2024-05-24T11:37:20.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.471+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c1015b0 con 0x7faf4c0ffed0 2024-05-24T11:37:20.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.472+0000 7faf33fff700 1 -- 172.21.15.39:0/1218562734 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faf3c004750 con 0x7faf4c0ffed0 2024-05-24T11:37:20.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.473+0000 7faf33fff700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf340a1230 0x7faf340a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:20.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.473+0000 7faf4a59c700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf340a1230 0x7faf340a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:20.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.473+0000 7faf33fff700 1 -- 172.21.15.39:0/1218562734 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf3c0cc890 con 0x7faf4c0ffed0 2024-05-24T11:37:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.474+0000 7faf4a59c700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf340a1230 0x7faf340a36f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faf4000b5c0 tx=0x7faf40005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.478+0000 7faf33fff700 1 -- 172.21.15.39:0/1218562734 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf3c095f70 con 0x7faf4c0ffed0 2024-05-24T11:37:20.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.692+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7faf4c04ec60 con 0x7faf4c0ffed0 2024-05-24T11:37:20.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.692+0000 7faf33fff700 1 -- 172.21.15.39:0/1218562734 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7faf3c01d020 con 0x7faf4c0ffed0 2024-05-24T11:37:20.697 INFO:teuthology.orchestra.run.smithi039.stdout:279172874259 2024-05-24T11:37:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf340a1230 msgr2=0x7faf340a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf340a1230 0x7faf340a36f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faf4000b5c0 tx=0x7faf40005fd0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 msgr2=0x7faf4c197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 0x7faf4c197310 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00c920 tx=0x7faf3c00cce0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 shutdown_connections 2024-05-24T11:37:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf340a1230 0x7faf340a36f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf4c105f00 0x7faf4c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf4c0ffed0 0x7faf4c197310 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 --2- 172.21.15.39:0/1218562734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf4c100890 0x7faf4c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.694+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 >> 172.21.15.39:0/1218562734 conn(0x7faf4c075f80 msgr2=0x7faf4c068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.695+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 shutdown_connections 2024-05-24T11:37:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:20.695+0000 7faf5105e700 1 -- 172.21.15.39:0/1218562734 wait complete. 2024-05-24T11:37:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:20 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1761691466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-24T11:37:20.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:20 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1218562734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-24T11:37:20.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:20 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1761691466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-24T11:37:20.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:20 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1218562734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-24T11:37:21.011 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:20 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1761691466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-24T11:37:21.011 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:20 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1218562734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-24T11:37:21.376 INFO:tasks.cephadm.ceph_manager.ceph:need seq 51539607602 got 51539607604 for osd.1 2024-05-24T11:37:21.376 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:21.611 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:21.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 -- 172.21.15.39:0/3065580805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d00ffe90 msgr2=0x7fe1d0100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:21.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/3065580805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d00ffe90 0x7fe1d0100310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009990 tx=0x7fe1c0009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:21.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 -- 172.21.15.39:0/3065580805 shutdown_connections 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/3065580805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d0105f40 0x7fe1d0106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/3065580805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d0100850 0x7fe1d010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/3065580805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d00ffe90 0x7fe1d0100310 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 -- 172.21.15.39:0/3065580805 >> 172.21.15.39:0/3065580805 conn(0x7fe1d0076e70 msgr2=0x7fe1d0077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 -- 172.21.15.39:0/3065580805 shutdown_connections 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 -- 172.21.15.39:0/3065580805 wait complete. 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 Processor -- start 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.921+0000 7fe1d81ec700 1 -- start start 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d81ec700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d00ffe90 0x7fe1d0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d81ec700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d0100850 0x7fe1d0197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d81ec700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d0105f40 0x7fe1d019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d81ec700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d019c200 con 0x7fe1d0105f40 2024-05-24T11:37:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d81ec700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d019c370 con 0x7fe1d0100850 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d81ec700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d019c530 con 0x7fe1d00ffe90 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d6789700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d0105f40 0x7fe1d019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d6789700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d0105f40 0x7fe1d019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45566/0 (socket says 172.21.15.39:45566) 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d6789700 1 -- 172.21.15.39:0/1835052788 learned_addr learned my addr 172.21.15.39:0/1835052788 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d5f88700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d00ffe90 0x7fe1d0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d5787700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d0100850 0x7fe1d0197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d6789700 1 -- 172.21.15.39:0/1835052788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d00ffe90 msgr2=0x7fe1d0197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d6789700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d00ffe90 0x7fe1d0197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d6789700 1 -- 172.21.15.39:0/1835052788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d0100850 msgr2=0x7fe1d0197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d6789700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d0100850 0x7fe1d0197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d6789700 1 -- 172.21.15.39:0/1835052788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c00095f0 con 0x7fe1d0105f40 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d5787700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d0100850 0x7fe1d0197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d6789700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d0105f40 0x7fe1d019bbf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8007ee0 tx=0x7fe1c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:21.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1c6ffd700 1 -- 172.21.15.39:0/1835052788 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8017070 con 0x7fe1d0105f40 2024-05-24T11:37:21.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1c6ffd700 1 -- 172.21.15.39:0/1835052788 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1c800f040 con 0x7fe1d0105f40 2024-05-24T11:37:21.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.922+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d019c810 con 0x7fe1d0105f40 2024-05-24T11:37:21.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.923+0000 7fe1c6ffd700 1 -- 172.21.15.39:0/1835052788 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8005710 con 0x7fe1d0105f40 2024-05-24T11:37:21.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.923+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d01a4070 con 0x7fe1d0105f40 2024-05-24T11:37:21.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.924+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d0101580 con 0x7fe1d0105f40 2024-05-24T11:37:21.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.925+0000 7fe1c6ffd700 1 -- 172.21.15.39:0/1835052788 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe1c8005870 con 0x7fe1d0105f40 2024-05-24T11:37:21.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.925+0000 7fe1c6ffd700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe1bc0a1230 0x7fe1bc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:21.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.926+0000 7fe1d5f88700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe1bc0a1230 0x7fe1bc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:21.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.926+0000 7fe1c6ffd700 1 -- 172.21.15.39:0/1835052788 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe1c80cc060 con 0x7fe1d0105f40 2024-05-24T11:37:21.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.926+0000 7fe1d5f88700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe1bc0a1230 0x7fe1bc0a36f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009a10 tx=0x7fe1cc009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:21.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:21.930+0000 7fe1c6ffd700 1 -- 172.21.15.39:0/1835052788 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe1c8095840 con 0x7fe1d0105f40 2024-05-24T11:37:21.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:21 smithi053 ceph-mon[47448]: pgmap v236: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-24T11:37:21.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:21 smithi137 ceph-mon[47907]: pgmap v236: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-24T11:37:22.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:21 smithi039 ceph-mon[42919]: pgmap v236: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-24T11:37:22.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.145+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fe1d010fcc0 con 0x7fe1d0105f40 2024-05-24T11:37:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.146+0000 7fe1c6ffd700 1 -- 172.21.15.39:0/1835052788 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1c8098e60 con 0x7fe1d0105f40 2024-05-24T11:37:22.151 INFO:teuthology.orchestra.run.smithi039.stdout:261993005079 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe1bc0a1230 msgr2=0x7fe1bc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe1bc0a1230 0x7fe1bc0a36f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009a10 tx=0x7fe1cc009190 comp rx=0 tx=0).stop 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d0105f40 msgr2=0x7fe1d019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d0105f40 0x7fe1d019bbf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8007ee0 tx=0x7fe1c800d3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 shutdown_connections 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe1bc0a1230 0x7fe1bc0a36f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d00ffe90 0x7fe1d0197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1d0105f40 0x7fe1d019bbf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 --2- 172.21.15.39:0/1835052788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d0100850 0x7fe1d0197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.148+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 >> 172.21.15.39:0/1835052788 conn(0x7fe1d0076e70 msgr2=0x7fe1d0104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:22.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.150+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 shutdown_connections 2024-05-24T11:37:22.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.150+0000 7fe1d81ec700 1 -- 172.21.15.39:0/1835052788 wait complete. 2024-05-24T11:37:22.165 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:22.191 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874258 got 279172874259 for osd.10 2024-05-24T11:37:22.191 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:22.419 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.872+0000 7f6239747700 1 -- 172.21.15.39:0/711230053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62341063c0 msgr2=0x7f623410dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.872+0000 7f6239747700 1 --2- 172.21.15.39:0/711230053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62341063c0 0x7f623410dee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b0d0 tx=0x7f622c00b3e0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.872+0000 7f6239747700 1 -- 172.21.15.39:0/711230053 shutdown_connections 2024-05-24T11:37:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.872+0000 7f6239747700 1 --2- 172.21.15.39:0/711230053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62341063c0 0x7f623410dee0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.872+0000 7f6239747700 1 --2- 172.21.15.39:0/711230053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62340ffba0 0x7f62340fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.872+0000 7f6239747700 1 --2- 172.21.15.39:0/711230053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6234100550 0x7f6234105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.872+0000 7f6239747700 1 -- 172.21.15.39:0/711230053 >> 172.21.15.39:0/711230053 conn(0x7f6234075260 msgr2=0x7f6234075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.872+0000 7f6239747700 1 -- 172.21.15.39:0/711230053 shutdown_connections 2024-05-24T11:37:22.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.872+0000 7f6239747700 1 -- 172.21.15.39:0/711230053 wait complete. 2024-05-24T11:37:22.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6239747700 1 Processor -- start 2024-05-24T11:37:22.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6239747700 1 -- start start 2024-05-24T11:37:22.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6239747700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62340ffba0 0x7f62341972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:22.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6239747700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6234100550 0x7f6234197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6239747700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62341063c0 0x7f623419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6239747700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623419c1e0 con 0x7f62340ffba0 2024-05-24T11:37:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6239747700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623419c350 con 0x7f62341063c0 2024-05-24T11:37:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6239747700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623419c510 con 0x7f6234100550 2024-05-24T11:37:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6232ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62340ffba0 0x7f62341972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6232ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62340ffba0 0x7f62341972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45580/0 (socket says 172.21.15.39:45580) 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f6232ffd700 1 -- 172.21.15.39:0/3283473234 learned_addr learned my addr 172.21.15.39:0/3283473234 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f62337fe700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62341063c0 0x7f623419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.873+0000 7f62327fc700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6234100550 0x7f6234197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f6232ffd700 1 -- 172.21.15.39:0/3283473234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6234100550 msgr2=0x7f6234197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f6232ffd700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6234100550 0x7f6234197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f6232ffd700 1 -- 172.21.15.39:0/3283473234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62341063c0 msgr2=0x7f623419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f6232ffd700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62341063c0 0x7f623419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f6232ffd700 1 -- 172.21.15.39:0/3283473234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c009d00 con 0x7f62340ffba0 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f6232ffd700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62340ffba0 0x7f62341972e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f622400d950 tx=0x7f622400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f621bfff700 1 -- 172.21.15.39:0/3283473234 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622400f890 con 0x7f62340ffba0 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623419c7f0 con 0x7f62340ffba0 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62341a4050 con 0x7f62340ffba0 2024-05-24T11:37:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f621bfff700 1 -- 172.21.15.39:0/3283473234 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f622400fe80 con 0x7f62340ffba0 2024-05-24T11:37:22.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.874+0000 7f621bfff700 1 -- 172.21.15.39:0/3283473234 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622400e3b0 con 0x7f62340ffba0 2024-05-24T11:37:22.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.875+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6220005320 con 0x7f62340ffba0 2024-05-24T11:37:22.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.877+0000 7f621bfff700 1 -- 172.21.15.39:0/3283473234 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f622400f9f0 con 0x7f62340ffba0 2024-05-24T11:37:22.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.878+0000 7f621bfff700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f621c0a11b0 0x7f621c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:22.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.878+0000 7f62327fc700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f621c0a11b0 0x7f621c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:22.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.878+0000 7f621bfff700 1 -- 172.21.15.39:0/3283473234 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f62240cc530 con 0x7f62340ffba0 2024-05-24T11:37:22.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.879+0000 7f62327fc700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f621c0a11b0 0x7f621c0a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6234198840 tx=0x7f622800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:22.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:22.881+0000 7f621bfff700 1 -- 172.21.15.39:0/3283473234 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f62240997a0 con 0x7f62340ffba0 2024-05-24T11:37:22.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:22 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1835052788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-24T11:37:22.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:22 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1835052788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-24T11:37:23.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:22 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1835052788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-24T11:37:23.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.093+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f6220005190 con 0x7f62340ffba0 2024-05-24T11:37:23.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.094+0000 7f621bfff700 1 -- 172.21.15.39:0/3283473234 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6224099330 con 0x7f62340ffba0 2024-05-24T11:37:23.099 INFO:teuthology.orchestra.run.smithi039.stdout:231928234011 2024-05-24T11:37:23.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.096+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f621c0a11b0 msgr2=0x7f621c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:23.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.096+0000 7f6239747700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f621c0a11b0 0x7f621c0a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6234198840 tx=0x7f622800b410 comp rx=0 tx=0).stop 2024-05-24T11:37:23.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.096+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62340ffba0 msgr2=0x7f62341972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:23.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.096+0000 7f6239747700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62340ffba0 0x7f62341972e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f622400d950 tx=0x7f622400dc60 comp rx=0 tx=0).stop 2024-05-24T11:37:23.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.097+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 shutdown_connections 2024-05-24T11:37:23.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.097+0000 7f6239747700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f621c0a11b0 0x7f621c0a3670 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.097+0000 7f6239747700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62341063c0 0x7f623419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.097+0000 7f6239747700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62340ffba0 0x7f62341972e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.097+0000 7f6239747700 1 --2- 172.21.15.39:0/3283473234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6234100550 0x7f6234197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.097+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 >> 172.21.15.39:0/3283473234 conn(0x7f6234075260 msgr2=0x7f62340fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:23.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.097+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 shutdown_connections 2024-05-24T11:37:23.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.097+0000 7f6239747700 1 -- 172.21.15.39:0/3283473234 wait complete. 2024-05-24T11:37:23.267 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005078 got 261993005079 for osd.9 2024-05-24T11:37:23.267 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 -- 172.21.15.39:0/72584626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110105f80 msgr2=0x7f1110106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 --2- 172.21.15.39:0/72584626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110105f80 0x7f1110106360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1104009990 tx=0x7f1104009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 -- 172.21.15.39:0/72584626 shutdown_connections 2024-05-24T11:37:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 --2- 172.21.15.39:0/72584626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1110100bb0 0x7f111010cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 --2- 172.21.15.39:0/72584626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110105f80 0x7f1110106360 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 --2- 172.21.15.39:0/72584626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11101001f0 0x7f1110100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 -- 172.21.15.39:0/72584626 >> 172.21.15.39:0/72584626 conn(0x7f1110076c50 msgr2=0x7f1110077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 -- 172.21.15.39:0/72584626 shutdown_connections 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 -- 172.21.15.39:0/72584626 wait complete. 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.127+0000 7f1117ab5700 1 Processor -- start 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1117ab5700 1 -- start start 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1117ab5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11101001f0 0x7f11101972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1117ab5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110100bb0 0x7f1110197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1117ab5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1110105f80 0x7f111019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1117ab5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111019c1c0 con 0x7f1110100bb0 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1117ab5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111019c330 con 0x7f1110105f80 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1117ab5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111019c4f0 con 0x7f11101001f0 2024-05-24T11:37:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115050700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110100bb0 0x7f1110197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115050700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110100bb0 0x7f1110197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45588/0 (socket says 172.21.15.39:45588) 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115050700 1 -- 172.21.15.39:0/1022228181 learned_addr learned my addr 172.21.15.39:0/1022228181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115851700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11101001f0 0x7f11101972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1116052700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1110105f80 0x7f111019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115050700 1 -- 172.21.15.39:0/1022228181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11101001f0 msgr2=0x7f11101972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115050700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11101001f0 0x7f11101972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115050700 1 -- 172.21.15.39:0/1022228181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1110105f80 msgr2=0x7f111019bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115050700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1110105f80 0x7f111019bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115050700 1 -- 172.21.15.39:0/1022228181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11040095f0 con 0x7f1110100bb0 2024-05-24T11:37:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1115050700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110100bb0 0x7f1110197800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f110c00c9b0 tx=0x7f110c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1102ffd700 1 -- 172.21.15.39:0/1022228181 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c007b00 con 0x7f1110100bb0 2024-05-24T11:37:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1102ffd700 1 -- 172.21.15.39:0/1022228181 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f110c007c60 con 0x7f1110100bb0 2024-05-24T11:37:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.128+0000 7f1102ffd700 1 -- 172.21.15.39:0/1022228181 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c018660 con 0x7f1110100bb0 2024-05-24T11:37:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.129+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111019c7d0 con 0x7f1110100bb0 2024-05-24T11:37:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.129+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11101a4030 con 0x7f1110100bb0 2024-05-24T11:37:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.130+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11101015a0 con 0x7f1110100bb0 2024-05-24T11:37:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.131+0000 7f1102ffd700 1 -- 172.21.15.39:0/1022228181 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f110c00f450 con 0x7f1110100bb0 2024-05-24T11:37:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.131+0000 7f1102ffd700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10fc0a1230 0x7f10fc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.131+0000 7f1102ffd700 1 -- 172.21.15.39:0/1022228181 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f110c0cc910 con 0x7f1110100bb0 2024-05-24T11:37:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.132+0000 7f1115851700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10fc0a1230 0x7f10fc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.132+0000 7f1115851700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10fc0a1230 0x7f10fc0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f110400b5c0 tx=0x7f1104005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.136+0000 7f1102ffd700 1 -- 172.21.15.39:0/1022228181 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f110c0960f0 con 0x7f1110100bb0 2024-05-24T11:37:23.307 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:23.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.354+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f111004ec60 con 0x7f1110100bb0 2024-05-24T11:37:23.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.354+0000 7f1102ffd700 1 -- 172.21.15.39:0/1022228181 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f110c099710 con 0x7f1110100bb0 2024-05-24T11:37:23.359 INFO:teuthology.orchestra.run.smithi039.stdout:107374182445 2024-05-24T11:37:23.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.356+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10fc0a1230 msgr2=0x7f10fc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:23.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.356+0000 7f1117ab5700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10fc0a1230 0x7f10fc0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f110400b5c0 tx=0x7f1104005fb0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.356+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110100bb0 msgr2=0x7f1110197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:23.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.356+0000 7f1117ab5700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110100bb0 0x7f1110197800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f110c00c9b0 tx=0x7f110c00cd70 comp rx=0 tx=0).stop 2024-05-24T11:37:23.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.357+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 shutdown_connections 2024-05-24T11:37:23.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.357+0000 7f1117ab5700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10fc0a1230 0x7f10fc0a36f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.357+0000 7f1117ab5700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11101001f0 0x7f11101972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.357+0000 7f1117ab5700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1110105f80 0x7f111019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.357+0000 7f1117ab5700 1 --2- 172.21.15.39:0/1022228181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1110100bb0 0x7f1110197800 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:23.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.357+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 >> 172.21.15.39:0/1022228181 conn(0x7f1110076c50 msgr2=0x7f11100ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:23.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.357+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 shutdown_connections 2024-05-24T11:37:23.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:23.357+0000 7f1117ab5700 1 -- 172.21.15.39:0/1022228181 wait complete. 2024-05-24T11:37:23.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:23 smithi137 ceph-mon[47907]: pgmap v237: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-24T11:37:23.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:23 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3283473234' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-24T11:37:23.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:23 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1022228181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-24T11:37:24.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:23 smithi039 ceph-mon[42919]: pgmap v237: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-24T11:37:24.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:23 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3283473234' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-24T11:37:24.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:23 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1022228181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-24T11:37:24.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:23 smithi053 ceph-mon[47448]: pgmap v237: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-24T11:37:24.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:23 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3283473234' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-24T11:37:24.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:23 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1022228181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-24T11:37:25.255 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234009 got 231928234011 for osd.8 2024-05-24T11:37:25.256 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:25.256 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182442 got 107374182445 for osd.3 2024-05-24T11:37:25.256 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:25.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.568+0000 7fecc8d6b700 1 -- 172.21.15.39:0/2828909611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc4069660 msgr2=0x7fecc4069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.568+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/2828909611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc4069660 0x7fecc4069ae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fecac009990 tx=0x7fecac009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.569+0000 7fecc8d6b700 1 -- 172.21.15.39:0/2828909611 shutdown_connections 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.569+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/2828909611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc4107050 0x7fecc4107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.569+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/2828909611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecc406a020 0x7fecc410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.569+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/2828909611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc4069660 0x7fecc4069ae0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.569+0000 7fecc8d6b700 1 -- 172.21.15.39:0/2828909611 >> 172.21.15.39:0/2828909611 conn(0x7fecc4075f80 msgr2=0x7fecc4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.569+0000 7fecc8d6b700 1 -- 172.21.15.39:0/2828909611 shutdown_connections 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.569+0000 7fecc8d6b700 1 -- 172.21.15.39:0/2828909611 wait complete. 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.569+0000 7fecc8d6b700 1 Processor -- start 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.569+0000 7fecc8d6b700 1 -- start start 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecc8d6b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecc4069660 0x7fecc4100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecc8d6b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc406a020 0x7fecc4100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecc8d6b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc4107050 0x7fecc4100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecc8d6b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc41a55e0 con 0x7fecc406a020 2024-05-24T11:37:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecc8d6b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc41a5750 con 0x7fecc4069660 2024-05-24T11:37:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecc8d6b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc41a5910 con 0x7fecc4107050 2024-05-24T11:37:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc406a020 0x7fecc4100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc406a020 0x7fecc4100540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45594/0 (socket says 172.21.15.39:45594) 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbbfff700 1 -- 172.21.15.39:0/27954333 learned_addr learned my addr 172.21.15.39:0/27954333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecc2d9d700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc4107050 0x7fecc4100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecc259c700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecc4069660 0x7fecc4100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbbfff700 1 -- 172.21.15.39:0/27954333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc4107050 msgr2=0x7fecc4100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbbfff700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc4107050 0x7fecc4100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbbfff700 1 -- 172.21.15.39:0/27954333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecc4069660 msgr2=0x7fecc4100000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbbfff700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecc4069660 0x7fecc4100000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbbfff700 1 -- 172.21.15.39:0/27954333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0095f0 con 0x7fecc406a020 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbbfff700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc406a020 0x7fecc4100540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fecac006010 tx=0x7fecac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecbb7fe700 1 -- 172.21.15.39:0/27954333 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac01d070 con 0x7fecc406a020 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.570+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc41a5b90 con 0x7fecc406a020 2024-05-24T11:37:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.571+0000 7fecbb7fe700 1 -- 172.21.15.39:0/27954333 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecac00fc00 con 0x7fecc406a020 2024-05-24T11:37:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.571+0000 7fecbb7fe700 1 -- 172.21.15.39:0/27954333 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac022c00 con 0x7fecc406a020 2024-05-24T11:37:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.571+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc41a6000 con 0x7fecc406a020 2024-05-24T11:37:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.572+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc41094e0 con 0x7fecc406a020 2024-05-24T11:37:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.573+0000 7fecbb7fe700 1 -- 172.21.15.39:0/27954333 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fecac022470 con 0x7fecc406a020 2024-05-24T11:37:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.573+0000 7fecbb7fe700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fecb00a11b0 0x7fecb00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.573+0000 7fecc259c700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fecb00a11b0 0x7fecb00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.574+0000 7fecbb7fe700 1 -- 172.21.15.39:0/27954333 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fecac0cd920 con 0x7fecc406a020 2024-05-24T11:37:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.574+0000 7fecc259c700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fecb00a11b0 0x7fecb00a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fecb4005fd0 tx=0x7fecb400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.578+0000 7fecbb7fe700 1 -- 172.21.15.39:0/27954333 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fecac097190 con 0x7fecc406a020 2024-05-24T11:37:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.785+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fecc4102690 con 0x7fecc406a020 2024-05-24T11:37:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.786+0000 7fecbb7fe700 1 -- 172.21.15.39:0/27954333 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fecac09a7b0 con 0x7fecc406a020 2024-05-24T11:37:25.791 INFO:teuthology.orchestra.run.smithi039.stdout:309237645328 2024-05-24T11:37:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fecb00a11b0 msgr2=0x7fecb00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fecb00a11b0 0x7fecb00a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fecb4005fd0 tx=0x7fecb400a560 comp rx=0 tx=0).stop 2024-05-24T11:37:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc406a020 msgr2=0x7fecc4100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc406a020 0x7fecc4100540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fecac006010 tx=0x7fecac004730 comp rx=0 tx=0).stop 2024-05-24T11:37:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 shutdown_connections 2024-05-24T11:37:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fecb00a11b0 0x7fecb00a3670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecc4107050 0x7fecc4100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fecc4069660 0x7fecc4100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 --2- 172.21.15.39:0/27954333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecc406a020 0x7fecc4100540 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 >> 172.21.15.39:0/27954333 conn(0x7fecc4075f80 msgr2=0x7fecc40fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 shutdown_connections 2024-05-24T11:37:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:25.788+0000 7fecc8d6b700 1 -- 172.21.15.39:0/27954333 wait complete. 2024-05-24T11:37:25.938 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:25.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:25 smithi137 ceph-mon[47907]: pgmap v238: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-24T11:37:26.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:25 smithi039 ceph-mon[42919]: pgmap v238: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-24T11:37:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:25 smithi053 ceph-mon[47448]: pgmap v238: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-24T11:37:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.861+0000 7fa8aecfd700 1 -- 172.21.15.39:0/1561686397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a8105ed0 msgr2=0x7fa8a80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.861+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/1561686397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a8105ed0 0x7fa8a80695e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009990 tx=0x7fa89c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.861+0000 7fa8aecfd700 1 -- 172.21.15.39:0/1561686397 shutdown_connections 2024-05-24T11:37:26.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.861+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/1561686397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a81039e0 0x7fa8a8103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.861+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/1561686397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a8105ed0 0x7fa8a80695e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.861+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/1561686397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8a8106340 0x7fa8a8069b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.861+0000 7fa8aecfd700 1 -- 172.21.15.39:0/1561686397 >> 172.21.15.39:0/1561686397 conn(0x7fa8a8076c40 msgr2=0x7fa8a8077050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:26.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.862+0000 7fa8aecfd700 1 -- 172.21.15.39:0/1561686397 shutdown_connections 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.862+0000 7fa8aecfd700 1 -- 172.21.15.39:0/1561686397 wait complete. 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.862+0000 7fa8aecfd700 1 Processor -- start 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.862+0000 7fa8aecfd700 1 -- start start 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8aecfd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a81039e0 0x7fa8a81143f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8aecfd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8a8105ed0 0x7fa8a8114930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8aecfd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a8106340 0x7fa8a8111f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8aecfd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a8112760 con 0x7fa8a8106340 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8aecfd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a81128d0 con 0x7fa8a8105ed0 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8aecfd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a8112a90 con 0x7fa8a81039e0 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8ad29a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a8106340 0x7fa8a8111f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8ad29a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a8106340 0x7fa8a8111f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45630/0 (socket says 172.21.15.39:45630) 2024-05-24T11:37:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8a7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8a8105ed0 0x7fa8a8114930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8ad29a700 1 -- 172.21.15.39:0/2947072712 learned_addr learned my addr 172.21.15.39:0/2947072712 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8aca99700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a81039e0 0x7fa8a81143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8a7fff700 1 -- 172.21.15.39:0/2947072712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a81039e0 msgr2=0x7fa8a81143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8a7fff700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a81039e0 0x7fa8a81143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8a7fff700 1 -- 172.21.15.39:0/2947072712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a8106340 msgr2=0x7fa8a8111f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8a7fff700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a8106340 0x7fa8a8111f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8a7fff700 1 -- 172.21.15.39:0/2947072712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c0095f0 con 0x7fa8a8105ed0 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.863+0000 7fa8aca99700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a81039e0 0x7fa8a81143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.864+0000 7fa8a7fff700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8a8105ed0 0x7fa8a8114930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa89400e9f0 tx=0x7fa89400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:26.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.864+0000 7fa8a5ffb700 1 -- 172.21.15.39:0/2947072712 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8940096d0 con 0x7fa8a8105ed0 2024-05-24T11:37:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.864+0000 7fa8a5ffb700 1 -- 172.21.15.39:0/2947072712 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa89400cbd0 con 0x7fa8a8105ed0 2024-05-24T11:37:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.864+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a8112d70 con 0x7fa8a8105ed0 2024-05-24T11:37:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.864+0000 7fa8a5ffb700 1 -- 172.21.15.39:0/2947072712 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894018680 con 0x7fa8a8105ed0 2024-05-24T11:37:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.864+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a810d860 con 0x7fa8a8105ed0 2024-05-24T11:37:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.865+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a81004c0 con 0x7fa8a8105ed0 2024-05-24T11:37:26.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.867+0000 7fa8a5ffb700 1 -- 172.21.15.39:0/2947072712 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa89400cd40 con 0x7fa8a8105ed0 2024-05-24T11:37:26.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.867+0000 7fa8a5ffb700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa8980a11b0 0x7fa8980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.868+0000 7fa8aca99700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa8980a11b0 0x7fa8980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.868+0000 7fa8a5ffb700 1 -- 172.21.15.39:0/2947072712 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa894014070 con 0x7fa8a8105ed0 2024-05-24T11:37:26.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.868+0000 7fa8aca99700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa8980a11b0 0x7fa8980a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa89c006010 tx=0x7fa89c005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.871+0000 7fa8a5ffb700 1 -- 172.21.15.39:0/2947072712 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa894096350 con 0x7fa8a8105ed0 2024-05-24T11:37:26.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.934+0000 7fac5d586700 1 -- 172.21.15.39:0/1707806458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac58104ed0 msgr2=0x7fac581052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.934+0000 7fac5d586700 1 --2- 172.21.15.39:0/1707806458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac58104ed0 0x7fac581052b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0099c0 tx=0x7fac4c009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.934+0000 7fac5d586700 1 -- 172.21.15.39:0/1707806458 shutdown_connections 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.934+0000 7fac5d586700 1 --2- 172.21.15.39:0/1707806458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac58101410 0x7fac5810cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.934+0000 7fac5d586700 1 --2- 172.21.15.39:0/1707806458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac58104ed0 0x7fac581052b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.934+0000 7fac5d586700 1 --2- 172.21.15.39:0/1707806458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac58100a50 0x7fac58100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.934+0000 7fac5d586700 1 -- 172.21.15.39:0/1707806458 >> 172.21.15.39:0/1707806458 conn(0x7fac58075260 msgr2=0x7fac58075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.934+0000 7fac5d586700 1 -- 172.21.15.39:0/1707806458 shutdown_connections 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.934+0000 7fac5d586700 1 -- 172.21.15.39:0/1707806458 wait complete. 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.935+0000 7fac5d586700 1 Processor -- start 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.935+0000 7fac5d586700 1 -- start start 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac5d586700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac58100a50 0x7fac58197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac5d586700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac58101410 0x7fac58197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac5d586700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac58104ed0 0x7fac5819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac5d586700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5819c0e0 con 0x7fac58101410 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac5d586700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5819c220 con 0x7fac58104ed0 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac5d586700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5819c3e0 con 0x7fac58100a50 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac56ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac58100a50 0x7fac58197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac567fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac58101410 0x7fac58197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac56ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac58100a50 0x7fac58197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:55612/0 (socket says 172.21.15.39:55612) 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac56ffd700 1 -- 172.21.15.39:0/880933338 learned_addr learned my addr 172.21.15.39:0/880933338 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac567fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac58101410 0x7fac58197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45652/0 (socket says 172.21.15.39:45652) 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac56ffd700 1 -- 172.21.15.39:0/880933338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac58104ed0 msgr2=0x7fac5819bba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac56ffd700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac58104ed0 0x7fac5819bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac56ffd700 1 -- 172.21.15.39:0/880933338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac58101410 msgr2=0x7fac58197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac56ffd700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac58101410 0x7fac58197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.936+0000 7fac56ffd700 1 -- 172.21.15.39:0/880933338 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0095f0 con 0x7fac58100a50 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.937+0000 7fac56ffd700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac58100a50 0x7fac58197300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fac4c005910 tx=0x7fac4c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.937+0000 7fac3ffff700 1 -- 172.21.15.39:0/880933338 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c01d070 con 0x7fac58100a50 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.937+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5819c660 con 0x7fac58100a50 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.937+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5819de20 con 0x7fac58100a50 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.938+0000 7fac3ffff700 1 -- 172.21.15.39:0/880933338 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac4c00f8f0 con 0x7fac58100a50 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.938+0000 7fac3ffff700 1 -- 172.21.15.39:0/880933338 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c021670 con 0x7fac58100a50 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.939+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac440052f0 con 0x7fac58100a50 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.941+0000 7fac3ffff700 1 -- 172.21.15.39:0/880933338 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fac4c00f460 con 0x7fac58100a50 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.941+0000 7fac3ffff700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fac400a11b0 0x7fac400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.942+0000 7fac567fc700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fac400a11b0 0x7fac400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.942+0000 7fac3ffff700 1 -- 172.21.15.39:0/880933338 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fac4c0cd0f0 con 0x7fac58100a50 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.942+0000 7fac567fc700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fac400a11b0 0x7fac400a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fac58198810 tx=0x7fac48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:26.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.944+0000 7fac3ffff700 1 -- 172.21.15.39:0/880933338 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fac4c096860 con 0x7fac58100a50 2024-05-24T11:37:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.980+0000 7faef31c6700 1 -- 172.21.15.39:0/2176228044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec0ffba0 msgr2=0x7faeec0fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.980+0000 7faef31c6700 1 --2- 172.21.15.39:0/2176228044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec0ffba0 0x7faeec0fff80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faee00099c0 tx=0x7faee0009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.981+0000 7faef31c6700 1 -- 172.21.15.39:0/2176228044 shutdown_connections 2024-05-24T11:37:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.981+0000 7faef31c6700 1 --2- 172.21.15.39:0/2176228044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeec1063c0 0x7faeec10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.981+0000 7faef31c6700 1 --2- 172.21.15.39:0/2176228044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec0ffba0 0x7faeec0fff80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.981+0000 7faef31c6700 1 --2- 172.21.15.39:0/2176228044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec100550 0x7faeec105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.981+0000 7faef31c6700 1 -- 172.21.15.39:0/2176228044 >> 172.21.15.39:0/2176228044 conn(0x7faeec075260 msgr2=0x7faeec075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.981+0000 7faef31c6700 1 -- 172.21.15.39:0/2176228044 shutdown_connections 2024-05-24T11:37:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.981+0000 7faef31c6700 1 -- 172.21.15.39:0/2176228044 wait complete. 2024-05-24T11:37:26.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.981+0000 7faef31c6700 1 Processor -- start 2024-05-24T11:37:26.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef31c6700 1 -- start start 2024-05-24T11:37:26.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef31c6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeec0ffba0 0x7faeec1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef31c6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec100550 0x7faeec197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef31c6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec1063c0 0x7faeec19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef31c6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec19c1f0 con 0x7faeec1063c0 2024-05-24T11:37:26.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef31c6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec19c360 con 0x7faeec100550 2024-05-24T11:37:26.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef31c6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec19c520 con 0x7faeec0ffba0 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef1763700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec1063c0 0x7faeec19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef0f62700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeec0ffba0 0x7faeec1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef0f62700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeec0ffba0 0x7faeec1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:55618/0 (socket says 172.21.15.39:55618) 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef1763700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec1063c0 0x7faeec19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45660/0 (socket says 172.21.15.39:45660) 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef0f62700 1 -- 172.21.15.39:0/2687356712 learned_addr learned my addr 172.21.15.39:0/2687356712 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faeebfff700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec100550 0x7faeec197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef1763700 1 -- 172.21.15.39:0/2687356712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeec0ffba0 msgr2=0x7faeec1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef1763700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeec0ffba0 0x7faeec1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef1763700 1 -- 172.21.15.39:0/2687356712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec100550 msgr2=0x7faeec197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef1763700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec100550 0x7faeec197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.982+0000 7faef1763700 1 -- 172.21.15.39:0/2687356712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee00095f0 con 0x7faeec1063c0 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.983+0000 7faef1763700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec1063c0 0x7faeec19bbe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faee400bb00 tx=0x7faee400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.983+0000 7faee9ffb700 1 -- 172.21.15.39:0/2687356712 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee400c9b0 con 0x7faeec1063c0 2024-05-24T11:37:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.983+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec19c800 con 0x7faeec1063c0 2024-05-24T11:37:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.983+0000 7faee9ffb700 1 -- 172.21.15.39:0/2687356712 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faee4014440 con 0x7faeec1063c0 2024-05-24T11:37:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.983+0000 7faee9ffb700 1 -- 172.21.15.39:0/2687356712 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee40125a0 con 0x7faeec1063c0 2024-05-24T11:37:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.983+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec100c20 con 0x7faeec1063c0 2024-05-24T11:37:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.984+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec1095a0 con 0x7faeec1063c0 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.408+0000 7f36a20a4700 1 -- 172.21.15.39:0/1752877850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 msgr2=0x7f369c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.408+0000 7f36a20a4700 1 --2- 172.21.15.39:0/1752877850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 0x7f369c107400 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f368c009990 tx=0x7f368c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.408+0000 7f36a20a4700 1 -- 172.21.15.39:0/1752877850 shutdown_connections 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.408+0000 7f36a20a4700 1 --2- 172.21.15.39:0/1752877850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f369c0692f0 0x7f369c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.408+0000 7f36a20a4700 1 --2- 172.21.15.39:0/1752877850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 0x7f369c107400 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.408+0000 7f36a20a4700 1 --2- 172.21.15.39:0/1752877850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f369c068950 0x7f369c068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.408+0000 7f36a20a4700 1 -- 172.21.15.39:0/1752877850 >> 172.21.15.39:0/1752877850 conn(0x7f369c075f80 msgr2=0x7f369c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.408+0000 7f36a20a4700 1 -- 172.21.15.39:0/1752877850 shutdown_connections 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.408+0000 7f36a20a4700 1 -- 172.21.15.39:0/1752877850 wait complete. 2024-05-24T11:37:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f36a20a4700 1 Processor -- start 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f36a20a4700 1 -- start start 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f36a20a4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f369c068950 0x7f369c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f36a20a4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f369c0692f0 0x7f369c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f36a20a4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 0x7f369c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f36a20a4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c19c1f0 con 0x7f369c107020 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f36a20a4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c19c360 con 0x7f369c068950 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f36a20a4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c19c520 con 0x7f369c0692f0 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f369bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 0x7f369c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f369bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 0x7f369c19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45612/0 (socket says 172.21.15.39:45612) 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.409+0000 7f369bfff700 1 -- 172.21.15.39:0/84423204 learned_addr learned my addr 172.21.15.39:0/84423204 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f369b7fe700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f369c068950 0x7f369c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f369bfff700 1 -- 172.21.15.39:0/84423204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f369c0692f0 msgr2=0x7f369c197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f369affd700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f369c0692f0 0x7f369c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f369bfff700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f369c0692f0 0x7f369c197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f369bfff700 1 -- 172.21.15.39:0/84423204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f369c068950 msgr2=0x7f369c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f369bfff700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f369c068950 0x7f369c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f369bfff700 1 -- 172.21.15.39:0/84423204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0095f0 con 0x7f369c107020 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f369b7fe700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f369c068950 0x7f369c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f369bfff700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 0x7f369c19bbe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f369400ada0 tx=0x7f369400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f3698ff9700 1 -- 172.21.15.39:0/84423204 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369400f070 con 0x7f369c107020 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f3698ff9700 1 -- 172.21.15.39:0/84423204 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3694007b30 con 0x7f369c107020 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c19c800 con 0x7f369c107020 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f3698ff9700 1 -- 172.21.15.39:0/84423204 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36940053e0 con 0x7f369c107020 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.410+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c1a4060 con 0x7f369c107020 2024-05-24T11:37:26.993 INFO:teuthology.orchestra.run.smithi039.stdout:154618822695 2024-05-24T11:37:26.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.411+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36880052f0 con 0x7f369c107020 2024-05-24T11:37:26.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.413+0000 7f3698ff9700 1 -- 172.21.15.39:0/84423204 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3694004ad0 con 0x7f369c107020 2024-05-24T11:37:26.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.413+0000 7f3698ff9700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f36840a1230 0x7f36840a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.413+0000 7f3698ff9700 1 -- 172.21.15.39:0/84423204 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f36940cc350 con 0x7f369c107020 2024-05-24T11:37:26.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.413+0000 7f369b7fe700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f36840a1230 0x7f36840a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.414+0000 7f369b7fe700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f36840a1230 0x7f36840a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f368c00b5c0 tx=0x7f368c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.417+0000 7f3698ff9700 1 -- 172.21.15.39:0/84423204 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3694095bc0 con 0x7f369c107020 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.626+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3688005160 con 0x7f369c107020 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.626+0000 7f3698ff9700 1 -- 172.21.15.39:0/84423204 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f36940991e0 con 0x7f369c107020 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f36840a1230 msgr2=0x7f36840a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f36840a1230 0x7f36840a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f368c00b5c0 tx=0x7f368c005fd0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 msgr2=0x7f369c19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 0x7f369c19bbe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f369400ada0 tx=0x7f369400c330 comp rx=0 tx=0).stop 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 shutdown_connections 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f36840a1230 0x7f36840a36f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f369c107020 0x7f369c19bbe0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f369c068950 0x7f369c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 --2- 172.21.15.39:0/84423204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f369c0692f0 0x7f369c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 >> 172.21.15.39:0/84423204 conn(0x7f369c075f80 msgr2=0x7f369c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 shutdown_connections 2024-05-24T11:37:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.629+0000 7f36a20a4700 1 -- 172.21.15.39:0/84423204 wait complete. 2024-05-24T11:37:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.986+0000 7faee9ffb700 1 -- 172.21.15.39:0/2687356712 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faee40145b0 con 0x7faeec1063c0 2024-05-24T11:37:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.987+0000 7faee9ffb700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faedc0a1230 0x7faedc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.987+0000 7faee9ffb700 1 -- 172.21.15.39:0/2687356712 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faee40cc780 con 0x7faeec1063c0 2024-05-24T11:37:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.989+0000 7faef0f62700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faedc0a1230 0x7faedc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.991+0000 7faef0f62700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faedc0a1230 0x7faedc0a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faee0009990 tx=0x7faee0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.994+0000 7faee9ffb700 1 -- 172.21.15.39:0/2687356712 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faee4095f70 con 0x7faeec1063c0 2024-05-24T11:37:27.021 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:26 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/27954333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-24T11:37:27.021 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:26 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/84423204' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-24T11:37:27.134 INFO:teuthology.orchestra.run.smithi039.stdout:184683593763 2024-05-24T11:37:27.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.129+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa8a804ec60 con 0x7fa8a8105ed0 2024-05-24T11:37:27.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.130+0000 7fa8a5ffb700 1 -- 172.21.15.39:0/2947072712 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa894099970 con 0x7fa8a8105ed0 2024-05-24T11:37:27.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.132+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa8980a11b0 msgr2=0x7fa8980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.132+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa8980a11b0 0x7fa8980a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa89c006010 tx=0x7fa89c005ab0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.132+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8a8105ed0 msgr2=0x7fa8a8114930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.132+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8a8105ed0 0x7fa8a8114930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa89400e9f0 tx=0x7fa89400ed00 comp rx=0 tx=0).stop 2024-05-24T11:37:27.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.133+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 shutdown_connections 2024-05-24T11:37:27.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.133+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa8980a11b0 0x7fa8980a3670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.133+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a81039e0 0x7fa8a81143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.133+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a8106340 0x7fa8a8111f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.133+0000 7fa8aecfd700 1 --2- 172.21.15.39:0/2947072712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8a8105ed0 0x7fa8a8114930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.133+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 >> 172.21.15.39:0/2947072712 conn(0x7fa8a8076c40 msgr2=0x7fa8a8102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:27.138 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645326 got 309237645328 for osd.11 2024-05-24T11:37:27.139 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:27.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.136+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 shutdown_connections 2024-05-24T11:37:27.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.136+0000 7fa8aecfd700 1 -- 172.21.15.39:0/2947072712 wait complete. 2024-05-24T11:37:27.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.174+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fac44005870 con 0x7fac58100a50 2024-05-24T11:37:27.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.174+0000 7fac3ffff700 1 -- 172.21.15.39:0/880933338 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fac4c099e80 con 0x7fac58100a50 2024-05-24T11:37:27.179 INFO:teuthology.orchestra.run.smithi039.stdout:34359738425 2024-05-24T11:37:27.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fac400a11b0 msgr2=0x7fac400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fac400a11b0 0x7fac400a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fac58198810 tx=0x7fac48009260 comp rx=0 tx=0).stop 2024-05-24T11:37:27.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac58100a50 msgr2=0x7fac58197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac58100a50 0x7fac58197300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fac4c005910 tx=0x7fac4c005270 comp rx=0 tx=0).stop 2024-05-24T11:37:27.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 shutdown_connections 2024-05-24T11:37:27.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fac400a11b0 0x7fac400a3670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac58100a50 0x7fac58197300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac58104ed0 0x7fac5819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 --2- 172.21.15.39:0/880933338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac58101410 0x7fac58197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 >> 172.21.15.39:0/880933338 conn(0x7fac58075260 msgr2=0x7fac58102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:27.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 shutdown_connections 2024-05-24T11:37:27.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.177+0000 7fac5d586700 1 -- 172.21.15.39:0/880933338 wait complete. 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.821+0000 7fefe1728700 1 -- 172.21.15.39:0/748668302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc101990 msgr2=0x7fefdc10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.821+0000 7fefe1728700 1 --2- 172.21.15.39:0/748668302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc101990 0x7fefdc10de70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fefd400b0a0 tx=0x7fefd400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.821+0000 7fefe1728700 1 -- 172.21.15.39:0/748668302 shutdown_connections 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.821+0000 7fefe1728700 1 --2- 172.21.15.39:0/748668302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc101990 0x7fefdc10de70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.821+0000 7fefe1728700 1 --2- 172.21.15.39:0/748668302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefdc107030 0x7fefdc107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.821+0000 7fefe1728700 1 --2- 172.21.15.39:0/748668302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc100ff0 0x7fefdc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.821+0000 7fefe1728700 1 -- 172.21.15.39:0/748668302 >> 172.21.15.39:0/748668302 conn(0x7fefdc078190 msgr2=0x7fefdc0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.821+0000 7fefe1728700 1 -- 172.21.15.39:0/748668302 shutdown_connections 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.821+0000 7fefe1728700 1 -- 172.21.15.39:0/748668302 wait complete. 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.822+0000 7fefe1728700 1 Processor -- start 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.822+0000 7fefe1728700 1 -- start start 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.822+0000 7fefe1728700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefdc100ff0 0x7fefdc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:27.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.822+0000 7fefe1728700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc101990 0x7fefdc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.822+0000 7fefe1728700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc107030 0x7fefdc073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.822+0000 7fefe1728700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc1a5720 con 0x7fefdc107030 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.822+0000 7fefe1728700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc1a5890 con 0x7fefdc100ff0 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.822+0000 7fefe1728700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc1a5a50 con 0x7fefdc101990 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc107030 0x7fefdc073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc107030 0x7fefdc073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45622/0 (socket says 172.21.15.39:45622) 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdb7fe700 1 -- 172.21.15.39:0/2086159961 learned_addr learned my addr 172.21.15.39:0/2086159961 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefda7fc700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc101990 0x7fefdc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdaffd700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefdc100ff0 0x7fefdc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdb7fe700 1 -- 172.21.15.39:0/2086159961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc101990 msgr2=0x7fefdc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdb7fe700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc101990 0x7fefdc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdb7fe700 1 -- 172.21.15.39:0/2086159961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefdc100ff0 msgr2=0x7fefdc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdb7fe700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefdc100ff0 0x7fefdc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdb7fe700 1 -- 172.21.15.39:0/2086159961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd4009d00 con 0x7fefdc107030 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdaffd700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefdc100ff0 0x7fefdc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefdb7fe700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc107030 0x7fefdc073d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefd400b830 tx=0x7fefd4009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefc3fff700 1 -- 172.21.15.39:0/2086159961 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd400e040 con 0x7fefdc107030 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc1a5cd0 con 0x7fefdc107030 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefc3fff700 1 -- 172.21.15.39:0/2086159961 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefd4007930 con 0x7fefdc107030 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefc3fff700 1 -- 172.21.15.39:0/2086159961 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4005280 con 0x7fefdc107030 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.823+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc1a6140 con 0x7fefdc107030 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.824+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc109550 con 0x7fefdc107030 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.826+0000 7fefc3fff700 1 -- 172.21.15.39:0/2086159961 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fefd401b9e0 con 0x7fefdc107030 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.826+0000 7fefc3fff700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fefc40a11b0 0x7fefc40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.827+0000 7fefc3fff700 1 -- 172.21.15.39:0/2086159961 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fefd40ce870 con 0x7fefdc107030 2024-05-24T11:37:27.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.827+0000 7fefdaffd700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fefc40a11b0 0x7fefc40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stdout:201863462943 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.827+0000 7fefdaffd700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fefc40a11b0 0x7fefc40a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefcc005fd0 tx=0x7fefcc005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:26.833+0000 7fefc3fff700 1 -- 172.21.15.39:0/2086159961 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fefd4097fe0 con 0x7fefdc107030 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.065+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fefdc074780 con 0x7fefdc107030 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.065+0000 7fefc3fff700 1 -- 172.21.15.39:0/2086159961 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefd409b600 con 0x7fefdc107030 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fefc40a11b0 msgr2=0x7fefc40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fefc40a11b0 0x7fefc40a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefcc005fd0 tx=0x7fefcc005db0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc107030 msgr2=0x7fefdc073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc107030 0x7fefdc073d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefd400b830 tx=0x7fefd4009380 comp rx=0 tx=0).stop 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 shutdown_connections 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fefc40a11b0 0x7fefc40a3670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefdc107030 0x7fefdc073d20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefdc100ff0 0x7fefdc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 --2- 172.21.15.39:0/2086159961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc101990 0x7fefdc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.067+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 >> 172.21.15.39:0/2086159961 conn(0x7fefdc078190 msgr2=0x7fefdc0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:27.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.068+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 shutdown_connections 2024-05-24T11:37:27.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.068+0000 7fefe1728700 1 -- 172.21.15.39:0/2086159961 wait complete. 2024-05-24T11:37:27.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.224+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faeec0683c0 con 0x7faeec1063c0 2024-05-24T11:37:27.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.224+0000 7faee9ffb700 1 -- 172.21.15.39:0/2687356712 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faee4019090 con 0x7faeec1063c0 2024-05-24T11:37:27.229 INFO:teuthology.orchestra.run.smithi039.stdout:137438953515 2024-05-24T11:37:27.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.226+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faedc0a1230 msgr2=0x7faedc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.226+0000 7faef31c6700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faedc0a1230 0x7faedc0a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faee0009990 tx=0x7faee0005fb0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.226+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec1063c0 msgr2=0x7faeec19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:27.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.226+0000 7faef31c6700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec1063c0 0x7faeec19bbe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faee400bb00 tx=0x7faee400be10 comp rx=0 tx=0).stop 2024-05-24T11:37:27.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.227+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 shutdown_connections 2024-05-24T11:37:27.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.227+0000 7faef31c6700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faedc0a1230 0x7faedc0a36f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.227+0000 7faef31c6700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeec0ffba0 0x7faeec1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.227+0000 7faef31c6700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeec1063c0 0x7faeec19bbe0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.227+0000 7faef31c6700 1 --2- 172.21.15.39:0/2687356712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec100550 0x7faeec197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:27.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.227+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 >> 172.21.15.39:0/2687356712 conn(0x7faeec075260 msgr2=0x7faeec105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:27.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.227+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 shutdown_connections 2024-05-24T11:37:27.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:27.227+0000 7faef31c6700 1 -- 172.21.15.39:0/2687356712 wait complete. 2024-05-24T11:37:27.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:26 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/27954333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-24T11:37:27.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:26 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/84423204' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-24T11:37:27.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:26 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/27954333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-24T11:37:27.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:26 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/84423204' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-24T11:37:27.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:27 smithi039 ceph-mon[42919]: pgmap v239: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-24T11:37:27.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:27 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2086159961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-24T11:37:27.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:27 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2947072712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-24T11:37:27.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:27 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/880933338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-24T11:37:27.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:27 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2687356712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-24T11:37:28.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953513 got 137438953515 for osd.4 2024-05-24T11:37:28.156 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:28.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822693 got 154618822695 for osd.5 2024-05-24T11:37:28.172 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:27 smithi053 ceph-mon[47448]: pgmap v239: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-24T11:37:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:27 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2086159961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-24T11:37:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:27 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2947072712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-24T11:37:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:27 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/880933338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-24T11:37:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:27 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2687356712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-24T11:37:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:27 smithi137 ceph-mon[47907]: pgmap v239: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-24T11:37:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:27 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2086159961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-24T11:37:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:27 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2947072712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-24T11:37:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:27 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/880933338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-24T11:37:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:27 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2687356712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-24T11:37:28.740 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462941 got 201863462943 for osd.7 2024-05-24T11:37:28.740 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:28.751 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738423 got 34359738425 for osd.0 2024-05-24T11:37:28.751 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:28.972 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593761 got 184683593763 for osd.6 2024-05-24T11:37:28.972 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:29.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.524+0000 7f12271a4700 1 -- 172.21.15.39:0/2946845753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 msgr2=0x7f122010ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:29.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.524+0000 7f12271a4700 1 --2- 172.21.15.39:0/2946845753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 0x7f122010ccf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f121800b0a0 tx=0x7f121800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.524+0000 7f12271a4700 1 -- 172.21.15.39:0/2946845753 shutdown_connections 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.524+0000 7f12271a4700 1 --2- 172.21.15.39:0/2946845753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 0x7f122010ccf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.524+0000 7f12271a4700 1 --2- 172.21.15.39:0/2946845753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220105f40 0x7f1220106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.524+0000 7f12271a4700 1 --2- 172.21.15.39:0/2946845753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12200ffe90 0x7f1220100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.524+0000 7f12271a4700 1 -- 172.21.15.39:0/2946845753 >> 172.21.15.39:0/2946845753 conn(0x7f1220076e70 msgr2=0x7f1220077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.524+0000 7f12271a4700 1 -- 172.21.15.39:0/2946845753 shutdown_connections 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f12271a4700 1 -- 172.21.15.39:0/2946845753 wait complete. 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f12271a4700 1 Processor -- start 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f12271a4700 1 -- start start 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f12271a4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12200ffe90 0x7f1220101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f12271a4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 0x7f1220102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:29.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f12271a4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1220105f40 0x7f12201025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f12271a4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12201a56c0 con 0x7f1220100850 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f12271a4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12201a5830 con 0x7f12200ffe90 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f12271a4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12201a59f0 con 0x7f1220105f40 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f121ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 0x7f1220102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f121ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 0x7f1220102080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53058/0 (socket says 172.21.15.39:53058) 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.525+0000 7f121ffff700 1 -- 172.21.15.39:0/2980308595 learned_addr learned my addr 172.21.15.39:0/2980308595 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f1224f40700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12200ffe90 0x7f1220101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f1225741700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1220105f40 0x7f12201025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f121ffff700 1 -- 172.21.15.39:0/2980308595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1220105f40 msgr2=0x7f12201025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f121ffff700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1220105f40 0x7f12201025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f121ffff700 1 -- 172.21.15.39:0/2980308595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12200ffe90 msgr2=0x7f1220101b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f121ffff700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12200ffe90 0x7f1220101b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f121ffff700 1 -- 172.21.15.39:0/2980308595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218009d00 con 0x7f1220100850 2024-05-24T11:37:29.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f121ffff700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 0x7f1220102080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f120c00d950 tx=0x7f120c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:29.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f121dffb700 1 -- 172.21.15.39:0/2980308595 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00f890 con 0x7f1220100850 2024-05-24T11:37:29.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12201a5cd0 con 0x7f1220100850 2024-05-24T11:37:29.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f121dffb700 1 -- 172.21.15.39:0/2980308595 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f120c00fe80 con 0x7f1220100850 2024-05-24T11:37:29.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f121dffb700 1 -- 172.21.15.39:0/2980308595 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00e3b0 con 0x7f1220100850 2024-05-24T11:37:29.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.526+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12201a61c0 con 0x7f1220100850 2024-05-24T11:37:29.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.527+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1220101580 con 0x7f1220100850 2024-05-24T11:37:29.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.528+0000 7f121dffb700 1 -- 172.21.15.39:0/2980308595 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f120c010460 con 0x7f1220100850 2024-05-24T11:37:29.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.529+0000 7f121dffb700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12100a11b0 0x7f12100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:29.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.529+0000 7f1224f40700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12100a11b0 0x7f12100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:29.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.529+0000 7f121dffb700 1 -- 172.21.15.39:0/2980308595 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f120c0cc670 con 0x7f1220100850 2024-05-24T11:37:29.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.530+0000 7f1224f40700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12100a11b0 0x7f12100a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1214005950 tx=0x7f121400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:29.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.534+0000 7f121dffb700 1 -- 172.21.15.39:0/2980308595 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f120c095e50 con 0x7f1220100850 2024-05-24T11:37:29.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.737+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f122010fc40 con 0x7f1220100850 2024-05-24T11:37:29.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.737+0000 7f121dffb700 1 -- 172.21.15.39:0/2980308595 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f120c099470 con 0x7f1220100850 2024-05-24T11:37:29.742 INFO:teuthology.orchestra.run.smithi039.stdout:73014444082 2024-05-24T11:37:29.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12100a11b0 msgr2=0x7f12100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:29.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12100a11b0 0x7f12100a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1214005950 tx=0x7f121400b410 comp rx=0 tx=0).stop 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 msgr2=0x7f1220102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 0x7f1220102080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f120c00d950 tx=0x7f120c00dc60 comp rx=0 tx=0).stop 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 shutdown_connections 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f12100a11b0 0x7f12100a3670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1220105f40 0x7f12201025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12200ffe90 0x7f1220101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 --2- 172.21.15.39:0/2980308595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1220100850 0x7f1220102080 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 >> 172.21.15.39:0/2980308595 conn(0x7f1220076e70 msgr2=0x7f12200ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 shutdown_connections 2024-05-24T11:37:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:29.740+0000 7f12271a4700 1 -- 172.21.15.39:0/2980308595 wait complete. 2024-05-24T11:37:30.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:29 smithi039 ceph-mon[42919]: pgmap v240: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-24T11:37:30.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:29 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2980308595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-24T11:37:30.121 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444080 got 73014444082 for osd.2 2024-05-24T11:37:30.122 DEBUG:teuthology.parallel:result is None 2024-05-24T11:37:30.122 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-24T11:37:30.122 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-24T11:37:30.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:29 smithi053 ceph-mon[47448]: pgmap v240: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-24T11:37:30.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:29 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2980308595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-24T11:37:30.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:29 smithi137 ceph-mon[47907]: pgmap v240: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-24T11:37:30.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:29 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2980308595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-24T11:37:30.353 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.299+0000 7fd3e2046700 1 -- 172.21.15.39:0/1205298721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 msgr2=0x7fd3dc10cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.299+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1205298721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 0x7fd3dc10cd40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400b0a0 tx=0x7fd3d400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.299+0000 7fd3e2046700 1 -- 172.21.15.39:0/1205298721 shutdown_connections 2024-05-24T11:37:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.299+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1205298721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 0x7fd3dc10cd40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.299+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1205298721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc105f00 0x7fd3dc1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.299+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1205298721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3dc0ffed0 0x7fd3dc100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.299+0000 7fd3e2046700 1 -- 172.21.15.39:0/1205298721 >> 172.21.15.39:0/1205298721 conn(0x7fd3dc075f80 msgr2=0x7fd3dc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.300+0000 7fd3e2046700 1 -- 172.21.15.39:0/1205298721 shutdown_connections 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.300+0000 7fd3e2046700 1 -- 172.21.15.39:0/1205298721 wait complete. 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.300+0000 7fd3e2046700 1 Processor -- start 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3e2046700 1 -- start start 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3e2046700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3dc0ffed0 0x7fd3dc196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3e2046700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 0x7fd3dc196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3e2046700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc105f00 0x7fd3dc19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3e2046700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc19b220 con 0x7fd3dc100890 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3e2046700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc19b390 con 0x7fd3dc105f00 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3e2046700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc19b550 con 0x7fd3dc0ffed0 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3daffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 0x7fd3dc196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3daffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 0x7fd3dc196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53084/0 (socket says 172.21.15.39:53084) 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3daffd700 1 -- 172.21.15.39:0/1107926636 learned_addr learned my addr 172.21.15.39:0/1107926636 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3db7fe700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3dc0ffed0 0x7fd3dc196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3dbfff700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc105f00 0x7fd3dc19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3daffd700 1 -- 172.21.15.39:0/1107926636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3dc0ffed0 msgr2=0x7fd3dc196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3daffd700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3dc0ffed0 0x7fd3dc196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3daffd700 1 -- 172.21.15.39:0/1107926636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc105f00 msgr2=0x7fd3dc19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3daffd700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc105f00 0x7fd3dc19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.301+0000 7fd3daffd700 1 -- 172.21.15.39:0/1107926636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4009d00 con 0x7fd3dc100890 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.302+0000 7fd3daffd700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 0x7fd3dc196860 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00098f0 tx=0x7fd3d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.302+0000 7fd3d8ff9700 1 -- 172.21.15.39:0/1107926636 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0012070 con 0x7fd3dc100890 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.302+0000 7fd3d8ff9700 1 -- 172.21.15.39:0/1107926636 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd3d0004830 con 0x7fd3dc100890 2024-05-24T11:37:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.302+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3dc19b830 con 0x7fd3dc100890 2024-05-24T11:37:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.302+0000 7fd3d8ff9700 1 -- 172.21.15.39:0/1107926636 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0005570 con 0x7fd3dc100890 2024-05-24T11:37:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.302+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc068b20 con 0x7fd3dc100890 2024-05-24T11:37:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.303+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3dc19ba50 con 0x7fd3dc100890 2024-05-24T11:37:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.304+0000 7fd3d8ff9700 1 -- 172.21.15.39:0/1107926636 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd3d0016070 con 0x7fd3dc100890 2024-05-24T11:37:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.304+0000 7fd3d8ff9700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd3c40a11b0 0x7fd3c40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:31.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.305+0000 7fd3db7fe700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd3c40a11b0 0x7fd3c40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:31.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.305+0000 7fd3d8ff9700 1 -- 172.21.15.39:0/1107926636 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd3d00ce020 con 0x7fd3dc100890 2024-05-24T11:37:31.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.305+0000 7fd3db7fe700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd3c40a11b0 0x7fd3c40a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00abd0 tx=0x7fd3cc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:31.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.308+0000 7fd3d8ff9700 1 -- 172.21.15.39:0/1107926636 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd3d0097800 con 0x7fd3dc100890 2024-05-24T11:37:31.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.508+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3dc197630 con 0x7fd3c40a11b0 2024-05-24T11:37:31.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.519+0000 7fd3d8ff9700 1 -- 172.21.15.39:0/1107926636 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341054 (secure 0 0 0) 0x7fd3dc197630 con 0x7fd3c40a11b0 2024-05-24T11:37:31.529 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:31.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd3c40a11b0 msgr2=0x7fd3c40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd3c40a11b0 0x7fd3c40a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00abd0 tx=0x7fd3cc00a380 comp rx=0 tx=0).stop 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 msgr2=0x7fd3dc196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 0x7fd3dc196860 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00098f0 tx=0x7fd3d0009c00 comp rx=0 tx=0).stop 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 shutdown_connections 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fd3c40a11b0 0x7fd3c40a3670 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3dc0ffed0 0x7fd3dc196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3dc105f00 0x7fd3dc19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 --2- 172.21.15.39:0/1107926636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3dc100890 0x7fd3dc196860 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 >> 172.21.15.39:0/1107926636 conn(0x7fd3dc075f80 msgr2=0x7fd3dc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 shutdown_connections 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:31.522+0000 7fd3e2046700 1 -- 172.21.15.39:0/1107926636 wait complete. 2024-05-24T11:37:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-24T11:37:31.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:31 smithi039 ceph-mon[42919]: pgmap v241: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:37:31.904 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-05-24T11:37:29.885300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654579,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":506,"num_read_kb":376,"num_write":412,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381172,"kb_used_data":11924,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124560652,"statfs":{"total":1151940427776,"available":1151550107648,"internally_reserved":0,"allocated":12210176,"data_stored":6732122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":377992517},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003068"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518423+0000","last_change":"2024-05-24T11:36:48.521973+0000","last_active":"2024-05-24T11:36:49.518423+0000","last_peered":"2024-05-24T11:36:49.518423+0000","last_clean":"2024-05-24T11:36:49.518423+0000","last_became_active":"2024-05-24T11:36:48.521587+0000","last_became_peered":"2024-05-24T11:36:48.521587+0000","last_unstale":"2024-05-24T11:36:49.518423+0000","last_undegraded":"2024-05-24T11:36:49.518423+0000","last_fullsized":"2024-05-24T11:36:49.518423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:25:29.234714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811230+0000","last_change":"2024-05-24T11:36:48.536290+0000","last_active":"2024-05-24T11:36:49.811230+0000","last_peered":"2024-05-24T11:36:49.811230+0000","last_clean":"2024-05-24T11:36:49.811230+0000","last_became_active":"2024-05-24T11:36:48.535961+0000","last_became_peered":"2024-05-24T11:36:48.535961+0000","last_unstale":"2024-05-24T11:36:49.811230+0000","last_undegraded":"2024-05-24T11:36:49.811230+0000","last_fullsized":"2024-05-24T11:36:49.811230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:52:37.951507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811216+0000","last_change":"2024-05-24T11:36:48.520039+0000","last_active":"2024-05-24T11:36:49.811216+0000","last_peered":"2024-05-24T11:36:49.811216+0000","last_clean":"2024-05-24T11:36:49.811216+0000","last_became_active":"2024-05-24T11:36:48.519809+0000","last_became_peered":"2024-05-24T11:36:48.519809+0000","last_unstale":"2024-05-24T11:36:49.811216+0000","last_undegraded":"2024-05-24T11:36:49.811216+0000","last_fullsized":"2024-05-24T11:36:49.811216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:44:20.133334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045745+0000","last_change":"2024-05-24T11:36:48.528100+0000","last_active":"2024-05-24T11:36:50.045745+0000","last_peered":"2024-05-24T11:36:50.045745+0000","last_clean":"2024-05-24T11:36:50.045745+0000","last_became_active":"2024-05-24T11:36:48.524918+0000","last_became_peered":"2024-05-24T11:36:48.524918+0000","last_unstale":"2024-05-24T11:36:50.045745+0000","last_undegraded":"2024-05-24T11:36:50.045745+0000","last_fullsized":"2024-05-24T11:36:50.045745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:20:34.628252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044676+0000","last_change":"2024-05-24T11:36:48.522735+0000","last_active":"2024-05-24T11:36:50.044676+0000","last_peered":"2024-05-24T11:36:50.044676+0000","last_clean":"2024-05-24T11:36:50.044676+0000","last_became_active":"2024-05-24T11:36:48.521657+0000","last_became_peered":"2024-05-24T11:36:48.521657+0000","last_unstale":"2024-05-24T11:36:50.044676+0000","last_undegraded":"2024-05-24T11:36:50.044676+0000","last_fullsized":"2024-05-24T11:36:50.044676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:03:32.056035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.835970+0000","last_change":"2024-05-24T11:36:48.534898+0000","last_active":"2024-05-24T11:36:48.835970+0000","last_peered":"2024-05-24T11:36:48.835970+0000","last_clean":"2024-05-24T11:36:48.835970+0000","last_became_active":"2024-05-24T11:36:48.534557+0000","last_became_peered":"2024-05-24T11:36:48.534557+0000","last_unstale":"2024-05-24T11:36:48.835970+0000","last_undegraded":"2024-05-24T11:36:48.835970+0000","last_fullsized":"2024-05-24T11:36:48.835970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:59:23.847279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.533394+0000","last_change":"2024-05-24T11:36:48.532435+0000","last_active":"2024-05-24T11:36:48.533394+0000","last_peered":"2024-05-24T11:36:48.533394+0000","last_clean":"2024-05-24T11:36:48.533394+0000","last_became_active":"2024-05-24T11:36:48.532058+0000","last_became_peered":"2024-05-24T11:36:48.532058+0000","last_unstale":"2024-05-24T11:36:48.533394+0000","last_undegraded":"2024-05-24T11:36:48.533394+0000","last_fullsized":"2024-05-24T11:36:48.533394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:33:26.178124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044447+0000","last_change":"2024-05-24T11:36:48.519054+0000","last_active":"2024-05-24T11:36:50.044447+0000","last_peered":"2024-05-24T11:36:50.044447+0000","last_clean":"2024-05-24T11:36:50.044447+0000","last_became_active":"2024-05-24T11:36:48.518756+0000","last_became_peered":"2024-05-24T11:36:48.518756+0000","last_unstale":"2024-05-24T11:36:50.044447+0000","last_undegraded":"2024-05-24T11:36:50.044447+0000","last_fullsized":"2024-05-24T11:36:50.044447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:56:19.599232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518761+0000","last_change":"2024-05-24T11:36:48.522937+0000","last_active":"2024-05-24T11:36:49.518761+0000","last_peered":"2024-05-24T11:36:49.518761+0000","last_clean":"2024-05-24T11:36:49.518761+0000","last_became_active":"2024-05-24T11:36:48.521531+0000","last_became_peered":"2024-05-24T11:36:48.521531+0000","last_unstale":"2024-05-24T11:36:49.518761+0000","last_undegraded":"2024-05-24T11:36:49.518761+0000","last_fullsized":"2024-05-24T11:36:49.518761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:16:54.392477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.836050+0000","last_change":"2024-05-24T11:36:48.536738+0000","last_active":"2024-05-24T11:36:48.836050+0000","last_peered":"2024-05-24T11:36:48.836050+0000","last_clean":"2024-05-24T11:36:48.836050+0000","last_became_active":"2024-05-24T11:36:48.536010+0000","last_became_peered":"2024-05-24T11:36:48.536010+0000","last_unstale":"2024-05-24T11:36:48.836050+0000","last_undegraded":"2024-05-24T11:36:48.836050+0000","last_fullsized":"2024-05-24T11:36:48.836050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:21:24.078676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.536403+0000","last_change":"2024-05-24T11:36:48.535036+0000","last_active":"2024-05-24T11:36:48.536403+0000","last_peered":"2024-05-24T11:36:48.536403+0000","last_clean":"2024-05-24T11:36:48.536403+0000","last_became_active":"2024-05-24T11:36:48.534232+0000","last_became_peered":"2024-05-24T11:36:48.534232+0000","last_unstale":"2024-05-24T11:36:48.536403+0000","last_undegraded":"2024-05-24T11:36:48.536403+0000","last_fullsized":"2024-05-24T11:36:48.536403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:56:28.111680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045685+0000","last_change":"2024-05-24T11:36:48.537837+0000","last_active":"2024-05-24T11:36:50.045685+0000","last_peered":"2024-05-24T11:36:50.045685+0000","last_clean":"2024-05-24T11:36:50.045685+0000","last_became_active":"2024-05-24T11:36:48.537570+0000","last_became_peered":"2024-05-24T11:36:48.537570+0000","last_unstale":"2024-05-24T11:36:50.045685+0000","last_undegraded":"2024-05-24T11:36:50.045685+0000","last_fullsized":"2024-05-24T11:36:50.045685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:47:14.769120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045004+0000","last_change":"2024-05-24T11:36:48.525333+0000","last_active":"2024-05-24T11:36:50.045004+0000","last_peered":"2024-05-24T11:36:50.045004+0000","last_clean":"2024-05-24T11:36:50.045004+0000","last_became_active":"2024-05-24T11:36:48.525146+0000","last_became_peered":"2024-05-24T11:36:48.525146+0000","last_unstale":"2024-05-24T11:36:50.045004+0000","last_undegraded":"2024-05-24T11:36:50.045004+0000","last_fullsized":"2024-05-24T11:36:50.045004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:22:58.950175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811013+0000","last_change":"2024-05-24T11:36:48.525438+0000","last_active":"2024-05-24T11:36:49.811013+0000","last_peered":"2024-05-24T11:36:49.811013+0000","last_clean":"2024-05-24T11:36:49.811013+0000","last_became_active":"2024-05-24T11:36:48.524866+0000","last_became_peered":"2024-05-24T11:36:48.524866+0000","last_unstale":"2024-05-24T11:36:49.811013+0000","last_undegraded":"2024-05-24T11:36:49.811013+0000","last_fullsized":"2024-05-24T11:36:49.811013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:39:11.826673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.810917+0000","last_change":"2024-05-24T11:36:48.529993+0000","last_active":"2024-05-24T11:36:49.810917+0000","last_peered":"2024-05-24T11:36:49.810917+0000","last_clean":"2024-05-24T11:36:49.810917+0000","last_became_active":"2024-05-24T11:36:48.529644+0000","last_became_peered":"2024-05-24T11:36:48.529644+0000","last_unstale":"2024-05-24T11:36:49.810917+0000","last_undegraded":"2024-05-24T11:36:49.810917+0000","last_fullsized":"2024-05-24T11:36:49.810917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:00:02.580642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518315+0000","last_change":"2024-05-24T11:36:48.521904+0000","last_active":"2024-05-24T11:36:49.518315+0000","last_peered":"2024-05-24T11:36:49.518315+0000","last_clean":"2024-05-24T11:36:49.518315+0000","last_became_active":"2024-05-24T11:36:48.521281+0000","last_became_peered":"2024-05-24T11:36:48.521281+0000","last_unstale":"2024-05-24T11:36:49.518315+0000","last_undegraded":"2024-05-24T11:36:49.518315+0000","last_fullsized":"2024-05-24T11:36:49.518315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:16:32.578404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518556+0000","last_change":"2024-05-24T11:36:48.526737+0000","last_active":"2024-05-24T11:36:49.518556+0000","last_peered":"2024-05-24T11:36:49.518556+0000","last_clean":"2024-05-24T11:36:49.518556+0000","last_became_active":"2024-05-24T11:36:48.526379+0000","last_became_peered":"2024-05-24T11:36:48.526379+0000","last_unstale":"2024-05-24T11:36:49.518556+0000","last_undegraded":"2024-05-24T11:36:49.518556+0000","last_fullsized":"2024-05-24T11:36:49.518556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:32:19.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.522182+0000","last_change":"2024-05-24T11:36:48.529315+0000","last_active":"2024-05-24T11:36:49.522182+0000","last_peered":"2024-05-24T11:36:49.522182+0000","last_clean":"2024-05-24T11:36:49.522182+0000","last_became_active":"2024-05-24T11:36:48.528963+0000","last_became_peered":"2024-05-24T11:36:48.528963+0000","last_unstale":"2024-05-24T11:36:49.522182+0000","last_undegraded":"2024-05-24T11:36:49.522182+0000","last_fullsized":"2024-05-24T11:36:49.522182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:54:23.922811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.544120+0000","last_change":"2024-05-24T11:36:48.520925+0000","last_active":"2024-05-24T11:36:49.544120+0000","last_peered":"2024-05-24T11:36:49.544120+0000","last_clean":"2024-05-24T11:36:49.544120+0000","last_became_active":"2024-05-24T11:36:48.520592+0000","last_became_peered":"2024-05-24T11:36:48.520592+0000","last_unstale":"2024-05-24T11:36:49.544120+0000","last_undegraded":"2024-05-24T11:36:49.544120+0000","last_fullsized":"2024-05-24T11:36:49.544120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:39:09.918746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:24.632656+0000","last_change":"2024-05-24T11:36:48.090815+0000","last_active":"2024-05-24T11:37:24.632656+0000","last_peered":"2024-05-24T11:37:24.632656+0000","last_clean":"2024-05-24T11:37:24.632656+0000","last_became_active":"2024-05-24T11:36:48.090481+0000","last_became_peered":"2024-05-24T11:36:48.090481+0000","last_unstale":"2024-05-24T11:37:24.632656+0000","last_undegraded":"2024-05-24T11:37:24.632656+0000","last_fullsized":"2024-05-24T11:37:24.632656+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:32:40.850290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044588+0000","last_change":"2024-05-24T11:36:48.519300+0000","last_active":"2024-05-24T11:36:50.044588+0000","last_peered":"2024-05-24T11:36:50.044588+0000","last_clean":"2024-05-24T11:36:50.044588+0000","last_became_active":"2024-05-24T11:36:48.519004+0000","last_became_peered":"2024-05-24T11:36:48.519004+0000","last_unstale":"2024-05-24T11:36:50.044588+0000","last_undegraded":"2024-05-24T11:36:50.044588+0000","last_fullsized":"2024-05-24T11:36:50.044588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:15:58.098701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.377448+0000","last_change":"2024-05-24T11:36:48.091695+0000","last_active":"2024-05-24T11:36:51.377448+0000","last_peered":"2024-05-24T11:36:51.377448+0000","last_clean":"2024-05-24T11:36:51.377448+0000","last_became_active":"2024-05-24T11:36:48.091379+0000","last_became_peered":"2024-05-24T11:36:48.091379+0000","last_unstale":"2024-05-24T11:36:51.377448+0000","last_undegraded":"2024-05-24T11:36:51.377448+0000","last_fullsized":"2024-05-24T11:36:51.377448+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:07:55.039339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811338+0000","last_change":"2024-05-24T11:36:48.519313+0000","last_active":"2024-05-24T11:36:49.811338+0000","last_peered":"2024-05-24T11:36:49.811338+0000","last_clean":"2024-05-24T11:36:49.811338+0000","last_became_active":"2024-05-24T11:36:48.518968+0000","last_became_peered":"2024-05-24T11:36:48.518968+0000","last_unstale":"2024-05-24T11:36:49.811338+0000","last_undegraded":"2024-05-24T11:36:49.811338+0000","last_fullsized":"2024-05-24T11:36:49.811338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:51:23.712853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'12","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591747+0000","last_change":"2024-05-24T11:36:48.096014+0000","last_active":"2024-05-24T11:36:54.591747+0000","last_peered":"2024-05-24T11:36:54.591747+0000","last_clean":"2024-05-24T11:36:54.591747+0000","last_became_active":"2024-05-24T11:36:48.095261+0000","last_became_peered":"2024-05-24T11:36:48.095261+0000","last_unstale":"2024-05-24T11:36:54.591747+0000","last_undegraded":"2024-05-24T11:36:54.591747+0000","last_fullsized":"2024-05-24T11:36:54.591747+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:19:32.675770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518671+0000","last_change":"2024-05-24T11:36:48.524464+0000","last_active":"2024-05-24T11:36:49.518671+0000","last_peered":"2024-05-24T11:36:49.518671+0000","last_clean":"2024-05-24T11:36:49.518671+0000","last_became_active":"2024-05-24T11:36:48.524128+0000","last_became_peered":"2024-05-24T11:36:48.524128+0000","last_unstale":"2024-05-24T11:36:49.518671+0000","last_undegraded":"2024-05-24T11:36:49.518671+0000","last_fullsized":"2024-05-24T11:36:49.518671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:57:21.920169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591236+0000","last_change":"2024-05-24T11:36:48.095739+0000","last_active":"2024-05-24T11:36:54.591236+0000","last_peered":"2024-05-24T11:36:54.591236+0000","last_clean":"2024-05-24T11:36:54.591236+0000","last_became_active":"2024-05-24T11:36:48.095313+0000","last_became_peered":"2024-05-24T11:36:48.095313+0000","last_unstale":"2024-05-24T11:36:54.591236+0000","last_undegraded":"2024-05-24T11:36:54.591236+0000","last_fullsized":"2024-05-24T11:36:54.591236+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:59:55.283423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045238+0000","last_change":"2024-05-24T11:36:48.528788+0000","last_active":"2024-05-24T11:36:50.045238+0000","last_peered":"2024-05-24T11:36:50.045238+0000","last_clean":"2024-05-24T11:36:50.045238+0000","last_became_active":"2024-05-24T11:36:48.528358+0000","last_became_peered":"2024-05-24T11:36:48.528358+0000","last_unstale":"2024-05-24T11:36:50.045238+0000","last_undegraded":"2024-05-24T11:36:50.045238+0000","last_fullsized":"2024-05-24T11:36:50.045238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:19:37.870208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":53,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:25.628174+0000","last_change":"2024-05-24T11:36:48.093252+0000","last_active":"2024-05-24T11:37:25.628174+0000","last_peered":"2024-05-24T11:37:25.628174+0000","last_clean":"2024-05-24T11:37:25.628174+0000","last_became_active":"2024-05-24T11:36:48.092920+0000","last_became_peered":"2024-05-24T11:36:48.092920+0000","last_unstale":"2024-05-24T11:37:25.628174+0000","last_undegraded":"2024-05-24T11:37:25.628174+0000","last_fullsized":"2024-05-24T11:37:25.628174+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:51:39.032589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044926+0000","last_change":"2024-05-24T11:36:48.518108+0000","last_active":"2024-05-24T11:36:50.044926+0000","last_peered":"2024-05-24T11:36:50.044926+0000","last_clean":"2024-05-24T11:36:50.044926+0000","last_became_active":"2024-05-24T11:36:48.517786+0000","last_became_peered":"2024-05-24T11:36:48.517786+0000","last_unstale":"2024-05-24T11:36:50.044926+0000","last_undegraded":"2024-05-24T11:36:50.044926+0000","last_fullsized":"2024-05-24T11:36:50.044926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:41:44.779354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:53.618566+0000","last_change":"2024-05-24T11:36:48.082348+0000","last_active":"2024-05-24T11:36:53.618566+0000","last_peered":"2024-05-24T11:36:53.618566+0000","last_clean":"2024-05-24T11:36:53.618566+0000","last_became_active":"2024-05-24T11:36:48.081964+0000","last_became_peered":"2024-05-24T11:36:48.081964+0000","last_unstale":"2024-05-24T11:36:53.618566+0000","last_undegraded":"2024-05-24T11:36:53.618566+0000","last_fullsized":"2024-05-24T11:36:53.618566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:45:54.028437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045904+0000","last_change":"2024-05-24T11:36:48.528990+0000","last_active":"2024-05-24T11:36:50.045904+0000","last_peered":"2024-05-24T11:36:50.045904+0000","last_clean":"2024-05-24T11:36:50.045904+0000","last_became_active":"2024-05-24T11:36:48.528444+0000","last_became_peered":"2024-05-24T11:36:48.528444+0000","last_unstale":"2024-05-24T11:36:50.045904+0000","last_undegraded":"2024-05-24T11:36:50.045904+0000","last_fullsized":"2024-05-24T11:36:50.045904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:35:56.108817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.610350+0000","last_change":"2024-05-24T11:36:48.079471+0000","last_active":"2024-05-24T11:36:52.610350+0000","last_peered":"2024-05-24T11:36:52.610350+0000","last_clean":"2024-05-24T11:36:52.610350+0000","last_became_active":"2024-05-24T11:36:48.079118+0000","last_became_peered":"2024-05-24T11:36:48.079118+0000","last_unstale":"2024-05-24T11:36:52.610350+0000","last_undegraded":"2024-05-24T11:36:52.610350+0000","last_fullsized":"2024-05-24T11:36:52.610350+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:01:35.897437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044314+0000","last_change":"2024-05-24T11:36:48.517960+0000","last_active":"2024-05-24T11:36:50.044314+0000","last_peered":"2024-05-24T11:36:50.044314+0000","last_clean":"2024-05-24T11:36:50.044314+0000","last_became_active":"2024-05-24T11:36:48.517675+0000","last_became_peered":"2024-05-24T11:36:48.517675+0000","last_unstale":"2024-05-24T11:36:50.044314+0000","last_undegraded":"2024-05-24T11:36:50.044314+0000","last_fullsized":"2024-05-24T11:36:50.044314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:08:21.209080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.582937+0000","last_change":"2024-05-24T11:36:48.091922+0000","last_active":"2024-05-24T11:36:54.582937+0000","last_peered":"2024-05-24T11:36:54.582937+0000","last_clean":"2024-05-24T11:36:54.582937+0000","last_became_active":"2024-05-24T11:36:48.091576+0000","last_became_peered":"2024-05-24T11:36:48.091576+0000","last_unstale":"2024-05-24T11:36:54.582937+0000","last_undegraded":"2024-05-24T11:36:54.582937+0000","last_fullsized":"2024-05-24T11:36:54.582937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:01:39.386649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.521890+0000","last_change":"2024-05-24T11:36:48.523769+0000","last_active":"2024-05-24T11:36:49.521890+0000","last_peered":"2024-05-24T11:36:49.521890+0000","last_clean":"2024-05-24T11:36:49.521890+0000","last_became_active":"2024-05-24T11:36:48.523394+0000","last_became_peered":"2024-05-24T11:36:48.523394+0000","last_unstale":"2024-05-24T11:36:49.521890+0000","last_undegraded":"2024-05-24T11:36:49.521890+0000","last_fullsized":"2024-05-24T11:36:49.521890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:03:49.209246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":86,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:28.631803+0000","last_change":"2024-05-24T11:36:48.089651+0000","last_active":"2024-05-24T11:37:28.631803+0000","last_peered":"2024-05-24T11:37:28.631803+0000","last_clean":"2024-05-24T11:37:28.631803+0000","last_became_active":"2024-05-24T11:36:48.089195+0000","last_became_peered":"2024-05-24T11:36:48.089195+0000","last_unstale":"2024-05-24T11:37:28.631803+0000","last_undegraded":"2024-05-24T11:37:28.631803+0000","last_fullsized":"2024-05-24T11:37:28.631803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:30:47.730594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518732+0000","last_change":"2024-05-24T11:36:48.531529+0000","last_active":"2024-05-24T11:36:49.518732+0000","last_peered":"2024-05-24T11:36:49.518732+0000","last_clean":"2024-05-24T11:36:49.518732+0000","last_became_active":"2024-05-24T11:36:48.530969+0000","last_became_peered":"2024-05-24T11:36:48.530969+0000","last_unstale":"2024-05-24T11:36:49.518732+0000","last_undegraded":"2024-05-24T11:36:49.518732+0000","last_fullsized":"2024-05-24T11:36:49.518732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:05:24.929278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591832+0000","last_change":"2024-05-24T11:36:48.094249+0000","last_active":"2024-05-24T11:36:54.591832+0000","last_peered":"2024-05-24T11:36:54.591832+0000","last_clean":"2024-05-24T11:36:54.591832+0000","last_became_active":"2024-05-24T11:36:48.093835+0000","last_became_peered":"2024-05-24T11:36:48.093835+0000","last_unstale":"2024-05-24T11:36:54.591832+0000","last_undegraded":"2024-05-24T11:36:54.591832+0000","last_fullsized":"2024-05-24T11:36:54.591832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:41:47.660513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811338+0000","last_change":"2024-05-24T11:36:48.531745+0000","last_active":"2024-05-24T11:36:49.811338+0000","last_peered":"2024-05-24T11:36:49.811338+0000","last_clean":"2024-05-24T11:36:49.811338+0000","last_became_active":"2024-05-24T11:36:48.531459+0000","last_became_peered":"2024-05-24T11:36:48.531459+0000","last_unstale":"2024-05-24T11:36:49.811338+0000","last_undegraded":"2024-05-24T11:36:49.811338+0000","last_fullsized":"2024-05-24T11:36:49.811338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:06:41.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:53.617054+0000","last_change":"2024-05-24T11:36:48.089567+0000","last_active":"2024-05-24T11:36:53.617054+0000","last_peered":"2024-05-24T11:36:53.617054+0000","last_clean":"2024-05-24T11:36:53.617054+0000","last_became_active":"2024-05-24T11:36:48.089248+0000","last_became_peered":"2024-05-24T11:36:48.089248+0000","last_unstale":"2024-05-24T11:36:53.617054+0000","last_undegraded":"2024-05-24T11:36:53.617054+0000","last_fullsized":"2024-05-24T11:36:53.617054+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:49:06.307799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518786+0000","last_change":"2024-05-24T11:36:48.529531+0000","last_active":"2024-05-24T11:36:49.518786+0000","last_peered":"2024-05-24T11:36:49.518786+0000","last_clean":"2024-05-24T11:36:49.518786+0000","last_became_active":"2024-05-24T11:36:48.529222+0000","last_became_peered":"2024-05-24T11:36:48.529222+0000","last_unstale":"2024-05-24T11:36:49.518786+0000","last_undegraded":"2024-05-24T11:36:49.518786+0000","last_fullsized":"2024-05-24T11:36:49.518786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:18:49.571222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'28","reported_seq":116,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:28.648510+0000","last_change":"2024-05-24T11:36:48.086123+0000","last_active":"2024-05-24T11:37:28.648510+0000","last_peered":"2024-05-24T11:37:28.648510+0000","last_clean":"2024-05-24T11:37:28.648510+0000","last_became_active":"2024-05-24T11:36:48.085681+0000","last_became_peered":"2024-05-24T11:36:48.085681+0000","last_unstale":"2024-05-24T11:37:28.648510+0000","last_undegraded":"2024-05-24T11:37:28.648510+0000","last_fullsized":"2024-05-24T11:37:28.648510+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:17:12.646332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56949,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":94,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045057+0000","last_change":"2024-05-24T11:36:48.524883+0000","last_active":"2024-05-24T11:36:50.045057+0000","last_peered":"2024-05-24T11:36:50.045057+0000","last_clean":"2024-05-24T11:36:50.045057+0000","last_became_active":"2024-05-24T11:36:48.524578+0000","last_became_peered":"2024-05-24T11:36:48.524578+0000","last_unstale":"2024-05-24T11:36:50.045057+0000","last_undegraded":"2024-05-24T11:36:50.045057+0000","last_fullsized":"2024-05-24T11:36:50.045057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:54:15.419099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591054+0000","last_change":"2024-05-24T11:36:48.090437+0000","last_active":"2024-05-24T11:36:54.591054+0000","last_peered":"2024-05-24T11:36:54.591054+0000","last_clean":"2024-05-24T11:36:54.591054+0000","last_became_active":"2024-05-24T11:36:48.088745+0000","last_became_peered":"2024-05-24T11:36:48.088745+0000","last_unstale":"2024-05-24T11:36:54.591054+0000","last_undegraded":"2024-05-24T11:36:54.591054+0000","last_fullsized":"2024-05-24T11:36:54.591054+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:12:48.829945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044821+0000","last_change":"2024-05-24T11:36:48.525723+0000","last_active":"2024-05-24T11:36:50.044821+0000","last_peered":"2024-05-24T11:36:50.044821+0000","last_clean":"2024-05-24T11:36:50.044821+0000","last_became_active":"2024-05-24T11:36:48.525267+0000","last_became_peered":"2024-05-24T11:36:48.525267+0000","last_unstale":"2024-05-24T11:36:50.044821+0000","last_undegraded":"2024-05-24T11:36:50.044821+0000","last_fullsized":"2024-05-24T11:36:50.044821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:27:59.404214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.590042+0000","last_change":"2024-05-24T11:36:48.094426+0000","last_active":"2024-05-24T11:36:54.590042+0000","last_peered":"2024-05-24T11:36:54.590042+0000","last_clean":"2024-05-24T11:36:54.590042+0000","last_became_active":"2024-05-24T11:36:48.094169+0000","last_became_peered":"2024-05-24T11:36:48.094169+0000","last_unstale":"2024-05-24T11:36:54.590042+0000","last_undegraded":"2024-05-24T11:36:54.590042+0000","last_fullsized":"2024-05-24T11:36:54.590042+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:31:02.682818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.531031+0000","last_change":"2024-05-24T11:36:48.529851+0000","last_active":"2024-05-24T11:36:48.531031+0000","last_peered":"2024-05-24T11:36:48.531031+0000","last_clean":"2024-05-24T11:36:48.531031+0000","last_became_active":"2024-05-24T11:36:48.529540+0000","last_became_peered":"2024-05-24T11:36:48.529540+0000","last_unstale":"2024-05-24T11:36:48.531031+0000","last_undegraded":"2024-05-24T11:36:48.531031+0000","last_fullsized":"2024-05-24T11:36:48.531031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:22:12.101608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'5","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:23.637466+0000","last_change":"2024-05-24T11:36:48.090611+0000","last_active":"2024-05-24T11:37:23.637466+0000","last_peered":"2024-05-24T11:37:23.637466+0000","last_clean":"2024-05-24T11:37:23.637466+0000","last_became_active":"2024-05-24T11:36:48.090283+0000","last_became_peered":"2024-05-24T11:36:48.090283+0000","last_unstale":"2024-05-24T11:37:23.637466+0000","last_undegraded":"2024-05-24T11:37:23.637466+0000","last_fullsized":"2024-05-24T11:37:23.637466+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:24:57.272371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.537097+0000","last_change":"2024-05-24T11:36:48.536763+0000","last_active":"2024-05-24T11:36:48.537097+0000","last_peered":"2024-05-24T11:36:48.537097+0000","last_clean":"2024-05-24T11:36:48.537097+0000","last_became_active":"2024-05-24T11:36:48.536398+0000","last_became_peered":"2024-05-24T11:36:48.536398+0000","last_unstale":"2024-05-24T11:36:48.537097+0000","last_undegraded":"2024-05-24T11:36:48.537097+0000","last_fullsized":"2024-05-24T11:36:48.537097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:13:58.446126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.581083+0000","last_change":"2024-05-24T11:36:48.080590+0000","last_active":"2024-05-24T11:36:54.581083+0000","last_peered":"2024-05-24T11:36:54.581083+0000","last_clean":"2024-05-24T11:36:54.581083+0000","last_became_active":"2024-05-24T11:36:48.080192+0000","last_became_peered":"2024-05-24T11:36:48.080192+0000","last_unstale":"2024-05-24T11:36:54.581083+0000","last_undegraded":"2024-05-24T11:36:54.581083+0000","last_fullsized":"2024-05-24T11:36:54.581083+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:13:35.452878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044986+0000","last_change":"2024-05-24T11:36:48.530444+0000","last_active":"2024-05-24T11:36:50.044986+0000","last_peered":"2024-05-24T11:36:50.044986+0000","last_clean":"2024-05-24T11:36:50.044986+0000","last_became_active":"2024-05-24T11:36:48.530063+0000","last_became_peered":"2024-05-24T11:36:48.530063+0000","last_unstale":"2024-05-24T11:36:50.044986+0000","last_undegraded":"2024-05-24T11:36:50.044986+0000","last_fullsized":"2024-05-24T11:36:50.044986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:23:12.380170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.581432+0000","last_change":"2024-05-24T11:36:48.086566+0000","last_active":"2024-05-24T11:36:54.581432+0000","last_peered":"2024-05-24T11:36:54.581432+0000","last_clean":"2024-05-24T11:36:54.581432+0000","last_became_active":"2024-05-24T11:36:48.086213+0000","last_became_peered":"2024-05-24T11:36:48.086213+0000","last_unstale":"2024-05-24T11:36:54.581432+0000","last_undegraded":"2024-05-24T11:36:54.581432+0000","last_fullsized":"2024-05-24T11:36:54.581432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:09:53.744478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518496+0000","last_change":"2024-05-24T11:36:48.528746+0000","last_active":"2024-05-24T11:36:49.518496+0000","last_peered":"2024-05-24T11:36:49.518496+0000","last_clean":"2024-05-24T11:36:49.518496+0000","last_became_active":"2024-05-24T11:36:48.528399+0000","last_became_peered":"2024-05-24T11:36:48.528399+0000","last_unstale":"2024-05-24T11:36:49.518496+0000","last_undegraded":"2024-05-24T11:36:49.518496+0000","last_fullsized":"2024-05-24T11:36:49.518496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:51:28.838280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.790552+0000","last_change":"2024-05-24T11:36:48.084268+0000","last_active":"2024-05-24T11:36:50.790552+0000","last_peered":"2024-05-24T11:36:50.790552+0000","last_clean":"2024-05-24T11:36:50.790552+0000","last_became_active":"2024-05-24T11:36:48.083968+0000","last_became_peered":"2024-05-24T11:36:48.083968+0000","last_unstale":"2024-05-24T11:36:50.790552+0000","last_undegraded":"2024-05-24T11:36:50.790552+0000","last_fullsized":"2024-05-24T11:36:50.790552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:02:00.989621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.046135+0000","last_change":"2024-05-24T11:36:48.523784+0000","last_active":"2024-05-24T11:36:50.046135+0000","last_peered":"2024-05-24T11:36:50.046135+0000","last_clean":"2024-05-24T11:36:50.046135+0000","last_became_active":"2024-05-24T11:36:48.523430+0000","last_became_peered":"2024-05-24T11:36:48.523430+0000","last_unstale":"2024-05-24T11:36:50.046135+0000","last_undegraded":"2024-05-24T11:36:50.046135+0000","last_fullsized":"2024-05-24T11:36:50.046135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:02:33.070861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'5","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:26.635773+0000","last_change":"2024-05-24T11:36:48.092166+0000","last_active":"2024-05-24T11:37:26.635773+0000","last_peered":"2024-05-24T11:37:26.635773+0000","last_clean":"2024-05-24T11:37:26.635773+0000","last_became_active":"2024-05-24T11:36:48.091682+0000","last_became_peered":"2024-05-24T11:36:48.091682+0000","last_unstale":"2024-05-24T11:37:26.635773+0000","last_undegraded":"2024-05-24T11:37:26.635773+0000","last_fullsized":"2024-05-24T11:37:26.635773+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:14:53.083387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.835997+0000","last_change":"2024-05-24T11:36:48.534618+0000","last_active":"2024-05-24T11:36:48.835997+0000","last_peered":"2024-05-24T11:36:48.835997+0000","last_clean":"2024-05-24T11:36:48.835997+0000","last_became_active":"2024-05-24T11:36:48.534251+0000","last_became_peered":"2024-05-24T11:36:48.534251+0000","last_unstale":"2024-05-24T11:36:48.835997+0000","last_undegraded":"2024-05-24T11:36:48.835997+0000","last_fullsized":"2024-05-24T11:36:48.835997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:14:46.889921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045023+0000","last_change":"2024-05-24T11:36:48.528214+0000","last_active":"2024-05-24T11:36:50.045023+0000","last_peered":"2024-05-24T11:36:50.045023+0000","last_clean":"2024-05-24T11:36:50.045023+0000","last_became_active":"2024-05-24T11:36:48.526711+0000","last_became_peered":"2024-05-24T11:36:48.526711+0000","last_unstale":"2024-05-24T11:36:50.045023+0000","last_undegraded":"2024-05-24T11:36:50.045023+0000","last_fullsized":"2024-05-24T11:36:50.045023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:06:27.935139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:24.638419+0000","last_change":"2024-05-24T11:36:48.089263+0000","last_active":"2024-05-24T11:37:24.638419+0000","last_peered":"2024-05-24T11:37:24.638419+0000","last_clean":"2024-05-24T11:37:24.638419+0000","last_became_active":"2024-05-24T11:36:48.087905+0000","last_became_peered":"2024-05-24T11:36:48.087905+0000","last_unstale":"2024-05-24T11:37:24.638419+0000","last_undegraded":"2024-05-24T11:37:24.638419+0000","last_fullsized":"2024-05-24T11:37:24.638419+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:06:37.941039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044877+0000","last_change":"2024-05-24T11:36:48.517569+0000","last_active":"2024-05-24T11:36:50.044877+0000","last_peered":"2024-05-24T11:36:50.044877+0000","last_clean":"2024-05-24T11:36:50.044877+0000","last_became_active":"2024-05-24T11:36:48.517202+0000","last_became_peered":"2024-05-24T11:36:48.517202+0000","last_unstale":"2024-05-24T11:36:50.044877+0000","last_undegraded":"2024-05-24T11:36:50.044877+0000","last_fullsized":"2024-05-24T11:36:50.044877+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:38:50.778284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591392+0000","last_change":"2024-05-24T11:36:48.077543+0000","last_active":"2024-05-24T11:36:54.591392+0000","last_peered":"2024-05-24T11:36:54.591392+0000","last_clean":"2024-05-24T11:36:54.591392+0000","last_became_active":"2024-05-24T11:36:48.077058+0000","last_became_peered":"2024-05-24T11:36:48.077058+0000","last_unstale":"2024-05-24T11:36:54.591392+0000","last_undegraded":"2024-05-24T11:36:54.591392+0000","last_fullsized":"2024-05-24T11:36:54.591392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:19:49.486922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":328,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.047880+0000","last_change":"2024-05-24T11:36:00.577887+0000","last_active":"2024-05-24T11:36:50.047880+0000","last_peered":"2024-05-24T11:36:50.047880+0000","last_clean":"2024-05-24T11:36:50.047880+0000","last_became_active":"2024-05-24T11:36:00.577291+0000","last_became_peered":"2024-05-24T11:36:00.577291+0000","last_unstale":"2024-05-24T11:36:50.047880+0000","last_undegraded":"2024-05-24T11:36:50.047880+0000","last_fullsized":"2024-05-24T11:36:50.047880+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:33:34.896223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:33:34.896223+0000","last_clean_scrub_stamp":"2024-05-24T11:33:34.896223+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:11:36.393700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044963+0000","last_change":"2024-05-24T11:36:48.527294+0000","last_active":"2024-05-24T11:36:50.044963+0000","last_peered":"2024-05-24T11:36:50.044963+0000","last_clean":"2024-05-24T11:36:50.044963+0000","last_became_active":"2024-05-24T11:36:48.526983+0000","last_became_peered":"2024-05-24T11:36:48.526983+0000","last_unstale":"2024-05-24T11:36:50.044963+0000","last_undegraded":"2024-05-24T11:36:50.044963+0000","last_fullsized":"2024-05-24T11:36:50.044963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:23:26.492502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.611289+0000","last_change":"2024-05-24T11:36:48.097290+0000","last_active":"2024-05-24T11:36:52.611289+0000","last_peered":"2024-05-24T11:36:52.611289+0000","last_clean":"2024-05-24T11:36:52.611289+0000","last_became_active":"2024-05-24T11:36:48.096942+0000","last_became_peered":"2024-05-24T11:36:48.096942+0000","last_unstale":"2024-05-24T11:36:52.611289+0000","last_undegraded":"2024-05-24T11:36:52.611289+0000","last_fullsized":"2024-05-24T11:36:52.611289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:24:20.221023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.579815+0000","last_change":"2024-05-24T11:36:48.079294+0000","last_active":"2024-05-24T11:36:54.579815+0000","last_peered":"2024-05-24T11:36:54.579815+0000","last_clean":"2024-05-24T11:36:54.579815+0000","last_became_active":"2024-05-24T11:36:48.078919+0000","last_became_peered":"2024-05-24T11:36:48.078919+0000","last_unstale":"2024-05-24T11:36:54.579815+0000","last_undegraded":"2024-05-24T11:36:54.579815+0000","last_fullsized":"2024-05-24T11:36:54.579815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:22:32.262674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045222+0000","last_change":"2024-05-24T11:36:48.524666+0000","last_active":"2024-05-24T11:36:50.045222+0000","last_peered":"2024-05-24T11:36:50.045222+0000","last_clean":"2024-05-24T11:36:50.045222+0000","last_became_active":"2024-05-24T11:36:48.524347+0000","last_became_peered":"2024-05-24T11:36:48.524347+0000","last_unstale":"2024-05-24T11:36:50.045222+0000","last_undegraded":"2024-05-24T11:36:50.045222+0000","last_fullsized":"2024-05-24T11:36:50.045222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:14:13.716982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:57.874846+0000","last_change":"2024-05-24T11:36:48.085665+0000","last_active":"2024-05-24T11:36:57.874846+0000","last_peered":"2024-05-24T11:36:57.874846+0000","last_clean":"2024-05-24T11:36:57.874846+0000","last_became_active":"2024-05-24T11:36:48.085344+0000","last_became_peered":"2024-05-24T11:36:48.085344+0000","last_unstale":"2024-05-24T11:36:57.874846+0000","last_undegraded":"2024-05-24T11:36:57.874846+0000","last_fullsized":"2024-05-24T11:36:57.874846+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:11:29.930656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044736+0000","last_change":"2024-05-24T11:36:48.523060+0000","last_active":"2024-05-24T11:36:50.044736+0000","last_peered":"2024-05-24T11:36:50.044736+0000","last_clean":"2024-05-24T11:36:50.044736+0000","last_became_active":"2024-05-24T11:36:48.522712+0000","last_became_peered":"2024-05-24T11:36:48.522712+0000","last_unstale":"2024-05-24T11:36:50.044736+0000","last_undegraded":"2024-05-24T11:36:50.044736+0000","last_fullsized":"2024-05-24T11:36:50.044736+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:07:54.311695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.589339+0000","last_change":"2024-05-24T11:36:48.092809+0000","last_active":"2024-05-24T11:36:54.589339+0000","last_peered":"2024-05-24T11:36:54.589339+0000","last_clean":"2024-05-24T11:36:54.589339+0000","last_became_active":"2024-05-24T11:36:48.092447+0000","last_became_peered":"2024-05-24T11:36:48.092447+0000","last_unstale":"2024-05-24T11:36:54.589339+0000","last_undegraded":"2024-05-24T11:36:54.589339+0000","last_fullsized":"2024-05-24T11:36:54.589339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:27:12.916761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811262+0000","last_change":"2024-05-24T11:36:48.528514+0000","last_active":"2024-05-24T11:36:49.811262+0000","last_peered":"2024-05-24T11:36:49.811262+0000","last_clean":"2024-05-24T11:36:49.811262+0000","last_became_active":"2024-05-24T11:36:48.528244+0000","last_became_peered":"2024-05-24T11:36:48.528244+0000","last_unstale":"2024-05-24T11:36:49.811262+0000","last_undegraded":"2024-05-24T11:36:49.811262+0000","last_fullsized":"2024-05-24T11:36:49.811262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:53:05.993805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599039+0000","last_change":"2024-05-24T11:36:48.094084+0000","last_active":"2024-05-24T11:36:51.599039+0000","last_peered":"2024-05-24T11:36:51.599039+0000","last_clean":"2024-05-24T11:36:51.599039+0000","last_became_active":"2024-05-24T11:36:48.093254+0000","last_became_peered":"2024-05-24T11:36:48.093254+0000","last_unstale":"2024-05-24T11:36:51.599039+0000","last_undegraded":"2024-05-24T11:36:51.599039+0000","last_fullsized":"2024-05-24T11:36:51.599039+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:56:18.896200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044843+0000","last_change":"2024-05-24T11:36:48.522292+0000","last_active":"2024-05-24T11:36:50.044843+0000","last_peered":"2024-05-24T11:36:50.044843+0000","last_clean":"2024-05-24T11:36:50.044843+0000","last_became_active":"2024-05-24T11:36:48.521356+0000","last_became_peered":"2024-05-24T11:36:48.521356+0000","last_unstale":"2024-05-24T11:36:50.044843+0000","last_undegraded":"2024-05-24T11:36:50.044843+0000","last_fullsized":"2024-05-24T11:36:50.044843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:51:44.805455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.588410+0000","last_change":"2024-05-24T11:36:48.086209+0000","last_active":"2024-05-24T11:36:54.588410+0000","last_peered":"2024-05-24T11:36:54.588410+0000","last_clean":"2024-05-24T11:36:54.588410+0000","last_became_active":"2024-05-24T11:36:48.084978+0000","last_became_peered":"2024-05-24T11:36:48.084978+0000","last_unstale":"2024-05-24T11:36:54.588410+0000","last_undegraded":"2024-05-24T11:36:54.588410+0000","last_fullsized":"2024-05-24T11:36:54.588410+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:26:39.181832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518178+0000","last_change":"2024-05-24T11:36:48.520890+0000","last_active":"2024-05-24T11:36:49.518178+0000","last_peered":"2024-05-24T11:36:49.518178+0000","last_clean":"2024-05-24T11:36:49.518178+0000","last_became_active":"2024-05-24T11:36:48.519860+0000","last_became_peered":"2024-05-24T11:36:48.519860+0000","last_unstale":"2024-05-24T11:36:49.518178+0000","last_undegraded":"2024-05-24T11:36:49.518178+0000","last_fullsized":"2024-05-24T11:36:49.518178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:54:35.691695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:57.239504+0000","last_change":"2024-05-24T11:36:48.089884+0000","last_active":"2024-05-24T11:36:57.239504+0000","last_peered":"2024-05-24T11:36:57.239504+0000","last_clean":"2024-05-24T11:36:57.239504+0000","last_became_active":"2024-05-24T11:36:48.089386+0000","last_became_peered":"2024-05-24T11:36:48.089386+0000","last_unstale":"2024-05-24T11:36:57.239504+0000","last_undegraded":"2024-05-24T11:36:57.239504+0000","last_fullsized":"2024-05-24T11:36:57.239504+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:56:14.463210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044730+0000","last_change":"2024-05-24T11:36:48.515178+0000","last_active":"2024-05-24T11:36:50.044730+0000","last_peered":"2024-05-24T11:36:50.044730+0000","last_clean":"2024-05-24T11:36:50.044730+0000","last_became_active":"2024-05-24T11:36:48.514277+0000","last_became_peered":"2024-05-24T11:36:48.514277+0000","last_unstale":"2024-05-24T11:36:50.044730+0000","last_undegraded":"2024-05-24T11:36:50.044730+0000","last_fullsized":"2024-05-24T11:36:50.044730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:46:38.526893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599557+0000","last_change":"2024-05-24T11:36:48.095493+0000","last_active":"2024-05-24T11:36:51.599557+0000","last_peered":"2024-05-24T11:36:51.599557+0000","last_clean":"2024-05-24T11:36:51.599557+0000","last_became_active":"2024-05-24T11:36:48.094863+0000","last_became_peered":"2024-05-24T11:36:48.094863+0000","last_unstale":"2024-05-24T11:36:51.599557+0000","last_undegraded":"2024-05-24T11:36:51.599557+0000","last_fullsized":"2024-05-24T11:36:51.599557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:45:06.827169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.530486+0000","last_change":"2024-05-24T11:36:48.528024+0000","last_active":"2024-05-24T11:36:48.530486+0000","last_peered":"2024-05-24T11:36:48.530486+0000","last_clean":"2024-05-24T11:36:48.530486+0000","last_became_active":"2024-05-24T11:36:48.527689+0000","last_became_peered":"2024-05-24T11:36:48.527689+0000","last_unstale":"2024-05-24T11:36:48.530486+0000","last_undegraded":"2024-05-24T11:36:48.530486+0000","last_fullsized":"2024-05-24T11:36:48.530486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:07:50.186661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'6","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:26.646321+0000","last_change":"2024-05-24T11:36:48.090660+0000","last_active":"2024-05-24T11:37:26.646321+0000","last_peered":"2024-05-24T11:37:26.646321+0000","last_clean":"2024-05-24T11:37:26.646321+0000","last_became_active":"2024-05-24T11:36:48.090298+0000","last_became_peered":"2024-05-24T11:36:48.090298+0000","last_unstale":"2024-05-24T11:37:26.646321+0000","last_undegraded":"2024-05-24T11:37:26.646321+0000","last_fullsized":"2024-05-24T11:37:26.646321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:07:30.366506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.528160+0000","last_change":"2024-05-24T11:36:48.526287+0000","last_active":"2024-05-24T11:36:48.528160+0000","last_peered":"2024-05-24T11:36:48.528160+0000","last_clean":"2024-05-24T11:36:48.528160+0000","last_became_active":"2024-05-24T11:36:48.525951+0000","last_became_peered":"2024-05-24T11:36:48.525951+0000","last_unstale":"2024-05-24T11:36:48.528160+0000","last_undegraded":"2024-05-24T11:36:48.528160+0000","last_fullsized":"2024-05-24T11:36:48.528160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:38:11.562370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.582546+0000","last_change":"2024-05-24T11:36:48.092808+0000","last_active":"2024-05-24T11:36:54.582546+0000","last_peered":"2024-05-24T11:36:54.582546+0000","last_clean":"2024-05-24T11:36:54.582546+0000","last_became_active":"2024-05-24T11:36:48.092466+0000","last_became_peered":"2024-05-24T11:36:48.092466+0000","last_unstale":"2024-05-24T11:36:54.582546+0000","last_undegraded":"2024-05-24T11:36:54.582546+0000","last_fullsized":"2024-05-24T11:36:54.582546+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:27:08.423449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.048076+0000","last_change":"2024-05-24T11:36:48.533175+0000","last_active":"2024-05-24T11:36:50.048076+0000","last_peered":"2024-05-24T11:36:50.048076+0000","last_clean":"2024-05-24T11:36:50.048076+0000","last_became_active":"2024-05-24T11:36:48.532779+0000","last_became_peered":"2024-05-24T11:36:48.532779+0000","last_unstale":"2024-05-24T11:36:50.048076+0000","last_undegraded":"2024-05-24T11:36:50.048076+0000","last_fullsized":"2024-05-24T11:36:50.048076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:39:40.041747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.589679+0000","last_change":"2024-05-24T11:36:48.080825+0000","last_active":"2024-05-24T11:36:54.589679+0000","last_peered":"2024-05-24T11:36:54.589679+0000","last_clean":"2024-05-24T11:36:54.589679+0000","last_became_active":"2024-05-24T11:36:48.080462+0000","last_became_peered":"2024-05-24T11:36:48.080462+0000","last_unstale":"2024-05-24T11:36:54.589679+0000","last_undegraded":"2024-05-24T11:36:54.589679+0000","last_fullsized":"2024-05-24T11:36:54.589679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:05:25.665222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.536673+0000","last_change":"2024-05-24T11:36:48.535094+0000","last_active":"2024-05-24T11:36:48.536673+0000","last_peered":"2024-05-24T11:36:48.536673+0000","last_clean":"2024-05-24T11:36:48.536673+0000","last_became_active":"2024-05-24T11:36:48.534670+0000","last_became_peered":"2024-05-24T11:36:48.534670+0000","last_unstale":"2024-05-24T11:36:48.536673+0000","last_undegraded":"2024-05-24T11:36:48.536673+0000","last_fullsized":"2024-05-24T11:36:48.536673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:24:49.492069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518418+0000","last_change":"2024-05-24T11:36:48.521113+0000","last_active":"2024-05-24T11:36:49.518418+0000","last_peered":"2024-05-24T11:36:49.518418+0000","last_clean":"2024-05-24T11:36:49.518418+0000","last_became_active":"2024-05-24T11:36:48.520379+0000","last_became_peered":"2024-05-24T11:36:48.520379+0000","last_unstale":"2024-05-24T11:36:49.518418+0000","last_undegraded":"2024-05-24T11:36:49.518418+0000","last_fullsized":"2024-05-24T11:36:49.518418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:07:17.112794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.543923+0000","last_change":"2024-05-24T11:36:48.529329+0000","last_active":"2024-05-24T11:36:49.543923+0000","last_peered":"2024-05-24T11:36:49.543923+0000","last_clean":"2024-05-24T11:36:49.543923+0000","last_became_active":"2024-05-24T11:36:48.528994+0000","last_became_peered":"2024-05-24T11:36:48.528994+0000","last_unstale":"2024-05-24T11:36:49.543923+0000","last_undegraded":"2024-05-24T11:36:49.543923+0000","last_fullsized":"2024-05-24T11:36:49.543923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:23:37.166318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.543981+0000","last_change":"2024-05-24T11:36:48.519429+0000","last_active":"2024-05-24T11:36:49.543981+0000","last_peered":"2024-05-24T11:36:49.543981+0000","last_clean":"2024-05-24T11:36:49.543981+0000","last_became_active":"2024-05-24T11:36:48.519085+0000","last_became_peered":"2024-05-24T11:36:48.519085+0000","last_unstale":"2024-05-24T11:36:49.543981+0000","last_undegraded":"2024-05-24T11:36:49.543981+0000","last_fullsized":"2024-05-24T11:36:49.543981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:41:55.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811057+0000","last_change":"2024-05-24T11:36:48.528932+0000","last_active":"2024-05-24T11:36:49.811057+0000","last_peered":"2024-05-24T11:36:49.811057+0000","last_clean":"2024-05-24T11:36:49.811057+0000","last_became_active":"2024-05-24T11:36:48.528413+0000","last_became_peered":"2024-05-24T11:36:48.528413+0000","last_unstale":"2024-05-24T11:36:49.811057+0000","last_undegraded":"2024-05-24T11:36:49.811057+0000","last_fullsized":"2024-05-24T11:36:49.811057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:55:47.051779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045272+0000","last_change":"2024-05-24T11:36:48.526905+0000","last_active":"2024-05-24T11:36:50.045272+0000","last_peered":"2024-05-24T11:36:50.045272+0000","last_clean":"2024-05-24T11:36:50.045272+0000","last_became_active":"2024-05-24T11:36:48.526608+0000","last_became_peered":"2024-05-24T11:36:48.526608+0000","last_unstale":"2024-05-24T11:36:50.045272+0000","last_undegraded":"2024-05-24T11:36:50.045272+0000","last_fullsized":"2024-05-24T11:36:50.045272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:32:04.831995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518371+0000","last_change":"2024-05-24T11:36:48.523597+0000","last_active":"2024-05-24T11:36:49.518371+0000","last_peered":"2024-05-24T11:36:49.518371+0000","last_clean":"2024-05-24T11:36:49.518371+0000","last_became_active":"2024-05-24T11:36:48.523279+0000","last_became_peered":"2024-05-24T11:36:48.523279+0000","last_unstale":"2024-05-24T11:36:49.518371+0000","last_undegraded":"2024-05-24T11:36:49.518371+0000","last_fullsized":"2024-05-24T11:36:49.518371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:49:46.063341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.544125+0000","last_change":"2024-05-24T11:36:48.526836+0000","last_active":"2024-05-24T11:36:49.544125+0000","last_peered":"2024-05-24T11:36:49.544125+0000","last_clean":"2024-05-24T11:36:49.544125+0000","last_became_active":"2024-05-24T11:36:48.523838+0000","last_became_peered":"2024-05-24T11:36:48.523838+0000","last_unstale":"2024-05-24T11:36:49.544125+0000","last_undegraded":"2024-05-24T11:36:49.544125+0000","last_fullsized":"2024-05-24T11:36:49.544125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:01:13.223240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044611+0000","last_change":"2024-05-24T11:36:48.532826+0000","last_active":"2024-05-24T11:36:50.044611+0000","last_peered":"2024-05-24T11:36:50.044611+0000","last_clean":"2024-05-24T11:36:50.044611+0000","last_became_active":"2024-05-24T11:36:48.532526+0000","last_became_peered":"2024-05-24T11:36:48.532526+0000","last_unstale":"2024-05-24T11:36:50.044611+0000","last_undegraded":"2024-05-24T11:36:50.044611+0000","last_fullsized":"2024-05-24T11:36:50.044611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:15:10.133493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044873+0000","last_change":"2024-05-24T11:36:48.527805+0000","last_active":"2024-05-24T11:36:50.044873+0000","last_peered":"2024-05-24T11:36:50.044873+0000","last_clean":"2024-05-24T11:36:50.044873+0000","last_became_active":"2024-05-24T11:36:48.526047+0000","last_became_peered":"2024-05-24T11:36:48.526047+0000","last_unstale":"2024-05-24T11:36:50.044873+0000","last_undegraded":"2024-05-24T11:36:50.044873+0000","last_fullsized":"2024-05-24T11:36:50.044873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:20:56.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044374+0000","last_change":"2024-05-24T11:36:48.519295+0000","last_active":"2024-05-24T11:36:50.044374+0000","last_peered":"2024-05-24T11:36:50.044374+0000","last_clean":"2024-05-24T11:36:50.044374+0000","last_became_active":"2024-05-24T11:36:48.518955+0000","last_became_peered":"2024-05-24T11:36:48.518955+0000","last_unstale":"2024-05-24T11:36:50.044374+0000","last_undegraded":"2024-05-24T11:36:50.044374+0000","last_fullsized":"2024-05-24T11:36:50.044374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:39:14.437903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518558+0000","last_change":"2024-05-24T11:36:48.525324+0000","last_active":"2024-05-24T11:36:49.518558+0000","last_peered":"2024-05-24T11:36:49.518558+0000","last_clean":"2024-05-24T11:36:49.518558+0000","last_became_active":"2024-05-24T11:36:48.525045+0000","last_became_peered":"2024-05-24T11:36:48.525045+0000","last_unstale":"2024-05-24T11:36:49.518558+0000","last_undegraded":"2024-05-24T11:36:49.518558+0000","last_fullsized":"2024-05-24T11:36:49.518558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:19:52.341136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045101+0000","last_change":"2024-05-24T11:36:48.528095+0000","last_active":"2024-05-24T11:36:50.045101+0000","last_peered":"2024-05-24T11:36:50.045101+0000","last_clean":"2024-05-24T11:36:50.045101+0000","last_became_active":"2024-05-24T11:36:48.527814+0000","last_became_peered":"2024-05-24T11:36:48.527814+0000","last_unstale":"2024-05-24T11:36:50.045101+0000","last_undegraded":"2024-05-24T11:36:50.045101+0000","last_fullsized":"2024-05-24T11:36:50.045101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:12:42.591269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518210+0000","last_change":"2024-05-24T11:36:48.528775+0000","last_active":"2024-05-24T11:36:49.518210+0000","last_peered":"2024-05-24T11:36:49.518210+0000","last_clean":"2024-05-24T11:36:49.518210+0000","last_became_active":"2024-05-24T11:36:48.528467+0000","last_became_peered":"2024-05-24T11:36:48.528467+0000","last_unstale":"2024-05-24T11:36:49.518210+0000","last_undegraded":"2024-05-24T11:36:49.518210+0000","last_fullsized":"2024-05-24T11:36:49.518210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:10:59.444142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63965,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":370,"num_read_kb":257,"num_write":229,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645329,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":835584,"data_stored":399105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":7.0039999999999996,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.94199999999999995}]},{"osd":2,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.91200000000000003}]},{"osd":5,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.2679999999999998,"5min":3.2679999999999998,"15min":3.2679999999999998},"last":1.659}]},{"osd":7,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.88200000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":9,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":7.125,"5min":7.125,"15min":7.125},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.76900000000000002}]},{"osd":10,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.83499999999999996}]}]},{"osd":10,"up_from":65,"seq":279172874262,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40800,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704352,"statfs":{"total":95995035648,"available":95953256448,"internally_reserved":0,"allocated":1531904,"data_stored":1046696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":40237505},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.751},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.43099999999999999}]},{"osd":5,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.626}]},{"osd":6,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.46700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":8,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.70199999999999996}]},{"osd":9,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.58999999999999997}]},{"osd":11,"last update":"Fri May 24 11:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.77800000000000002}]}]},{"osd":9,"up_from":61,"seq":261993005081,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":860160,"data_stored":398690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.80000000000000004}]},{"osd":8,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.624},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Fri May 24 11:37:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.83799999999999997}]}]},{"osd":8,"up_from":54,"seq":231928234012,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":399068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.442},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.89800000000000002}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.85999999999999999}]},{"osd":10,"last update":"Fri May 24 11:37:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.91100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":7,"up_from":47,"seq":201863462944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1445888,"data_stored":990846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.438}]},{"osd":1,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47799999999999998,"15min":0.47199999999999998},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.42599999999999999}]},{"osd":2,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.47299999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.54400000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.47199999999999998}]},{"osd":4,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85999999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.498}]},{"osd":5,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.65400000000000003},"min":{"1min":0.60199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70499999999999996,"15min":0.70199999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.73999999999999999,"15min":0.72799999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.2330000000000001,"5min":3.2330000000000001,"15min":3.2330000000000001},"last":0.60099999999999998}]},{"osd":8,"last update":"Fri May 24 11:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.74099999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.89200000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.65400000000000003}]},{"osd":10,"last update":"Fri May 24 11:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":6,"up_from":43,"seq":184683593764,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31936,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713216,"statfs":{"total":95995035648,"available":95962333184,"internally_reserved":0,"allocated":843776,"data_stored":398365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.33100000000000002}]},{"osd":1,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.39700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73499999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.34899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71999999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45000000000000001,"15min":0.45900000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.623,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.35999999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68600000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89500000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80700000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.497}]},{"osd":5,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80800000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86099999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri May 24 11:37:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80700000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.65800000000000003,"15min":0.66700000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60399999999999998}]},{"osd":8,"last update":"Fri May 24 11:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69299999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.505,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73299999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.73299999999999998}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.46100000000000002}]},{"osd":10,"last update":"Fri May 24 11:37:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63400000000000001}]}]},{"osd":1,"up_from":12,"seq":51539607606,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40184,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704968,"statfs":{"total":95995035648,"available":95953887232,"internally_reserved":0,"allocated":901120,"data_stored":456221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.32600000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.93500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.497}]},{"osd":2,"last update":"Fri May 24 11:36:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.63}]},{"osd":3,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85499999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82399999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri May 24 11:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.50600000000000001,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.752},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri May 24 11:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.872},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri May 24 11:36:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88200000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.48699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85299999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri May 24 11:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.46700000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.56599999999999995,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.96099999999999997}]},{"osd":8,"last update":"Fri May 24 11:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.746,"15min":0.72699999999999998},"min":{"1min":0.621,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.59099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.82299999999999995}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.76600000000000001}]},{"osd":10,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.90500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":400091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 24 11:37:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.35899999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72099999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri May 24 11:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.45000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":1.069}]},{"osd":3,"last update":"Fri May 24 11:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.40600000000000003,"15min":0.39600000000000002},"min":{"1min":0.23799999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.40999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64700000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Fri May 24 11:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.47599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72199999999999998,"5min":0.755,"15min":0.755},"last":1.038},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.46999999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri May 24 11:36:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.32800000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Fri May 24 11:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55000000000000004}]},{"osd":8,"last update":"Fri May 24 11:37:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.93300000000000005}]},{"osd":9,"last update":"Fri May 24 11:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":1.137}]},{"osd":10,"last update":"Fri May 24 11:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.98399999999999999}]},{"osd":11,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.82499999999999996}]}]},{"osd":2,"up_from":17,"seq":73014444083,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":456287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.746,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.85699999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri May 24 11:36:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76100000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84899999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri May 24 11:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72599999999999998,"5min":0.746,"15min":0.746},"last":0.877},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.95899999999999996}]},{"osd":5,"last update":"Fri May 24 11:36:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.433,"15min":0.42399999999999999},"min":{"1min":0.313,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri May 24 11:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri May 24 11:37:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82999999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.82899999999999996}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.88800000000000001}]},{"osd":10,"last update":"Fri May 24 11:36:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":856064,"data_stored":397913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.28699999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.375,"15min":0.36599999999999999},"min":{"1min":0.308,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.435}]},{"osd":1,"last update":"Fri May 24 11:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.48099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri May 24 11:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.76600000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.622},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri May 24 11:37:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.41699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70799999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.40400000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84799999999999998,"5min":0.88,"15min":0.88},"last":0.501}]},{"osd":5,"last update":"Fri May 24 11:36:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84299999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.749}]},{"osd":6,"last update":"Fri May 24 11:36:52 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri May 24 11:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89900000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70699999999999996,"15min":0.71199999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91600000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.56399999999999995}]},{"osd":8,"last update":"Fri May 24 11:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.66900000000000004}]},{"osd":9,"last update":"Fri May 24 11:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.72099999999999997}]},{"osd":10,"last update":"Fri May 24 11:36:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.60999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953515,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":398220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.502,"15min":0.495},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.37,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri May 24 11:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40100000000000002,"15min":0.38400000000000001},"min":{"1min":0.33100000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.40699999999999997},"min":{"1min":0.372,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60199999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri May 24 11:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.61099999999999999,"15min":0.58699999999999997},"min":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77800000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53500000000000003}]},{"osd":3,"last update":"Fri May 24 11:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.57399999999999995},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89300000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.438,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri May 24 11:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.874,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri May 24 11:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61599999999999999,"15min":0.60299999999999998},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.433,"5min":0.378,"15min":0.378},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri May 24 11:37:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.379,"15min":0.379},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.41999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66200000000000003}]},{"osd":8,"last update":"Fri May 24 11:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76000000000000001}]},{"osd":9,"last update":"Fri May 24 11:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.70499999999999996}]},{"osd":10,"last update":"Fri May 24 11:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.82299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822696,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32536,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712616,"statfs":{"total":95995035648,"available":95961718784,"internally_reserved":0,"allocated":1458176,"data_stored":990620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76100000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.28499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.51300000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.40500000000000003,"15min":0.40899999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.45200000000000001}]},{"osd":3,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.441,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri May 24 11:36:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.52200000000000002}]},{"osd":7,"last update":"Fri May 24 11:37:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.51100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.755}]},{"osd":8,"last update":"Fri May 24 11:37:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.70099999999999996}]},{"osd":9,"last update":"Fri May 24 11:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.67600000000000005}]},{"osd":10,"last update":"Fri May 24 11:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.78600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-24T11:37:31.912 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-24T11:37:32.145 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:32.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:31 smithi053 ceph-mon[47448]: pgmap v241: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:37:32.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:31 smithi137 ceph-mon[47907]: pgmap v241: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:37:32.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:32 smithi039 ceph-mon[42919]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:37:33.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.198+0000 7f47fb50b700 1 -- 172.21.15.39:0/866614961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4101990 msgr2=0x7f47f410de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:33.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.198+0000 7f47fb50b700 1 --2- 172.21.15.39:0/866614961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4101990 0x7f47f410de70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00b0a0 tx=0x7f47ec00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.198+0000 7f47fb50b700 1 -- 172.21.15.39:0/866614961 shutdown_connections 2024-05-24T11:37:33.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.198+0000 7f47fb50b700 1 --2- 172.21.15.39:0/866614961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f4107030 0x7f47f4107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.198+0000 7f47fb50b700 1 --2- 172.21.15.39:0/866614961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4101990 0x7f47f410de70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.198+0000 7f47fb50b700 1 --2- 172.21.15.39:0/866614961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100ff0 0x7f47f4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.198+0000 7f47fb50b700 1 -- 172.21.15.39:0/866614961 >> 172.21.15.39:0/866614961 conn(0x7f47f4078190 msgr2=0x7f47f40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:33.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.198+0000 7f47fb50b700 1 -- 172.21.15.39:0/866614961 shutdown_connections 2024-05-24T11:37:33.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.198+0000 7f47fb50b700 1 -- 172.21.15.39:0/866614961 wait complete. 2024-05-24T11:37:33.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.199+0000 7f47fb50b700 1 Processor -- start 2024-05-24T11:37:33.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.199+0000 7f47fb50b700 1 -- start start 2024-05-24T11:37:33.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.199+0000 7f47fb50b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100ff0 0x7f47f41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:33.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.199+0000 7f47fb50b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f4101990 0x7f47f4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:33.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.199+0000 7f47fb50b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4107030 0x7f47f419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:33.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.199+0000 7f47fb50b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f419c3d0 con 0x7f47f4107030 2024-05-24T11:37:33.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.199+0000 7f47fb50b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f419c540 con 0x7f47f4100ff0 2024-05-24T11:37:33.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.199+0000 7f47fb50b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f419c700 con 0x7f47f4101990 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f9aa8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4107030 0x7f47f419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f9aa8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4107030 0x7f47f419bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53116/0 (socket says 172.21.15.39:53116) 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f9aa8700 1 -- 172.21.15.39:0/626655541 learned_addr learned my addr 172.21.15.39:0/626655541 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f92a7700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100ff0 0x7f47f41974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f8aa6700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f4101990 0x7f47f4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f9aa8700 1 -- 172.21.15.39:0/626655541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f4101990 msgr2=0x7f47f4197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f9aa8700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f4101990 0x7f47f4197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f9aa8700 1 -- 172.21.15.39:0/626655541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100ff0 msgr2=0x7f47f41974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f9aa8700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100ff0 0x7f47f41974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f9aa8700 1 -- 172.21.15.39:0/626655541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec009d00 con 0x7f47f4107030 2024-05-24T11:37:33.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f8aa6700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f4101990 0x7f47f4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:33.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47f9aa8700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4107030 0x7f47f419bdc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00b830 tx=0x7f47ec009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:33.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47ea7fc700 1 -- 172.21.15.39:0/626655541 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00e040 con 0x7f47f4107030 2024-05-24T11:37:33.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47ea7fc700 1 -- 172.21.15.39:0/626655541 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47ec007930 con 0x7f47f4107030 2024-05-24T11:37:33.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f419c980 con 0x7f47f4107030 2024-05-24T11:37:33.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47ea7fc700 1 -- 172.21.15.39:0/626655541 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec005280 con 0x7f47f4107030 2024-05-24T11:37:33.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.200+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f41a4240 con 0x7f47f4107030 2024-05-24T11:37:33.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.201+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f4109550 con 0x7f47f4107030 2024-05-24T11:37:33.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.202+0000 7f47ea7fc700 1 -- 172.21.15.39:0/626655541 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f47ec01b9e0 con 0x7f47f4107030 2024-05-24T11:37:33.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.203+0000 7f47ea7fc700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47e00a1230 0x7f47e00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:33.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.203+0000 7f47f92a7700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47e00a1230 0x7f47e00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:33.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.203+0000 7f47ea7fc700 1 -- 172.21.15.39:0/626655541 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f47ec0ce750 con 0x7f47f4107030 2024-05-24T11:37:33.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.204+0000 7f47f92a7700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47e00a1230 0x7f47e00a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009a10 tx=0x7f47e4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:33.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.206+0000 7f47ea7fc700 1 -- 172.21.15.39:0/626655541 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f47ec097fc0 con 0x7f47f4107030 2024-05-24T11:37:33.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:32 smithi053 ceph-mon[47448]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:37:33.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:32 smithi137 ceph-mon[47907]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:37:33.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.404+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47f4198850 con 0x7f47e00a1230 2024-05-24T11:37:33.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.415+0000 7f47ea7fc700 1 -- 172.21.15.39:0/626655541 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340868 (secure 0 0 0) 0x7f47f4198850 con 0x7f47e00a1230 2024-05-24T11:37:33.424 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47e00a1230 msgr2=0x7f47e00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47e00a1230 0x7f47e00a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009a10 tx=0x7f47e4009190 comp rx=0 tx=0).stop 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4107030 msgr2=0x7f47f419bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4107030 0x7f47f419bdc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00b830 tx=0x7f47ec009380 comp rx=0 tx=0).stop 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 shutdown_connections 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f47e00a1230 0x7f47e00a36f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f4107030 0x7f47f419bdc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100ff0 0x7f47f41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 --2- 172.21.15.39:0/626655541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f4101990 0x7f47f4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 >> 172.21.15.39:0/626655541 conn(0x7f47f4078190 msgr2=0x7f47f40fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 shutdown_connections 2024-05-24T11:37:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:33.418+0000 7f47fb50b700 1 -- 172.21.15.39:0/626655541 wait complete. 2024-05-24T11:37:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-24T11:37:33.832 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-05-24T11:37:31.885801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654579,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":526,"num_read_kb":386,"num_write":412,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381172,"kb_used_data":11924,"kb_used_omap":18,"kb_used_meta":369133,"kb_avail":1124560652,"statfs":{"total":1151940427776,"available":1151550107648,"internally_reserved":0,"allocated":12210176,"data_stored":6732122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":377992517},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518423+0000","last_change":"2024-05-24T11:36:48.521973+0000","last_active":"2024-05-24T11:36:49.518423+0000","last_peered":"2024-05-24T11:36:49.518423+0000","last_clean":"2024-05-24T11:36:49.518423+0000","last_became_active":"2024-05-24T11:36:48.521587+0000","last_became_peered":"2024-05-24T11:36:48.521587+0000","last_unstale":"2024-05-24T11:36:49.518423+0000","last_undegraded":"2024-05-24T11:36:49.518423+0000","last_fullsized":"2024-05-24T11:36:49.518423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:25:29.234714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811230+0000","last_change":"2024-05-24T11:36:48.536290+0000","last_active":"2024-05-24T11:36:49.811230+0000","last_peered":"2024-05-24T11:36:49.811230+0000","last_clean":"2024-05-24T11:36:49.811230+0000","last_became_active":"2024-05-24T11:36:48.535961+0000","last_became_peered":"2024-05-24T11:36:48.535961+0000","last_unstale":"2024-05-24T11:36:49.811230+0000","last_undegraded":"2024-05-24T11:36:49.811230+0000","last_fullsized":"2024-05-24T11:36:49.811230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:52:37.951507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811216+0000","last_change":"2024-05-24T11:36:48.520039+0000","last_active":"2024-05-24T11:36:49.811216+0000","last_peered":"2024-05-24T11:36:49.811216+0000","last_clean":"2024-05-24T11:36:49.811216+0000","last_became_active":"2024-05-24T11:36:48.519809+0000","last_became_peered":"2024-05-24T11:36:48.519809+0000","last_unstale":"2024-05-24T11:36:49.811216+0000","last_undegraded":"2024-05-24T11:36:49.811216+0000","last_fullsized":"2024-05-24T11:36:49.811216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:44:20.133334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045745+0000","last_change":"2024-05-24T11:36:48.528100+0000","last_active":"2024-05-24T11:36:50.045745+0000","last_peered":"2024-05-24T11:36:50.045745+0000","last_clean":"2024-05-24T11:36:50.045745+0000","last_became_active":"2024-05-24T11:36:48.524918+0000","last_became_peered":"2024-05-24T11:36:48.524918+0000","last_unstale":"2024-05-24T11:36:50.045745+0000","last_undegraded":"2024-05-24T11:36:50.045745+0000","last_fullsized":"2024-05-24T11:36:50.045745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:20:34.628252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044676+0000","last_change":"2024-05-24T11:36:48.522735+0000","last_active":"2024-05-24T11:36:50.044676+0000","last_peered":"2024-05-24T11:36:50.044676+0000","last_clean":"2024-05-24T11:36:50.044676+0000","last_became_active":"2024-05-24T11:36:48.521657+0000","last_became_peered":"2024-05-24T11:36:48.521657+0000","last_unstale":"2024-05-24T11:36:50.044676+0000","last_undegraded":"2024-05-24T11:36:50.044676+0000","last_fullsized":"2024-05-24T11:36:50.044676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:03:32.056035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.835970+0000","last_change":"2024-05-24T11:36:48.534898+0000","last_active":"2024-05-24T11:36:48.835970+0000","last_peered":"2024-05-24T11:36:48.835970+0000","last_clean":"2024-05-24T11:36:48.835970+0000","last_became_active":"2024-05-24T11:36:48.534557+0000","last_became_peered":"2024-05-24T11:36:48.534557+0000","last_unstale":"2024-05-24T11:36:48.835970+0000","last_undegraded":"2024-05-24T11:36:48.835970+0000","last_fullsized":"2024-05-24T11:36:48.835970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:59:23.847279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.533394+0000","last_change":"2024-05-24T11:36:48.532435+0000","last_active":"2024-05-24T11:36:48.533394+0000","last_peered":"2024-05-24T11:36:48.533394+0000","last_clean":"2024-05-24T11:36:48.533394+0000","last_became_active":"2024-05-24T11:36:48.532058+0000","last_became_peered":"2024-05-24T11:36:48.532058+0000","last_unstale":"2024-05-24T11:36:48.533394+0000","last_undegraded":"2024-05-24T11:36:48.533394+0000","last_fullsized":"2024-05-24T11:36:48.533394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:33:26.178124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044447+0000","last_change":"2024-05-24T11:36:48.519054+0000","last_active":"2024-05-24T11:36:50.044447+0000","last_peered":"2024-05-24T11:36:50.044447+0000","last_clean":"2024-05-24T11:36:50.044447+0000","last_became_active":"2024-05-24T11:36:48.518756+0000","last_became_peered":"2024-05-24T11:36:48.518756+0000","last_unstale":"2024-05-24T11:36:50.044447+0000","last_undegraded":"2024-05-24T11:36:50.044447+0000","last_fullsized":"2024-05-24T11:36:50.044447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:56:19.599232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518761+0000","last_change":"2024-05-24T11:36:48.522937+0000","last_active":"2024-05-24T11:36:49.518761+0000","last_peered":"2024-05-24T11:36:49.518761+0000","last_clean":"2024-05-24T11:36:49.518761+0000","last_became_active":"2024-05-24T11:36:48.521531+0000","last_became_peered":"2024-05-24T11:36:48.521531+0000","last_unstale":"2024-05-24T11:36:49.518761+0000","last_undegraded":"2024-05-24T11:36:49.518761+0000","last_fullsized":"2024-05-24T11:36:49.518761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:16:54.392477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.836050+0000","last_change":"2024-05-24T11:36:48.536738+0000","last_active":"2024-05-24T11:36:48.836050+0000","last_peered":"2024-05-24T11:36:48.836050+0000","last_clean":"2024-05-24T11:36:48.836050+0000","last_became_active":"2024-05-24T11:36:48.536010+0000","last_became_peered":"2024-05-24T11:36:48.536010+0000","last_unstale":"2024-05-24T11:36:48.836050+0000","last_undegraded":"2024-05-24T11:36:48.836050+0000","last_fullsized":"2024-05-24T11:36:48.836050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:21:24.078676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.536403+0000","last_change":"2024-05-24T11:36:48.535036+0000","last_active":"2024-05-24T11:36:48.536403+0000","last_peered":"2024-05-24T11:36:48.536403+0000","last_clean":"2024-05-24T11:36:48.536403+0000","last_became_active":"2024-05-24T11:36:48.534232+0000","last_became_peered":"2024-05-24T11:36:48.534232+0000","last_unstale":"2024-05-24T11:36:48.536403+0000","last_undegraded":"2024-05-24T11:36:48.536403+0000","last_fullsized":"2024-05-24T11:36:48.536403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:56:28.111680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045685+0000","last_change":"2024-05-24T11:36:48.537837+0000","last_active":"2024-05-24T11:36:50.045685+0000","last_peered":"2024-05-24T11:36:50.045685+0000","last_clean":"2024-05-24T11:36:50.045685+0000","last_became_active":"2024-05-24T11:36:48.537570+0000","last_became_peered":"2024-05-24T11:36:48.537570+0000","last_unstale":"2024-05-24T11:36:50.045685+0000","last_undegraded":"2024-05-24T11:36:50.045685+0000","last_fullsized":"2024-05-24T11:36:50.045685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:47:14.769120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045004+0000","last_change":"2024-05-24T11:36:48.525333+0000","last_active":"2024-05-24T11:36:50.045004+0000","last_peered":"2024-05-24T11:36:50.045004+0000","last_clean":"2024-05-24T11:36:50.045004+0000","last_became_active":"2024-05-24T11:36:48.525146+0000","last_became_peered":"2024-05-24T11:36:48.525146+0000","last_unstale":"2024-05-24T11:36:50.045004+0000","last_undegraded":"2024-05-24T11:36:50.045004+0000","last_fullsized":"2024-05-24T11:36:50.045004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:22:58.950175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811013+0000","last_change":"2024-05-24T11:36:48.525438+0000","last_active":"2024-05-24T11:36:49.811013+0000","last_peered":"2024-05-24T11:36:49.811013+0000","last_clean":"2024-05-24T11:36:49.811013+0000","last_became_active":"2024-05-24T11:36:48.524866+0000","last_became_peered":"2024-05-24T11:36:48.524866+0000","last_unstale":"2024-05-24T11:36:49.811013+0000","last_undegraded":"2024-05-24T11:36:49.811013+0000","last_fullsized":"2024-05-24T11:36:49.811013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:39:11.826673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.810917+0000","last_change":"2024-05-24T11:36:48.529993+0000","last_active":"2024-05-24T11:36:49.810917+0000","last_peered":"2024-05-24T11:36:49.810917+0000","last_clean":"2024-05-24T11:36:49.810917+0000","last_became_active":"2024-05-24T11:36:48.529644+0000","last_became_peered":"2024-05-24T11:36:48.529644+0000","last_unstale":"2024-05-24T11:36:49.810917+0000","last_undegraded":"2024-05-24T11:36:49.810917+0000","last_fullsized":"2024-05-24T11:36:49.810917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:00:02.580642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518315+0000","last_change":"2024-05-24T11:36:48.521904+0000","last_active":"2024-05-24T11:36:49.518315+0000","last_peered":"2024-05-24T11:36:49.518315+0000","last_clean":"2024-05-24T11:36:49.518315+0000","last_became_active":"2024-05-24T11:36:48.521281+0000","last_became_peered":"2024-05-24T11:36:48.521281+0000","last_unstale":"2024-05-24T11:36:49.518315+0000","last_undegraded":"2024-05-24T11:36:49.518315+0000","last_fullsized":"2024-05-24T11:36:49.518315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:16:32.578404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518556+0000","last_change":"2024-05-24T11:36:48.526737+0000","last_active":"2024-05-24T11:36:49.518556+0000","last_peered":"2024-05-24T11:36:49.518556+0000","last_clean":"2024-05-24T11:36:49.518556+0000","last_became_active":"2024-05-24T11:36:48.526379+0000","last_became_peered":"2024-05-24T11:36:48.526379+0000","last_unstale":"2024-05-24T11:36:49.518556+0000","last_undegraded":"2024-05-24T11:36:49.518556+0000","last_fullsized":"2024-05-24T11:36:49.518556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:32:19.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.522182+0000","last_change":"2024-05-24T11:36:48.529315+0000","last_active":"2024-05-24T11:36:49.522182+0000","last_peered":"2024-05-24T11:36:49.522182+0000","last_clean":"2024-05-24T11:36:49.522182+0000","last_became_active":"2024-05-24T11:36:48.528963+0000","last_became_peered":"2024-05-24T11:36:48.528963+0000","last_unstale":"2024-05-24T11:36:49.522182+0000","last_undegraded":"2024-05-24T11:36:49.522182+0000","last_fullsized":"2024-05-24T11:36:49.522182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:54:23.922811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.544120+0000","last_change":"2024-05-24T11:36:48.520925+0000","last_active":"2024-05-24T11:36:49.544120+0000","last_peered":"2024-05-24T11:36:49.544120+0000","last_clean":"2024-05-24T11:36:49.544120+0000","last_became_active":"2024-05-24T11:36:48.520592+0000","last_became_peered":"2024-05-24T11:36:48.520592+0000","last_unstale":"2024-05-24T11:36:49.544120+0000","last_undegraded":"2024-05-24T11:36:49.544120+0000","last_fullsized":"2024-05-24T11:36:49.544120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:39:09.918746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"77'13","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:29.639198+0000","last_change":"2024-05-24T11:36:48.090815+0000","last_active":"2024-05-24T11:37:29.639198+0000","last_peered":"2024-05-24T11:37:29.639198+0000","last_clean":"2024-05-24T11:37:29.639198+0000","last_became_active":"2024-05-24T11:36:48.090481+0000","last_became_peered":"2024-05-24T11:36:48.090481+0000","last_unstale":"2024-05-24T11:37:29.639198+0000","last_undegraded":"2024-05-24T11:37:29.639198+0000","last_fullsized":"2024-05-24T11:37:29.639198+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:32:40.850290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044588+0000","last_change":"2024-05-24T11:36:48.519300+0000","last_active":"2024-05-24T11:36:50.044588+0000","last_peered":"2024-05-24T11:36:50.044588+0000","last_clean":"2024-05-24T11:36:50.044588+0000","last_became_active":"2024-05-24T11:36:48.519004+0000","last_became_peered":"2024-05-24T11:36:48.519004+0000","last_unstale":"2024-05-24T11:36:50.044588+0000","last_undegraded":"2024-05-24T11:36:50.044588+0000","last_fullsized":"2024-05-24T11:36:50.044588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:15:58.098701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.377448+0000","last_change":"2024-05-24T11:36:48.091695+0000","last_active":"2024-05-24T11:36:51.377448+0000","last_peered":"2024-05-24T11:36:51.377448+0000","last_clean":"2024-05-24T11:36:51.377448+0000","last_became_active":"2024-05-24T11:36:48.091379+0000","last_became_peered":"2024-05-24T11:36:48.091379+0000","last_unstale":"2024-05-24T11:36:51.377448+0000","last_undegraded":"2024-05-24T11:36:51.377448+0000","last_fullsized":"2024-05-24T11:36:51.377448+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:07:55.039339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811338+0000","last_change":"2024-05-24T11:36:48.519313+0000","last_active":"2024-05-24T11:36:49.811338+0000","last_peered":"2024-05-24T11:36:49.811338+0000","last_clean":"2024-05-24T11:36:49.811338+0000","last_became_active":"2024-05-24T11:36:48.518968+0000","last_became_peered":"2024-05-24T11:36:48.518968+0000","last_unstale":"2024-05-24T11:36:49.811338+0000","last_undegraded":"2024-05-24T11:36:49.811338+0000","last_fullsized":"2024-05-24T11:36:49.811338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:51:23.712853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"77'12","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591747+0000","last_change":"2024-05-24T11:36:48.096014+0000","last_active":"2024-05-24T11:36:54.591747+0000","last_peered":"2024-05-24T11:36:54.591747+0000","last_clean":"2024-05-24T11:36:54.591747+0000","last_became_active":"2024-05-24T11:36:48.095261+0000","last_became_peered":"2024-05-24T11:36:48.095261+0000","last_unstale":"2024-05-24T11:36:54.591747+0000","last_undegraded":"2024-05-24T11:36:54.591747+0000","last_fullsized":"2024-05-24T11:36:54.591747+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:19:32.675770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518671+0000","last_change":"2024-05-24T11:36:48.524464+0000","last_active":"2024-05-24T11:36:49.518671+0000","last_peered":"2024-05-24T11:36:49.518671+0000","last_clean":"2024-05-24T11:36:49.518671+0000","last_became_active":"2024-05-24T11:36:48.524128+0000","last_became_peered":"2024-05-24T11:36:48.524128+0000","last_unstale":"2024-05-24T11:36:49.518671+0000","last_undegraded":"2024-05-24T11:36:49.518671+0000","last_fullsized":"2024-05-24T11:36:49.518671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:57:21.920169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'5","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591236+0000","last_change":"2024-05-24T11:36:48.095739+0000","last_active":"2024-05-24T11:36:54.591236+0000","last_peered":"2024-05-24T11:36:54.591236+0000","last_clean":"2024-05-24T11:36:54.591236+0000","last_became_active":"2024-05-24T11:36:48.095313+0000","last_became_peered":"2024-05-24T11:36:48.095313+0000","last_unstale":"2024-05-24T11:36:54.591236+0000","last_undegraded":"2024-05-24T11:36:54.591236+0000","last_fullsized":"2024-05-24T11:36:54.591236+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:59:55.283423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045238+0000","last_change":"2024-05-24T11:36:48.528788+0000","last_active":"2024-05-24T11:36:50.045238+0000","last_peered":"2024-05-24T11:36:50.045238+0000","last_clean":"2024-05-24T11:36:50.045238+0000","last_became_active":"2024-05-24T11:36:48.528358+0000","last_became_peered":"2024-05-24T11:36:48.528358+0000","last_unstale":"2024-05-24T11:36:50.045238+0000","last_undegraded":"2024-05-24T11:36:50.045238+0000","last_fullsized":"2024-05-24T11:36:50.045238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:19:37.870208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"77'6","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:30.635279+0000","last_change":"2024-05-24T11:36:48.093252+0000","last_active":"2024-05-24T11:37:30.635279+0000","last_peered":"2024-05-24T11:37:30.635279+0000","last_clean":"2024-05-24T11:37:30.635279+0000","last_became_active":"2024-05-24T11:36:48.092920+0000","last_became_peered":"2024-05-24T11:36:48.092920+0000","last_unstale":"2024-05-24T11:37:30.635279+0000","last_undegraded":"2024-05-24T11:37:30.635279+0000","last_fullsized":"2024-05-24T11:37:30.635279+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:51:39.032589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044926+0000","last_change":"2024-05-24T11:36:48.518108+0000","last_active":"2024-05-24T11:36:50.044926+0000","last_peered":"2024-05-24T11:36:50.044926+0000","last_clean":"2024-05-24T11:36:50.044926+0000","last_became_active":"2024-05-24T11:36:48.517786+0000","last_became_peered":"2024-05-24T11:36:48.517786+0000","last_unstale":"2024-05-24T11:36:50.044926+0000","last_undegraded":"2024-05-24T11:36:50.044926+0000","last_fullsized":"2024-05-24T11:36:50.044926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:41:44.779354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:53.618566+0000","last_change":"2024-05-24T11:36:48.082348+0000","last_active":"2024-05-24T11:36:53.618566+0000","last_peered":"2024-05-24T11:36:53.618566+0000","last_clean":"2024-05-24T11:36:53.618566+0000","last_became_active":"2024-05-24T11:36:48.081964+0000","last_became_peered":"2024-05-24T11:36:48.081964+0000","last_unstale":"2024-05-24T11:36:53.618566+0000","last_undegraded":"2024-05-24T11:36:53.618566+0000","last_fullsized":"2024-05-24T11:36:53.618566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:45:54.028437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045904+0000","last_change":"2024-05-24T11:36:48.528990+0000","last_active":"2024-05-24T11:36:50.045904+0000","last_peered":"2024-05-24T11:36:50.045904+0000","last_clean":"2024-05-24T11:36:50.045904+0000","last_became_active":"2024-05-24T11:36:48.528444+0000","last_became_peered":"2024-05-24T11:36:48.528444+0000","last_unstale":"2024-05-24T11:36:50.045904+0000","last_undegraded":"2024-05-24T11:36:50.045904+0000","last_fullsized":"2024-05-24T11:36:50.045904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:35:56.108817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.610350+0000","last_change":"2024-05-24T11:36:48.079471+0000","last_active":"2024-05-24T11:36:52.610350+0000","last_peered":"2024-05-24T11:36:52.610350+0000","last_clean":"2024-05-24T11:36:52.610350+0000","last_became_active":"2024-05-24T11:36:48.079118+0000","last_became_peered":"2024-05-24T11:36:48.079118+0000","last_unstale":"2024-05-24T11:36:52.610350+0000","last_undegraded":"2024-05-24T11:36:52.610350+0000","last_fullsized":"2024-05-24T11:36:52.610350+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:01:35.897437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044314+0000","last_change":"2024-05-24T11:36:48.517960+0000","last_active":"2024-05-24T11:36:50.044314+0000","last_peered":"2024-05-24T11:36:50.044314+0000","last_clean":"2024-05-24T11:36:50.044314+0000","last_became_active":"2024-05-24T11:36:48.517675+0000","last_became_peered":"2024-05-24T11:36:48.517675+0000","last_unstale":"2024-05-24T11:36:50.044314+0000","last_undegraded":"2024-05-24T11:36:50.044314+0000","last_fullsized":"2024-05-24T11:36:50.044314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:08:21.209080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"77'3","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.582937+0000","last_change":"2024-05-24T11:36:48.091922+0000","last_active":"2024-05-24T11:36:54.582937+0000","last_peered":"2024-05-24T11:36:54.582937+0000","last_clean":"2024-05-24T11:36:54.582937+0000","last_became_active":"2024-05-24T11:36:48.091576+0000","last_became_peered":"2024-05-24T11:36:48.091576+0000","last_unstale":"2024-05-24T11:36:54.582937+0000","last_undegraded":"2024-05-24T11:36:54.582937+0000","last_fullsized":"2024-05-24T11:36:54.582937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:01:39.386649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.521890+0000","last_change":"2024-05-24T11:36:48.523769+0000","last_active":"2024-05-24T11:36:49.521890+0000","last_peered":"2024-05-24T11:36:49.521890+0000","last_clean":"2024-05-24T11:36:49.521890+0000","last_became_active":"2024-05-24T11:36:48.523394+0000","last_became_peered":"2024-05-24T11:36:48.523394+0000","last_unstale":"2024-05-24T11:36:49.521890+0000","last_undegraded":"2024-05-24T11:36:49.521890+0000","last_fullsized":"2024-05-24T11:36:49.521890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:03:49.209246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":86,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:28.631803+0000","last_change":"2024-05-24T11:36:48.089651+0000","last_active":"2024-05-24T11:37:28.631803+0000","last_peered":"2024-05-24T11:37:28.631803+0000","last_clean":"2024-05-24T11:37:28.631803+0000","last_became_active":"2024-05-24T11:36:48.089195+0000","last_became_peered":"2024-05-24T11:36:48.089195+0000","last_unstale":"2024-05-24T11:37:28.631803+0000","last_undegraded":"2024-05-24T11:37:28.631803+0000","last_fullsized":"2024-05-24T11:37:28.631803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:30:47.730594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518732+0000","last_change":"2024-05-24T11:36:48.531529+0000","last_active":"2024-05-24T11:36:49.518732+0000","last_peered":"2024-05-24T11:36:49.518732+0000","last_clean":"2024-05-24T11:36:49.518732+0000","last_became_active":"2024-05-24T11:36:48.530969+0000","last_became_peered":"2024-05-24T11:36:48.530969+0000","last_unstale":"2024-05-24T11:36:49.518732+0000","last_undegraded":"2024-05-24T11:36:49.518732+0000","last_fullsized":"2024-05-24T11:36:49.518732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:05:24.929278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591832+0000","last_change":"2024-05-24T11:36:48.094249+0000","last_active":"2024-05-24T11:36:54.591832+0000","last_peered":"2024-05-24T11:36:54.591832+0000","last_clean":"2024-05-24T11:36:54.591832+0000","last_became_active":"2024-05-24T11:36:48.093835+0000","last_became_peered":"2024-05-24T11:36:48.093835+0000","last_unstale":"2024-05-24T11:36:54.591832+0000","last_undegraded":"2024-05-24T11:36:54.591832+0000","last_fullsized":"2024-05-24T11:36:54.591832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:41:47.660513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811338+0000","last_change":"2024-05-24T11:36:48.531745+0000","last_active":"2024-05-24T11:36:49.811338+0000","last_peered":"2024-05-24T11:36:49.811338+0000","last_clean":"2024-05-24T11:36:49.811338+0000","last_became_active":"2024-05-24T11:36:48.531459+0000","last_became_peered":"2024-05-24T11:36:48.531459+0000","last_unstale":"2024-05-24T11:36:49.811338+0000","last_undegraded":"2024-05-24T11:36:49.811338+0000","last_fullsized":"2024-05-24T11:36:49.811338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:06:41.008780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"77'5","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:53.617054+0000","last_change":"2024-05-24T11:36:48.089567+0000","last_active":"2024-05-24T11:36:53.617054+0000","last_peered":"2024-05-24T11:36:53.617054+0000","last_clean":"2024-05-24T11:36:53.617054+0000","last_became_active":"2024-05-24T11:36:48.089248+0000","last_became_peered":"2024-05-24T11:36:48.089248+0000","last_unstale":"2024-05-24T11:36:53.617054+0000","last_undegraded":"2024-05-24T11:36:53.617054+0000","last_fullsized":"2024-05-24T11:36:53.617054+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:49:06.307799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518786+0000","last_change":"2024-05-24T11:36:48.529531+0000","last_active":"2024-05-24T11:36:49.518786+0000","last_peered":"2024-05-24T11:36:49.518786+0000","last_clean":"2024-05-24T11:36:49.518786+0000","last_became_active":"2024-05-24T11:36:48.529222+0000","last_became_peered":"2024-05-24T11:36:48.529222+0000","last_unstale":"2024-05-24T11:36:49.518786+0000","last_undegraded":"2024-05-24T11:36:49.518786+0000","last_fullsized":"2024-05-24T11:36:49.518786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:18:49.571222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'28","reported_seq":116,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:28.648510+0000","last_change":"2024-05-24T11:36:48.086123+0000","last_active":"2024-05-24T11:37:28.648510+0000","last_peered":"2024-05-24T11:37:28.648510+0000","last_clean":"2024-05-24T11:37:28.648510+0000","last_became_active":"2024-05-24T11:36:48.085681+0000","last_became_peered":"2024-05-24T11:36:48.085681+0000","last_unstale":"2024-05-24T11:37:28.648510+0000","last_undegraded":"2024-05-24T11:37:28.648510+0000","last_fullsized":"2024-05-24T11:37:28.648510+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:17:12.646332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56949,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":94,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045057+0000","last_change":"2024-05-24T11:36:48.524883+0000","last_active":"2024-05-24T11:36:50.045057+0000","last_peered":"2024-05-24T11:36:50.045057+0000","last_clean":"2024-05-24T11:36:50.045057+0000","last_became_active":"2024-05-24T11:36:48.524578+0000","last_became_peered":"2024-05-24T11:36:48.524578+0000","last_unstale":"2024-05-24T11:36:50.045057+0000","last_undegraded":"2024-05-24T11:36:50.045057+0000","last_fullsized":"2024-05-24T11:36:50.045057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:54:15.419099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"77'9","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591054+0000","last_change":"2024-05-24T11:36:48.090437+0000","last_active":"2024-05-24T11:36:54.591054+0000","last_peered":"2024-05-24T11:36:54.591054+0000","last_clean":"2024-05-24T11:36:54.591054+0000","last_became_active":"2024-05-24T11:36:48.088745+0000","last_became_peered":"2024-05-24T11:36:48.088745+0000","last_unstale":"2024-05-24T11:36:54.591054+0000","last_undegraded":"2024-05-24T11:36:54.591054+0000","last_fullsized":"2024-05-24T11:36:54.591054+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:12:48.829945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044821+0000","last_change":"2024-05-24T11:36:48.525723+0000","last_active":"2024-05-24T11:36:50.044821+0000","last_peered":"2024-05-24T11:36:50.044821+0000","last_clean":"2024-05-24T11:36:50.044821+0000","last_became_active":"2024-05-24T11:36:48.525267+0000","last_became_peered":"2024-05-24T11:36:48.525267+0000","last_unstale":"2024-05-24T11:36:50.044821+0000","last_undegraded":"2024-05-24T11:36:50.044821+0000","last_fullsized":"2024-05-24T11:36:50.044821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:27:59.404214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"77'5","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.590042+0000","last_change":"2024-05-24T11:36:48.094426+0000","last_active":"2024-05-24T11:36:54.590042+0000","last_peered":"2024-05-24T11:36:54.590042+0000","last_clean":"2024-05-24T11:36:54.590042+0000","last_became_active":"2024-05-24T11:36:48.094169+0000","last_became_peered":"2024-05-24T11:36:48.094169+0000","last_unstale":"2024-05-24T11:36:54.590042+0000","last_undegraded":"2024-05-24T11:36:54.590042+0000","last_fullsized":"2024-05-24T11:36:54.590042+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:31:02.682818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.531031+0000","last_change":"2024-05-24T11:36:48.529851+0000","last_active":"2024-05-24T11:36:48.531031+0000","last_peered":"2024-05-24T11:36:48.531031+0000","last_clean":"2024-05-24T11:36:48.531031+0000","last_became_active":"2024-05-24T11:36:48.529540+0000","last_became_peered":"2024-05-24T11:36:48.529540+0000","last_unstale":"2024-05-24T11:36:48.531031+0000","last_undegraded":"2024-05-24T11:36:48.531031+0000","last_fullsized":"2024-05-24T11:36:48.531031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:22:12.101608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'5","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:28.645770+0000","last_change":"2024-05-24T11:36:48.090611+0000","last_active":"2024-05-24T11:37:28.645770+0000","last_peered":"2024-05-24T11:37:28.645770+0000","last_clean":"2024-05-24T11:37:28.645770+0000","last_became_active":"2024-05-24T11:36:48.090283+0000","last_became_peered":"2024-05-24T11:36:48.090283+0000","last_unstale":"2024-05-24T11:37:28.645770+0000","last_undegraded":"2024-05-24T11:37:28.645770+0000","last_fullsized":"2024-05-24T11:37:28.645770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:24:57.272371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.537097+0000","last_change":"2024-05-24T11:36:48.536763+0000","last_active":"2024-05-24T11:36:48.537097+0000","last_peered":"2024-05-24T11:36:48.537097+0000","last_clean":"2024-05-24T11:36:48.537097+0000","last_became_active":"2024-05-24T11:36:48.536398+0000","last_became_peered":"2024-05-24T11:36:48.536398+0000","last_unstale":"2024-05-24T11:36:48.537097+0000","last_undegraded":"2024-05-24T11:36:48.537097+0000","last_fullsized":"2024-05-24T11:36:48.537097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:13:58.446126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"77'17","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.581083+0000","last_change":"2024-05-24T11:36:48.080590+0000","last_active":"2024-05-24T11:36:54.581083+0000","last_peered":"2024-05-24T11:36:54.581083+0000","last_clean":"2024-05-24T11:36:54.581083+0000","last_became_active":"2024-05-24T11:36:48.080192+0000","last_became_peered":"2024-05-24T11:36:48.080192+0000","last_unstale":"2024-05-24T11:36:54.581083+0000","last_undegraded":"2024-05-24T11:36:54.581083+0000","last_fullsized":"2024-05-24T11:36:54.581083+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:13:35.452878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044986+0000","last_change":"2024-05-24T11:36:48.530444+0000","last_active":"2024-05-24T11:36:50.044986+0000","last_peered":"2024-05-24T11:36:50.044986+0000","last_clean":"2024-05-24T11:36:50.044986+0000","last_became_active":"2024-05-24T11:36:48.530063+0000","last_became_peered":"2024-05-24T11:36:48.530063+0000","last_unstale":"2024-05-24T11:36:50.044986+0000","last_undegraded":"2024-05-24T11:36:50.044986+0000","last_fullsized":"2024-05-24T11:36:50.044986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:23:12.380170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'2","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.581432+0000","last_change":"2024-05-24T11:36:48.086566+0000","last_active":"2024-05-24T11:36:54.581432+0000","last_peered":"2024-05-24T11:36:54.581432+0000","last_clean":"2024-05-24T11:36:54.581432+0000","last_became_active":"2024-05-24T11:36:48.086213+0000","last_became_peered":"2024-05-24T11:36:48.086213+0000","last_unstale":"2024-05-24T11:36:54.581432+0000","last_undegraded":"2024-05-24T11:36:54.581432+0000","last_fullsized":"2024-05-24T11:36:54.581432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:09:53.744478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518496+0000","last_change":"2024-05-24T11:36:48.528746+0000","last_active":"2024-05-24T11:36:49.518496+0000","last_peered":"2024-05-24T11:36:49.518496+0000","last_clean":"2024-05-24T11:36:49.518496+0000","last_became_active":"2024-05-24T11:36:48.528399+0000","last_became_peered":"2024-05-24T11:36:48.528399+0000","last_unstale":"2024-05-24T11:36:49.518496+0000","last_undegraded":"2024-05-24T11:36:49.518496+0000","last_fullsized":"2024-05-24T11:36:49.518496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:51:28.838280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.790552+0000","last_change":"2024-05-24T11:36:48.084268+0000","last_active":"2024-05-24T11:36:50.790552+0000","last_peered":"2024-05-24T11:36:50.790552+0000","last_clean":"2024-05-24T11:36:50.790552+0000","last_became_active":"2024-05-24T11:36:48.083968+0000","last_became_peered":"2024-05-24T11:36:48.083968+0000","last_unstale":"2024-05-24T11:36:50.790552+0000","last_undegraded":"2024-05-24T11:36:50.790552+0000","last_fullsized":"2024-05-24T11:36:50.790552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:02:00.989621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.046135+0000","last_change":"2024-05-24T11:36:48.523784+0000","last_active":"2024-05-24T11:36:50.046135+0000","last_peered":"2024-05-24T11:36:50.046135+0000","last_clean":"2024-05-24T11:36:50.046135+0000","last_became_active":"2024-05-24T11:36:48.523430+0000","last_became_peered":"2024-05-24T11:36:48.523430+0000","last_unstale":"2024-05-24T11:36:50.046135+0000","last_undegraded":"2024-05-24T11:36:50.046135+0000","last_fullsized":"2024-05-24T11:36:50.046135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:02:33.070861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"77'5","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:26.635773+0000","last_change":"2024-05-24T11:36:48.092166+0000","last_active":"2024-05-24T11:37:26.635773+0000","last_peered":"2024-05-24T11:37:26.635773+0000","last_clean":"2024-05-24T11:37:26.635773+0000","last_became_active":"2024-05-24T11:36:48.091682+0000","last_became_peered":"2024-05-24T11:36:48.091682+0000","last_unstale":"2024-05-24T11:37:26.635773+0000","last_undegraded":"2024-05-24T11:37:26.635773+0000","last_fullsized":"2024-05-24T11:37:26.635773+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:14:53.083387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.835997+0000","last_change":"2024-05-24T11:36:48.534618+0000","last_active":"2024-05-24T11:36:48.835997+0000","last_peered":"2024-05-24T11:36:48.835997+0000","last_clean":"2024-05-24T11:36:48.835997+0000","last_became_active":"2024-05-24T11:36:48.534251+0000","last_became_peered":"2024-05-24T11:36:48.534251+0000","last_unstale":"2024-05-24T11:36:48.835997+0000","last_undegraded":"2024-05-24T11:36:48.835997+0000","last_fullsized":"2024-05-24T11:36:48.835997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:14:46.889921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045023+0000","last_change":"2024-05-24T11:36:48.528214+0000","last_active":"2024-05-24T11:36:50.045023+0000","last_peered":"2024-05-24T11:36:50.045023+0000","last_clean":"2024-05-24T11:36:50.045023+0000","last_became_active":"2024-05-24T11:36:48.526711+0000","last_became_peered":"2024-05-24T11:36:48.526711+0000","last_unstale":"2024-05-24T11:36:50.045023+0000","last_undegraded":"2024-05-24T11:36:50.045023+0000","last_fullsized":"2024-05-24T11:36:50.045023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:06:27.935139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:29.646864+0000","last_change":"2024-05-24T11:36:48.089263+0000","last_active":"2024-05-24T11:37:29.646864+0000","last_peered":"2024-05-24T11:37:29.646864+0000","last_clean":"2024-05-24T11:37:29.646864+0000","last_became_active":"2024-05-24T11:36:48.087905+0000","last_became_peered":"2024-05-24T11:36:48.087905+0000","last_unstale":"2024-05-24T11:37:29.646864+0000","last_undegraded":"2024-05-24T11:37:29.646864+0000","last_fullsized":"2024-05-24T11:37:29.646864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:06:37.941039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044877+0000","last_change":"2024-05-24T11:36:48.517569+0000","last_active":"2024-05-24T11:36:50.044877+0000","last_peered":"2024-05-24T11:36:50.044877+0000","last_clean":"2024-05-24T11:36:50.044877+0000","last_became_active":"2024-05-24T11:36:48.517202+0000","last_became_peered":"2024-05-24T11:36:48.517202+0000","last_unstale":"2024-05-24T11:36:50.044877+0000","last_undegraded":"2024-05-24T11:36:50.044877+0000","last_fullsized":"2024-05-24T11:36:50.044877+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:38:50.778284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"77'6","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.591392+0000","last_change":"2024-05-24T11:36:48.077543+0000","last_active":"2024-05-24T11:36:54.591392+0000","last_peered":"2024-05-24T11:36:54.591392+0000","last_clean":"2024-05-24T11:36:54.591392+0000","last_became_active":"2024-05-24T11:36:48.077058+0000","last_became_peered":"2024-05-24T11:36:48.077058+0000","last_unstale":"2024-05-24T11:36:54.591392+0000","last_undegraded":"2024-05-24T11:36:54.591392+0000","last_fullsized":"2024-05-24T11:36:54.591392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:19:49.486922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":328,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.047880+0000","last_change":"2024-05-24T11:36:00.577887+0000","last_active":"2024-05-24T11:36:50.047880+0000","last_peered":"2024-05-24T11:36:50.047880+0000","last_clean":"2024-05-24T11:36:50.047880+0000","last_became_active":"2024-05-24T11:36:00.577291+0000","last_became_peered":"2024-05-24T11:36:00.577291+0000","last_unstale":"2024-05-24T11:36:50.047880+0000","last_undegraded":"2024-05-24T11:36:50.047880+0000","last_fullsized":"2024-05-24T11:36:50.047880+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:33:34.896223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:33:34.896223+0000","last_clean_scrub_stamp":"2024-05-24T11:33:34.896223+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:11:36.393700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044963+0000","last_change":"2024-05-24T11:36:48.527294+0000","last_active":"2024-05-24T11:36:50.044963+0000","last_peered":"2024-05-24T11:36:50.044963+0000","last_clean":"2024-05-24T11:36:50.044963+0000","last_became_active":"2024-05-24T11:36:48.526983+0000","last_became_peered":"2024-05-24T11:36:48.526983+0000","last_unstale":"2024-05-24T11:36:50.044963+0000","last_undegraded":"2024-05-24T11:36:50.044963+0000","last_fullsized":"2024-05-24T11:36:50.044963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:23:26.492502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:52.611289+0000","last_change":"2024-05-24T11:36:48.097290+0000","last_active":"2024-05-24T11:36:52.611289+0000","last_peered":"2024-05-24T11:36:52.611289+0000","last_clean":"2024-05-24T11:36:52.611289+0000","last_became_active":"2024-05-24T11:36:48.096942+0000","last_became_peered":"2024-05-24T11:36:48.096942+0000","last_unstale":"2024-05-24T11:36:52.611289+0000","last_undegraded":"2024-05-24T11:36:52.611289+0000","last_fullsized":"2024-05-24T11:36:52.611289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:24:20.221023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"77'4","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.579815+0000","last_change":"2024-05-24T11:36:48.079294+0000","last_active":"2024-05-24T11:36:54.579815+0000","last_peered":"2024-05-24T11:36:54.579815+0000","last_clean":"2024-05-24T11:36:54.579815+0000","last_became_active":"2024-05-24T11:36:48.078919+0000","last_became_peered":"2024-05-24T11:36:48.078919+0000","last_unstale":"2024-05-24T11:36:54.579815+0000","last_undegraded":"2024-05-24T11:36:54.579815+0000","last_fullsized":"2024-05-24T11:36:54.579815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:22:32.262674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045222+0000","last_change":"2024-05-24T11:36:48.524666+0000","last_active":"2024-05-24T11:36:50.045222+0000","last_peered":"2024-05-24T11:36:50.045222+0000","last_clean":"2024-05-24T11:36:50.045222+0000","last_became_active":"2024-05-24T11:36:48.524347+0000","last_became_peered":"2024-05-24T11:36:48.524347+0000","last_unstale":"2024-05-24T11:36:50.045222+0000","last_undegraded":"2024-05-24T11:36:50.045222+0000","last_fullsized":"2024-05-24T11:36:50.045222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:14:13.716982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:57.874846+0000","last_change":"2024-05-24T11:36:48.085665+0000","last_active":"2024-05-24T11:36:57.874846+0000","last_peered":"2024-05-24T11:36:57.874846+0000","last_clean":"2024-05-24T11:36:57.874846+0000","last_became_active":"2024-05-24T11:36:48.085344+0000","last_became_peered":"2024-05-24T11:36:48.085344+0000","last_unstale":"2024-05-24T11:36:57.874846+0000","last_undegraded":"2024-05-24T11:36:57.874846+0000","last_fullsized":"2024-05-24T11:36:57.874846+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:11:29.930656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044736+0000","last_change":"2024-05-24T11:36:48.523060+0000","last_active":"2024-05-24T11:36:50.044736+0000","last_peered":"2024-05-24T11:36:50.044736+0000","last_clean":"2024-05-24T11:36:50.044736+0000","last_became_active":"2024-05-24T11:36:48.522712+0000","last_became_peered":"2024-05-24T11:36:48.522712+0000","last_unstale":"2024-05-24T11:36:50.044736+0000","last_undegraded":"2024-05-24T11:36:50.044736+0000","last_fullsized":"2024-05-24T11:36:50.044736+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:07:54.311695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"77'7","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.589339+0000","last_change":"2024-05-24T11:36:48.092809+0000","last_active":"2024-05-24T11:36:54.589339+0000","last_peered":"2024-05-24T11:36:54.589339+0000","last_clean":"2024-05-24T11:36:54.589339+0000","last_became_active":"2024-05-24T11:36:48.092447+0000","last_became_peered":"2024-05-24T11:36:48.092447+0000","last_unstale":"2024-05-24T11:36:54.589339+0000","last_undegraded":"2024-05-24T11:36:54.589339+0000","last_fullsized":"2024-05-24T11:36:54.589339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:27:12.916761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811262+0000","last_change":"2024-05-24T11:36:48.528514+0000","last_active":"2024-05-24T11:36:49.811262+0000","last_peered":"2024-05-24T11:36:49.811262+0000","last_clean":"2024-05-24T11:36:49.811262+0000","last_became_active":"2024-05-24T11:36:48.528244+0000","last_became_peered":"2024-05-24T11:36:48.528244+0000","last_unstale":"2024-05-24T11:36:49.811262+0000","last_undegraded":"2024-05-24T11:36:49.811262+0000","last_fullsized":"2024-05-24T11:36:49.811262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:53:05.993805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599039+0000","last_change":"2024-05-24T11:36:48.094084+0000","last_active":"2024-05-24T11:36:51.599039+0000","last_peered":"2024-05-24T11:36:51.599039+0000","last_clean":"2024-05-24T11:36:51.599039+0000","last_became_active":"2024-05-24T11:36:48.093254+0000","last_became_peered":"2024-05-24T11:36:48.093254+0000","last_unstale":"2024-05-24T11:36:51.599039+0000","last_undegraded":"2024-05-24T11:36:51.599039+0000","last_fullsized":"2024-05-24T11:36:51.599039+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:56:18.896200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044843+0000","last_change":"2024-05-24T11:36:48.522292+0000","last_active":"2024-05-24T11:36:50.044843+0000","last_peered":"2024-05-24T11:36:50.044843+0000","last_clean":"2024-05-24T11:36:50.044843+0000","last_became_active":"2024-05-24T11:36:48.521356+0000","last_became_peered":"2024-05-24T11:36:48.521356+0000","last_unstale":"2024-05-24T11:36:50.044843+0000","last_undegraded":"2024-05-24T11:36:50.044843+0000","last_fullsized":"2024-05-24T11:36:50.044843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:51:44.805455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.588410+0000","last_change":"2024-05-24T11:36:48.086209+0000","last_active":"2024-05-24T11:36:54.588410+0000","last_peered":"2024-05-24T11:36:54.588410+0000","last_clean":"2024-05-24T11:36:54.588410+0000","last_became_active":"2024-05-24T11:36:48.084978+0000","last_became_peered":"2024-05-24T11:36:48.084978+0000","last_unstale":"2024-05-24T11:36:54.588410+0000","last_undegraded":"2024-05-24T11:36:54.588410+0000","last_fullsized":"2024-05-24T11:36:54.588410+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:26:39.181832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518178+0000","last_change":"2024-05-24T11:36:48.520890+0000","last_active":"2024-05-24T11:36:49.518178+0000","last_peered":"2024-05-24T11:36:49.518178+0000","last_clean":"2024-05-24T11:36:49.518178+0000","last_became_active":"2024-05-24T11:36:48.519860+0000","last_became_peered":"2024-05-24T11:36:48.519860+0000","last_unstale":"2024-05-24T11:36:49.518178+0000","last_undegraded":"2024-05-24T11:36:49.518178+0000","last_fullsized":"2024-05-24T11:36:49.518178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:54:35.691695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:57.239504+0000","last_change":"2024-05-24T11:36:48.089884+0000","last_active":"2024-05-24T11:36:57.239504+0000","last_peered":"2024-05-24T11:36:57.239504+0000","last_clean":"2024-05-24T11:36:57.239504+0000","last_became_active":"2024-05-24T11:36:48.089386+0000","last_became_peered":"2024-05-24T11:36:48.089386+0000","last_unstale":"2024-05-24T11:36:57.239504+0000","last_undegraded":"2024-05-24T11:36:57.239504+0000","last_fullsized":"2024-05-24T11:36:57.239504+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T11:56:14.463210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044730+0000","last_change":"2024-05-24T11:36:48.515178+0000","last_active":"2024-05-24T11:36:50.044730+0000","last_peered":"2024-05-24T11:36:50.044730+0000","last_clean":"2024-05-24T11:36:50.044730+0000","last_became_active":"2024-05-24T11:36:48.514277+0000","last_became_peered":"2024-05-24T11:36:48.514277+0000","last_unstale":"2024-05-24T11:36:50.044730+0000","last_undegraded":"2024-05-24T11:36:50.044730+0000","last_fullsized":"2024-05-24T11:36:50.044730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:46:38.526893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"77'4","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:51.599557+0000","last_change":"2024-05-24T11:36:48.095493+0000","last_active":"2024-05-24T11:36:51.599557+0000","last_peered":"2024-05-24T11:36:51.599557+0000","last_clean":"2024-05-24T11:36:51.599557+0000","last_became_active":"2024-05-24T11:36:48.094863+0000","last_became_peered":"2024-05-24T11:36:48.094863+0000","last_unstale":"2024-05-24T11:36:51.599557+0000","last_undegraded":"2024-05-24T11:36:51.599557+0000","last_fullsized":"2024-05-24T11:36:51.599557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T12:45:06.827169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.530486+0000","last_change":"2024-05-24T11:36:48.528024+0000","last_active":"2024-05-24T11:36:48.530486+0000","last_peered":"2024-05-24T11:36:48.530486+0000","last_clean":"2024-05-24T11:36:48.530486+0000","last_became_active":"2024-05-24T11:36:48.527689+0000","last_became_peered":"2024-05-24T11:36:48.527689+0000","last_unstale":"2024-05-24T11:36:48.530486+0000","last_undegraded":"2024-05-24T11:36:48.530486+0000","last_fullsized":"2024-05-24T11:36:48.530486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T22:07:50.186661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'6","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:37:26.646321+0000","last_change":"2024-05-24T11:36:48.090660+0000","last_active":"2024-05-24T11:37:26.646321+0000","last_peered":"2024-05-24T11:37:26.646321+0000","last_clean":"2024-05-24T11:37:26.646321+0000","last_became_active":"2024-05-24T11:36:48.090298+0000","last_became_peered":"2024-05-24T11:36:48.090298+0000","last_unstale":"2024-05-24T11:37:26.646321+0000","last_undegraded":"2024-05-24T11:37:26.646321+0000","last_fullsized":"2024-05-24T11:37:26.646321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:07:30.366506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.528160+0000","last_change":"2024-05-24T11:36:48.526287+0000","last_active":"2024-05-24T11:36:48.528160+0000","last_peered":"2024-05-24T11:36:48.528160+0000","last_clean":"2024-05-24T11:36:48.528160+0000","last_became_active":"2024-05-24T11:36:48.525951+0000","last_became_peered":"2024-05-24T11:36:48.525951+0000","last_unstale":"2024-05-24T11:36:48.528160+0000","last_undegraded":"2024-05-24T11:36:48.528160+0000","last_fullsized":"2024-05-24T11:36:48.528160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T21:38:11.562370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"77'3","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.582546+0000","last_change":"2024-05-24T11:36:48.092808+0000","last_active":"2024-05-24T11:36:54.582546+0000","last_peered":"2024-05-24T11:36:54.582546+0000","last_clean":"2024-05-24T11:36:54.582546+0000","last_became_active":"2024-05-24T11:36:48.092466+0000","last_became_peered":"2024-05-24T11:36:48.092466+0000","last_unstale":"2024-05-24T11:36:54.582546+0000","last_undegraded":"2024-05-24T11:36:54.582546+0000","last_fullsized":"2024-05-24T11:36:54.582546+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:27:08.423449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.048076+0000","last_change":"2024-05-24T11:36:48.533175+0000","last_active":"2024-05-24T11:36:50.048076+0000","last_peered":"2024-05-24T11:36:50.048076+0000","last_clean":"2024-05-24T11:36:50.048076+0000","last_became_active":"2024-05-24T11:36:48.532779+0000","last_became_peered":"2024-05-24T11:36:48.532779+0000","last_unstale":"2024-05-24T11:36:50.048076+0000","last_undegraded":"2024-05-24T11:36:50.048076+0000","last_fullsized":"2024-05-24T11:36:50.048076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:39:40.041747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'5","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:54.589679+0000","last_change":"2024-05-24T11:36:48.080825+0000","last_active":"2024-05-24T11:36:54.589679+0000","last_peered":"2024-05-24T11:36:54.589679+0000","last_clean":"2024-05-24T11:36:54.589679+0000","last_became_active":"2024-05-24T11:36:48.080462+0000","last_became_peered":"2024-05-24T11:36:48.080462+0000","last_unstale":"2024-05-24T11:36:54.589679+0000","last_undegraded":"2024-05-24T11:36:54.589679+0000","last_fullsized":"2024-05-24T11:36:54.589679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:47.056908+0000","last_clean_scrub_stamp":"2024-05-24T11:36:47.056908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T16:05:25.665222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-24T11:36:48.536673+0000","last_change":"2024-05-24T11:36:48.535094+0000","last_active":"2024-05-24T11:36:48.536673+0000","last_peered":"2024-05-24T11:36:48.536673+0000","last_clean":"2024-05-24T11:36:48.536673+0000","last_became_active":"2024-05-24T11:36:48.534670+0000","last_became_peered":"2024-05-24T11:36:48.534670+0000","last_unstale":"2024-05-24T11:36:48.536673+0000","last_undegraded":"2024-05-24T11:36:48.536673+0000","last_fullsized":"2024-05-24T11:36:48.536673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:24:49.492069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518418+0000","last_change":"2024-05-24T11:36:48.521113+0000","last_active":"2024-05-24T11:36:49.518418+0000","last_peered":"2024-05-24T11:36:49.518418+0000","last_clean":"2024-05-24T11:36:49.518418+0000","last_became_active":"2024-05-24T11:36:48.520379+0000","last_became_peered":"2024-05-24T11:36:48.520379+0000","last_unstale":"2024-05-24T11:36:49.518418+0000","last_undegraded":"2024-05-24T11:36:49.518418+0000","last_fullsized":"2024-05-24T11:36:49.518418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T20:07:17.112794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.543923+0000","last_change":"2024-05-24T11:36:48.529329+0000","last_active":"2024-05-24T11:36:49.543923+0000","last_peered":"2024-05-24T11:36:49.543923+0000","last_clean":"2024-05-24T11:36:49.543923+0000","last_became_active":"2024-05-24T11:36:48.528994+0000","last_became_peered":"2024-05-24T11:36:48.528994+0000","last_unstale":"2024-05-24T11:36:49.543923+0000","last_undegraded":"2024-05-24T11:36:49.543923+0000","last_fullsized":"2024-05-24T11:36:49.543923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:23:37.166318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.543981+0000","last_change":"2024-05-24T11:36:48.519429+0000","last_active":"2024-05-24T11:36:49.543981+0000","last_peered":"2024-05-24T11:36:49.543981+0000","last_clean":"2024-05-24T11:36:49.543981+0000","last_became_active":"2024-05-24T11:36:48.519085+0000","last_became_peered":"2024-05-24T11:36:48.519085+0000","last_unstale":"2024-05-24T11:36:49.543981+0000","last_undegraded":"2024-05-24T11:36:49.543981+0000","last_fullsized":"2024-05-24T11:36:49.543981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:41:55.524352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.811057+0000","last_change":"2024-05-24T11:36:48.528932+0000","last_active":"2024-05-24T11:36:49.811057+0000","last_peered":"2024-05-24T11:36:49.811057+0000","last_clean":"2024-05-24T11:36:49.811057+0000","last_became_active":"2024-05-24T11:36:48.528413+0000","last_became_peered":"2024-05-24T11:36:48.528413+0000","last_unstale":"2024-05-24T11:36:49.811057+0000","last_undegraded":"2024-05-24T11:36:49.811057+0000","last_fullsized":"2024-05-24T11:36:49.811057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:55:47.051779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045272+0000","last_change":"2024-05-24T11:36:48.526905+0000","last_active":"2024-05-24T11:36:50.045272+0000","last_peered":"2024-05-24T11:36:50.045272+0000","last_clean":"2024-05-24T11:36:50.045272+0000","last_became_active":"2024-05-24T11:36:48.526608+0000","last_became_peered":"2024-05-24T11:36:48.526608+0000","last_unstale":"2024-05-24T11:36:50.045272+0000","last_undegraded":"2024-05-24T11:36:50.045272+0000","last_fullsized":"2024-05-24T11:36:50.045272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:32:04.831995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518371+0000","last_change":"2024-05-24T11:36:48.523597+0000","last_active":"2024-05-24T11:36:49.518371+0000","last_peered":"2024-05-24T11:36:49.518371+0000","last_clean":"2024-05-24T11:36:49.518371+0000","last_became_active":"2024-05-24T11:36:48.523279+0000","last_became_peered":"2024-05-24T11:36:48.523279+0000","last_unstale":"2024-05-24T11:36:49.518371+0000","last_undegraded":"2024-05-24T11:36:49.518371+0000","last_fullsized":"2024-05-24T11:36:49.518371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T15:49:46.063341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.544125+0000","last_change":"2024-05-24T11:36:48.526836+0000","last_active":"2024-05-24T11:36:49.544125+0000","last_peered":"2024-05-24T11:36:49.544125+0000","last_clean":"2024-05-24T11:36:49.544125+0000","last_became_active":"2024-05-24T11:36:48.523838+0000","last_became_peered":"2024-05-24T11:36:48.523838+0000","last_unstale":"2024-05-24T11:36:49.544125+0000","last_undegraded":"2024-05-24T11:36:49.544125+0000","last_fullsized":"2024-05-24T11:36:49.544125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T18:01:13.223240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044611+0000","last_change":"2024-05-24T11:36:48.532826+0000","last_active":"2024-05-24T11:36:50.044611+0000","last_peered":"2024-05-24T11:36:50.044611+0000","last_clean":"2024-05-24T11:36:50.044611+0000","last_became_active":"2024-05-24T11:36:48.532526+0000","last_became_peered":"2024-05-24T11:36:48.532526+0000","last_unstale":"2024-05-24T11:36:50.044611+0000","last_undegraded":"2024-05-24T11:36:50.044611+0000","last_fullsized":"2024-05-24T11:36:50.044611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:15:10.133493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044873+0000","last_change":"2024-05-24T11:36:48.527805+0000","last_active":"2024-05-24T11:36:50.044873+0000","last_peered":"2024-05-24T11:36:50.044873+0000","last_clean":"2024-05-24T11:36:50.044873+0000","last_became_active":"2024-05-24T11:36:48.526047+0000","last_became_peered":"2024-05-24T11:36:48.526047+0000","last_unstale":"2024-05-24T11:36:50.044873+0000","last_undegraded":"2024-05-24T11:36:50.044873+0000","last_fullsized":"2024-05-24T11:36:50.044873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T19:20:56.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.044374+0000","last_change":"2024-05-24T11:36:48.519295+0000","last_active":"2024-05-24T11:36:50.044374+0000","last_peered":"2024-05-24T11:36:50.044374+0000","last_clean":"2024-05-24T11:36:50.044374+0000","last_became_active":"2024-05-24T11:36:48.518955+0000","last_became_peered":"2024-05-24T11:36:48.518955+0000","last_unstale":"2024-05-24T11:36:50.044374+0000","last_undegraded":"2024-05-24T11:36:50.044374+0000","last_fullsized":"2024-05-24T11:36:50.044374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T13:39:14.437903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518558+0000","last_change":"2024-05-24T11:36:48.525324+0000","last_active":"2024-05-24T11:36:49.518558+0000","last_peered":"2024-05-24T11:36:49.518558+0000","last_clean":"2024-05-24T11:36:49.518558+0000","last_became_active":"2024-05-24T11:36:48.525045+0000","last_became_peered":"2024-05-24T11:36:48.525045+0000","last_unstale":"2024-05-24T11:36:49.518558+0000","last_undegraded":"2024-05-24T11:36:49.518558+0000","last_fullsized":"2024-05-24T11:36:49.518558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T14:19:52.341136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:50.045101+0000","last_change":"2024-05-24T11:36:48.528095+0000","last_active":"2024-05-24T11:36:50.045101+0000","last_peered":"2024-05-24T11:36:50.045101+0000","last_clean":"2024-05-24T11:36:50.045101+0000","last_became_active":"2024-05-24T11:36:48.527814+0000","last_became_peered":"2024-05-24T11:36:48.527814+0000","last_unstale":"2024-05-24T11:36:50.045101+0000","last_undegraded":"2024-05-24T11:36:50.045101+0000","last_fullsized":"2024-05-24T11:36:50.045101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T17:12:42.591269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-24T11:36:49.518210+0000","last_change":"2024-05-24T11:36:48.528775+0000","last_active":"2024-05-24T11:36:49.518210+0000","last_peered":"2024-05-24T11:36:49.518210+0000","last_clean":"2024-05-24T11:36:49.518210+0000","last_became_active":"2024-05-24T11:36:48.528467+0000","last_became_peered":"2024-05-24T11:36:48.528467+0000","last_unstale":"2024-05-24T11:36:49.518210+0000","last_undegraded":"2024-05-24T11:36:49.518210+0000","last_fullsized":"2024-05-24T11:36:49.518210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-24T11:36:48.063001+0000","last_clean_scrub_stamp":"2024-05-24T11:36:48.063001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T23:10:59.444142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63965,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":390,"num_read_kb":267,"num_write":229,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645330,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":835584,"data_stored":399105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":7.0039999999999996,"5min":7.0039999999999996,"15min":7.0039999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.438}]},{"osd":1,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.68300000000000005}]},{"osd":2,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.2679999999999998,"5min":3.2679999999999998,"15min":3.2679999999999998},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.71899999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":9,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":7.125,"5min":7.125,"15min":7.125},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.81100000000000005}]},{"osd":10,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.753}]}]},{"osd":10,"up_from":65,"seq":279172874262,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40800,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704352,"statfs":{"total":95995035648,"available":95953256448,"internally_reserved":0,"allocated":1531904,"data_stored":1046696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":40237505},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.751},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.43099999999999999}]},{"osd":5,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.626}]},{"osd":6,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.46700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":8,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.70199999999999996}]},{"osd":9,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.58999999999999997}]},{"osd":11,"last update":"Fri May 24 11:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.77800000000000002}]}]},{"osd":9,"up_from":61,"seq":261993005082,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":860160,"data_stored":398690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.29999999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.621}]},{"osd":2,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.435},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.377}]},{"osd":4,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.68100000000000005}]},{"osd":8,"last update":"Fri May 24 11:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.57499999999999996}]},{"osd":10,"last update":"Fri May 24 11:37:00 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.751},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.71499999999999997}]}]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":399068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri May 24 11:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri May 24 11:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.42999999999999999,"15min":0.42099999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.374},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65500000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri May 24 11:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.432,"5min":0.377,"15min":0.377},"max":{"1min":0.71199999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47199999999999998}]},{"osd":4,"last update":"Fri May 24 11:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81000000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.62}]},{"osd":5,"last update":"Fri May 24 11:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54900000000000004,"15min":0.54200000000000004},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79500000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri May 24 11:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.81599999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.876,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri May 24 11:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68799999999999994,"15min":0.68300000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.99199999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.95599999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.81399999999999995}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.751},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.78800000000000003}]},{"osd":10,"last update":"Fri May 24 11:37:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":7,"up_from":47,"seq":201863462944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1445888,"data_stored":990846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.438}]},{"osd":1,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67700000000000005,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47799999999999998,"15min":0.47199999999999998},"min":{"1min":0.377,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.42599999999999999}]},{"osd":2,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.47299999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.54400000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.47199999999999998}]},{"osd":4,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85999999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.498}]},{"osd":5,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.65400000000000003},"min":{"1min":0.60199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70499999999999996,"15min":0.70199999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri May 24 11:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.73999999999999999,"15min":0.72799999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.2330000000000001,"5min":3.2330000000000001,"15min":3.2330000000000001},"last":0.60099999999999998}]},{"osd":8,"last update":"Fri May 24 11:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.74099999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.89200000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.65400000000000003}]},{"osd":10,"last update":"Fri May 24 11:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":6,"up_from":43,"seq":184683593765,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31936,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713216,"statfs":{"total":95995035648,"available":95962333184,"internally_reserved":0,"allocated":843776,"data_stored":398365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.318},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.33800000000000002}]},{"osd":1,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.432},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.39700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73499999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.502},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.34899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71999999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996}]},{"osd":3,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45000000000000001,"15min":0.45900000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.623,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.35999999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68600000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89500000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80700000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri May 24 11:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80800000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.86099999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri May 24 11:37:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80700000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.65800000000000003,"15min":0.66700000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64700000000000002}]},{"osd":8,"last update":"Fri May 24 11:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69299999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.625},{"interface":"front","average":{"1min":0.625,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.505,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73299999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68500000000000005}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.52500000000000002}]},{"osd":10,"last update":"Fri May 24 11:37:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66900000000000004}]}]},{"osd":1,"up_from":12,"seq":51539607607,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40184,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704968,"statfs":{"total":95995035648,"available":95953887232,"internally_reserved":0,"allocated":901120,"data_stored":456221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.32600000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.93500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri May 24 11:36:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.748}]},{"osd":3,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85499999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82399999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri May 24 11:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.50600000000000001,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri May 24 11:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri May 24 11:36:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88200000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.48699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85299999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri May 24 11:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.46700000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.62},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.56599999999999995,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.70599999999999996}]},{"osd":8,"last update":"Fri May 24 11:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.746,"15min":0.72699999999999998},"min":{"1min":0.621,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.59099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69199999999999995}]},{"osd":10,"last update":"Fri May 24 11:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.83599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":864256,"data_stored":400091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 24 11:37:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51100000000000001,"15min":0.497},"min":{"1min":0.35899999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.72099999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.34999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri May 24 11:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.754,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.45000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":1.069}]},{"osd":3,"last update":"Fri May 24 11:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.40600000000000003,"15min":0.39600000000000002},"min":{"1min":0.23799999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.40999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64700000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Fri May 24 11:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.47599999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72199999999999998,"5min":0.755,"15min":0.755},"last":1.038},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.46999999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri May 24 11:36:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.32800000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Fri May 24 11:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55000000000000004}]},{"osd":8,"last update":"Fri May 24 11:37:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.93300000000000005}]},{"osd":9,"last update":"Fri May 24 11:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":1.137}]},{"osd":10,"last update":"Fri May 24 11:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.98399999999999999}]},{"osd":11,"last update":"Fri May 24 11:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.82499999999999996}]}]},{"osd":2,"up_from":17,"seq":73014444083,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":456287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.746,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56599999999999995,"15min":0.56999999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.85699999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri May 24 11:36:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76100000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84899999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri May 24 11:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72599999999999998,"5min":0.746,"15min":0.746},"last":0.877},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.95899999999999996}]},{"osd":5,"last update":"Fri May 24 11:36:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.433,"15min":0.42399999999999999},"min":{"1min":0.313,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri May 24 11:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri May 24 11:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri May 24 11:37:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82999999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.82899999999999996}]},{"osd":9,"last update":"Fri May 24 11:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.88800000000000001}]},{"osd":10,"last update":"Fri May 24 11:36:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":856064,"data_stored":397913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.28699999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.375,"15min":0.36599999999999999},"min":{"1min":0.308,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.29699999999999999}]},{"osd":1,"last update":"Fri May 24 11:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.48099999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri May 24 11:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.76600000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.436},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri May 24 11:37:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.41699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70799999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.40400000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84799999999999998,"5min":0.88,"15min":0.88},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri May 24 11:36:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84299999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73599999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri May 24 11:36:52 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40999999999999998}]},{"osd":7,"last update":"Fri May 24 11:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89900000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70699999999999996,"15min":0.71199999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91600000000000004,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70799999999999996}]},{"osd":8,"last update":"Fri May 24 11:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.66900000000000004}]},{"osd":9,"last update":"Fri May 24 11:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.50700000000000001}]},{"osd":10,"last update":"Fri May 24 11:36:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.624}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953516,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":398220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.502,"15min":0.495},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.37,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri May 24 11:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40100000000000002,"15min":0.38400000000000001},"min":{"1min":0.33100000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.40699999999999997},"min":{"1min":0.372,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.60199999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri May 24 11:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.61099999999999999,"15min":0.58699999999999997},"min":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77800000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.497}]},{"osd":3,"last update":"Fri May 24 11:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.57399999999999995},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89300000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.433},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.438,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri May 24 11:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.435,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.874,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri May 24 11:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.61599999999999999,"15min":0.60299999999999998},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.621},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.433,"5min":0.378,"15min":0.378},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri May 24 11:37:06 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.379,"15min":0.379},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.41999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82999999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57499999999999996}]},{"osd":8,"last update":"Fri May 24 11:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91900000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67200000000000004,"15min":0.66200000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.98799999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.64200000000000002}]},{"osd":9,"last update":"Fri May 24 11:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.60399999999999998}]},{"osd":10,"last update":"Fri May 24 11:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.79300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":5,"up_from":36,"seq":154618822697,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32536,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712616,"statfs":{"total":95995035648,"available":95961718784,"internally_reserved":0,"allocated":1458176,"data_stored":990620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.307,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68899999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76100000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.748}]},{"osd":2,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.28499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.51300000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.40500000000000003,"15min":0.40899999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.38100000000000001}]},{"osd":3,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri May 24 11:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.441,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53200000000000003}]},{"osd":6,"last update":"Fri May 24 11:36:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri May 24 11:37:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.51100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.63500000000000001}]},{"osd":8,"last update":"Fri May 24 11:37:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.5},{"interface":"front","average":{"1min":0.626,"5min":0.57199999999999995,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.44600000000000001}]},{"osd":9,"last update":"Fri May 24 11:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.62},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.60599999999999998}]},{"osd":10,"last update":"Fri May 24 11:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.75700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-24T11:37:33.839 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-24T11:37:33.839 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-24T11:37:33.839 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-24T11:37:33.839 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-24T11:37:34.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:33 smithi039 ceph-mon[42919]: pgmap v242: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-24T11:37:34.080 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:34.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:33 smithi053 ceph-mon[47448]: pgmap v242: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-24T11:37:34.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:33 smithi137 ceph-mon[47907]: pgmap v242: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-24T11:37:35.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:34 smithi039 ceph-mon[42919]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:37:35.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.189+0000 7f0c081d7700 1 -- 172.21.15.39:0/1737177998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00100550 msgr2=0x7f0c00105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.189+0000 7f0c081d7700 1 --2- 172.21.15.39:0/1737177998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00100550 0x7f0c00105c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0bf00099c0 tx=0x7f0bf0009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.189+0000 7f0c081d7700 1 -- 172.21.15.39:0/1737177998 shutdown_connections 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.189+0000 7f0c081d7700 1 --2- 172.21.15.39:0/1737177998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c000ffba0 0x7f0c000fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.189+0000 7f0c081d7700 1 --2- 172.21.15.39:0/1737177998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c001063c0 0x7f0c0010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.189+0000 7f0c081d7700 1 --2- 172.21.15.39:0/1737177998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00100550 0x7f0c00105c90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.189+0000 7f0c081d7700 1 -- 172.21.15.39:0/1737177998 >> 172.21.15.39:0/1737177998 conn(0x7f0c00075260 msgr2=0x7f0c00075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.189+0000 7f0c081d7700 1 -- 172.21.15.39:0/1737177998 shutdown_connections 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.189+0000 7f0c081d7700 1 -- 172.21.15.39:0/1737177998 wait complete. 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c081d7700 1 Processor -- start 2024-05-24T11:37:35.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c081d7700 1 -- start start 2024-05-24T11:37:35.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c081d7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c000ffba0 0x7f0c001972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:35.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c081d7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c00100550 0x7f0c001977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:35.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c081d7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001063c0 0x7f0c0019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:35.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c081d7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0019c1b0 con 0x7f0c000ffba0 2024-05-24T11:37:35.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c081d7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0019c320 con 0x7f0c00100550 2024-05-24T11:37:35.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c081d7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0019c4e0 con 0x7f0c001063c0 2024-05-24T11:37:35.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c05f73700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c000ffba0 0x7f0c001972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:35.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c05f73700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c000ffba0 0x7f0c001972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53126/0 (socket says 172.21.15.39:53126) 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c05f73700 1 -- 172.21.15.39:0/3338978350 learned_addr learned my addr 172.21.15.39:0/3338978350 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.190+0000 7f0c05772700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c00100550 0x7f0c001977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0c06774700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001063c0 0x7f0c0019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0c05f73700 1 -- 172.21.15.39:0/3338978350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001063c0 msgr2=0x7f0c0019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0c05f73700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001063c0 0x7f0c0019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0c05f73700 1 -- 172.21.15.39:0/3338978350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c00100550 msgr2=0x7f0c001977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0c05f73700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c00100550 0x7f0c001977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0c05f73700 1 -- 172.21.15.39:0/3338978350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf00095f0 con 0x7f0c000ffba0 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0c05f73700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c000ffba0 0x7f0c001972b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00d950 tx=0x7f0bfc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0bf6ffd700 1 -- 172.21.15.39:0/3338978350 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00f890 con 0x7f0c000ffba0 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0bf6ffd700 1 -- 172.21.15.39:0/3338978350 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bfc00fe80 con 0x7f0c000ffba0 2024-05-24T11:37:35.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0bf6ffd700 1 -- 172.21.15.39:0/3338978350 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00e3b0 con 0x7f0c000ffba0 2024-05-24T11:37:35.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0019c7c0 con 0x7f0c000ffba0 2024-05-24T11:37:35.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.191+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c001a4020 con 0x7f0c000ffba0 2024-05-24T11:37:35.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.192+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c001095a0 con 0x7f0c000ffba0 2024-05-24T11:37:35.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.193+0000 7f0bf6ffd700 1 -- 172.21.15.39:0/3338978350 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0bfc010460 con 0x7f0c000ffba0 2024-05-24T11:37:35.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.194+0000 7f0bf6ffd700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0bec0a11b0 0x7f0bec0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:35.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.194+0000 7f0c05772700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0bec0a11b0 0x7f0bec0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:35.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.194+0000 7f0bf6ffd700 1 -- 172.21.15.39:0/3338978350 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0bfc0cc770 con 0x7f0c000ffba0 2024-05-24T11:37:35.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.194+0000 7f0c05772700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0bec0a11b0 0x7f0bec0a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009990 tx=0x7f0bf0005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:35.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.197+0000 7f0bf6ffd700 1 -- 172.21.15.39:0/3338978350 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0bfc095e50 con 0x7f0c000ffba0 2024-05-24T11:37:35.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:34 smithi053 ceph-mon[47448]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:37:35.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:34 smithi137 ceph-mon[47907]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-24T11:37:35.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.447+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0c0019c9b0 con 0x7f0c000ffba0 2024-05-24T11:37:35.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.447+0000 7f0bf6ffd700 1 -- 172.21.15.39:0/3338978350 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0bfc016040 con 0x7f0c000ffba0 2024-05-24T11:37:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:35.453 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-24T11:37:35.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0bec0a11b0 msgr2=0x7f0bec0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:35.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0bec0a11b0 0x7f0bec0a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009990 tx=0x7f0bf0005a90 comp rx=0 tx=0).stop 2024-05-24T11:37:35.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c000ffba0 msgr2=0x7f0c001972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:35.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c000ffba0 0x7f0c001972b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00d950 tx=0x7f0bfc00dc60 comp rx=0 tx=0).stop 2024-05-24T11:37:35.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 shutdown_connections 2024-05-24T11:37:35.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0bec0a11b0 0x7f0bec0a3670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c001063c0 0x7f0c0019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c000ffba0 0x7f0c001972b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 --2- 172.21.15.39:0/3338978350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c00100550 0x7f0c001977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.449+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 >> 172.21.15.39:0/3338978350 conn(0x7f0c00075260 msgr2=0x7f0c00105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:35.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.450+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 shutdown_connections 2024-05-24T11:37:35.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.450+0000 7f0c081d7700 1 -- 172.21.15.39:0/3338978350 wait complete. 2024-05-24T11:37:35.846 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-24T11:37:35.846 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:37:35.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.974+0000 7f429bfbb700 1 -- 172.21.15.39:0/3344332398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 msgr2=0x7f42941066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:35.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.974+0000 7f429bfbb700 1 --2- 172.21.15.39:0/3344332398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 0x7f42941066d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4284009990 tx=0x7f4284009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.974+0000 7f429bfbb700 1 -- 172.21.15.39:0/3344332398 shutdown_connections 2024-05-24T11:37:35.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.974+0000 7f429bfbb700 1 --2- 172.21.15.39:0/3344332398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42941019b0 0x7f4294103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.974+0000 7f429bfbb700 1 --2- 172.21.15.39:0/3344332398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4294106c10 0x7f4294109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.974+0000 7f429bfbb700 1 --2- 172.21.15.39:0/3344332398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 0x7f42941066d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.974+0000 7f429bfbb700 1 -- 172.21.15.39:0/3344332398 >> 172.21.15.39:0/3344332398 conn(0x7f42940fb300 msgr2=0x7f42940fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.974+0000 7f429bfbb700 1 -- 172.21.15.39:0/3344332398 shutdown_connections 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.974+0000 7f429bfbb700 1 -- 172.21.15.39:0/3344332398 wait complete. 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f429bfbb700 1 Processor -- start 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f429bfbb700 1 -- start start 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f429bfbb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42941019b0 0x7f4294101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f429bfbb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 0x7f42940ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f429bfbb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106c10 0x7f42940ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f429bfbb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42941005a0 con 0x7f42941042e0 2024-05-24T11:37:35.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f429bfbb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4294100710 con 0x7f42941019b0 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f429bfbb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42941a0d80 con 0x7f4294106c10 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299556700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 0x7f42940ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299556700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 0x7f42940ff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53132/0 (socket says 172.21.15.39:53132) 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299556700 1 -- 172.21.15.39:0/1779922167 learned_addr learned my addr 172.21.15.39:0/1779922167 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299d57700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42941019b0 0x7f4294101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f429a558700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106c10 0x7f42940ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299556700 1 -- 172.21.15.39:0/1779922167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106c10 msgr2=0x7f42940ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299556700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106c10 0x7f42940ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299556700 1 -- 172.21.15.39:0/1779922167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42941019b0 msgr2=0x7f4294101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299556700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42941019b0 0x7f4294101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299556700 1 -- 172.21.15.39:0/1779922167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42840095f0 con 0x7f42941042e0 2024-05-24T11:37:35.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.975+0000 7f4299556700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 0x7f42940ff960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f42840098e0 tx=0x7f4284003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:35.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.976+0000 7f428affd700 1 -- 172.21.15.39:0/1779922167 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428401d070 con 0x7f42941042e0 2024-05-24T11:37:35.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.976+0000 7f428affd700 1 -- 172.21.15.39:0/1779922167 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f428400bd90 con 0x7f42941042e0 2024-05-24T11:37:35.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.976+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42941a0f80 con 0x7f42941042e0 2024-05-24T11:37:35.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.976+0000 7f428affd700 1 -- 172.21.15.39:0/1779922167 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284022c10 con 0x7f42941042e0 2024-05-24T11:37:35.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.976+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42941a1340 con 0x7f42941042e0 2024-05-24T11:37:35.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.977+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42940fce80 con 0x7f42941042e0 2024-05-24T11:37:35.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.978+0000 7f428affd700 1 -- 172.21.15.39:0/1779922167 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4284022470 con 0x7f42941042e0 2024-05-24T11:37:35.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.978+0000 7f428affd700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42800a11b0 0x7f42800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:35.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.979+0000 7f4299d57700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42800a11b0 0x7f42800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:35.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.979+0000 7f428affd700 1 -- 172.21.15.39:0/1779922167 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f42840ce620 con 0x7f42941042e0 2024-05-24T11:37:35.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.979+0000 7f4299d57700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42800a11b0 0x7f42800a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4290005950 tx=0x7f429000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:35.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:35.982+0000 7f428affd700 1 -- 172.21.15.39:0/1779922167 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4284097d90 con 0x7f42941042e0 2024-05-24T11:37:36.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:35 smithi039 ceph-mon[42919]: pgmap v243: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:37:36.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:35 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3338978350' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-24T11:37:36.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.192+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f429410c520 con 0x7f42941042e0 2024-05-24T11:37:36.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.194+0000 7f428affd700 1 -- 172.21.15.39:0/1779922167 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+22061 (secure 0 0 0) 0x7f428409b3b0 con 0x7f42941042e0 2024-05-24T11:37:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:36.199 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:49.502598+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42800a11b0 msgr2=0x7f42800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42800a11b0 0x7f42800a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4290005950 tx=0x7f429000b410 comp rx=0 tx=0).stop 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 msgr2=0x7f42940ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 0x7f42940ff960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f42840098e0 tx=0x7f4284003a50 comp rx=0 tx=0).stop 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 shutdown_connections 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f42800a11b0 0x7f42800a3670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106c10 0x7f42940ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42941019b0 0x7f4294101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 --2- 172.21.15.39:0/1779922167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42941042e0 0x7f42940ff960 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 >> 172.21.15.39:0/1779922167 conn(0x7f42940fb300 msgr2=0x7f4294103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:36.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 shutdown_connections 2024-05-24T11:37:36.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.196+0000 7f429bfbb700 1 -- 172.21.15.39:0/1779922167 wait complete. 2024-05-24T11:37:36.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-24T11:36:46.509049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-24T11:36:47.866055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:37:36.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-24T11:37:36.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:35 smithi053 ceph-mon[47448]: pgmap v243: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:37:36.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:35 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3338978350' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-24T11:37:36.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:35 smithi137 ceph-mon[47907]: pgmap v243: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:37:36.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:35 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3338978350' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-24T11:37:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.339+0000 7f727db33700 1 -- 172.21.15.39:0/1824905613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278101990 msgr2=0x7f72781082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.339+0000 7f727db33700 1 --2- 172.21.15.39:0/1824905613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278101990 0x7f72781082b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f727000b0a0 tx=0x7f727000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.339+0000 7f727db33700 1 -- 172.21.15.39:0/1824905613 shutdown_connections 2024-05-24T11:37:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.339+0000 7f727db33700 1 --2- 172.21.15.39:0/1824905613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278101990 0x7f72781082b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.339+0000 7f727db33700 1 --2- 172.21.15.39:0/1824905613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72780ffe80 0x7f72781002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.339+0000 7f727db33700 1 --2- 172.21.15.39:0/1824905613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7278100fd0 0x7f7278101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.339+0000 7f727db33700 1 -- 172.21.15.39:0/1824905613 >> 172.21.15.39:0/1824905613 conn(0x7f72780fb3f0 msgr2=0x7f72780fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.339+0000 7f727db33700 1 -- 172.21.15.39:0/1824905613 shutdown_connections 2024-05-24T11:37:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.339+0000 7f727db33700 1 -- 172.21.15.39:0/1824905613 wait complete. 2024-05-24T11:37:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f727db33700 1 Processor -- start 2024-05-24T11:37:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f727db33700 1 -- start start 2024-05-24T11:37:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f727db33700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72780ffe80 0x7f7278195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f727db33700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278100fd0 0x7f72781963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f727db33700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7278101990 0x7f727819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f727db33700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727819d990 con 0x7f7278100fd0 2024-05-24T11:37:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f727db33700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727819dad0 con 0x7f7278101990 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f727db33700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727819dc60 con 0x7f72780ffe80 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f7276ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278100fd0 0x7f72781963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f7276ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278100fd0 0x7f72781963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53158/0 (socket says 172.21.15.39:53158) 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.340+0000 7f7276ffd700 1 -- 172.21.15.39:0/231745927 learned_addr learned my addr 172.21.15.39:0/231745927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7277fff700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7278101990 0x7f727819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f72777fe700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72780ffe80 0x7f7278195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7276ffd700 1 -- 172.21.15.39:0/231745927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72780ffe80 msgr2=0x7f7278195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7276ffd700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72780ffe80 0x7f7278195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7276ffd700 1 -- 172.21.15.39:0/231745927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7278101990 msgr2=0x7f727819d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7276ffd700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7278101990 0x7f727819d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7276ffd700 1 -- 172.21.15.39:0/231745927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7270009d00 con 0x7f7278100fd0 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7277fff700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7278101990 0x7f727819d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7276ffd700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278100fd0 0x7f72781963a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f726800d950 tx=0x7f726800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7274ff9700 1 -- 172.21.15.39:0/231745927 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268009b40 con 0x7f7278100fd0 2024-05-24T11:37:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7274ff9700 1 -- 172.21.15.39:0/231745927 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7268009ca0 con 0x7f7278100fd0 2024-05-24T11:37:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727819df40 con 0x7f7278100fd0 2024-05-24T11:37:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f7274ff9700 1 -- 172.21.15.39:0/231745927 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800f620 con 0x7f7278100fd0 2024-05-24T11:37:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.341+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727819e3c0 con 0x7f7278100fd0 2024-05-24T11:37:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.342+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7278107340 con 0x7f7278100fd0 2024-05-24T11:37:36.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.343+0000 7f7274ff9700 1 -- 172.21.15.39:0/231745927 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7268010460 con 0x7f7278100fd0 2024-05-24T11:37:36.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.344+0000 7f7274ff9700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72640a11b0 0x7f72640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:36.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.344+0000 7f72777fe700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72640a11b0 0x7f72640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:36.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.344+0000 7f7274ff9700 1 -- 172.21.15.39:0/231745927 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72680ccc70 con 0x7f7278100fd0 2024-05-24T11:37:36.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.344+0000 7f72777fe700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72640a11b0 0x7f72640a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f726000bbd0 tx=0x7f726000b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:36.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.347+0000 7f7274ff9700 1 -- 172.21.15.39:0/231745927 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7268096350 con 0x7f7278100fd0 2024-05-24T11:37:36.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.557+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f727810b580 con 0x7f7278100fd0 2024-05-24T11:37:36.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.557+0000 7f7274ff9700 1 -- 172.21.15.39:0/231745927 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7268016020 con 0x7f7278100fd0 2024-05-24T11:37:36.562 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:37:36.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.559+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72640a11b0 msgr2=0x7f72640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.559+0000 7f727db33700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72640a11b0 0x7f72640a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f726000bbd0 tx=0x7f726000b5b0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.559+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278100fd0 msgr2=0x7f72781963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.559+0000 7f727db33700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278100fd0 0x7f72781963a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f726800d950 tx=0x7f726800dd10 comp rx=0 tx=0).stop 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.559+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 shutdown_connections 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.560+0000 7f727db33700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72640a11b0 0x7f72640a3670 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.560+0000 7f727db33700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72780ffe80 0x7f7278195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.560+0000 7f727db33700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7278101990 0x7f727819d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.560+0000 7f727db33700 1 --2- 172.21.15.39:0/231745927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278100fd0 0x7f72781963a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.560+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 >> 172.21.15.39:0/231745927 conn(0x7f72780fb3f0 msgr2=0x7f7278104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.560+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 shutdown_connections 2024-05-24T11:37:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.560+0000 7f727db33700 1 -- 172.21.15.39:0/231745927 wait complete. 2024-05-24T11:37:36.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-24T11:37:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.701+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2688471475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac1019e0 msgr2=0x7fdcac103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.701+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2688471475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac1019e0 0x7fdcac103dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009990 tx=0x7fdc9c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.702+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2688471475 shutdown_connections 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.702+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2688471475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcac106c40 0x7fdcac1090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.702+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2688471475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac1019e0 0x7fdcac103dd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.702+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2688471475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcac104310 0x7fdcac106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.702+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2688471475 >> 172.21.15.39:0/2688471475 conn(0x7fdcac0fb3f0 msgr2=0x7fdcac0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.702+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2688471475 shutdown_connections 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.702+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2688471475 wait complete. 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.702+0000 7fdcb0bb6700 1 Processor -- start 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.702+0000 7fdcb0bb6700 1 -- start start 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcb0bb6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcac1019e0 0x7fdcac196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcb0bb6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac104310 0x7fdcac197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcb0bb6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcac106c40 0x7fdcac19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcb0bb6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac19e970 con 0x7fdcac104310 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcb0bb6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac19eab0 con 0x7fdcac106c40 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcb0bb6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac19ec40 con 0x7fdcac1019e0 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdca9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac104310 0x7fdcac197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdca9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac104310 0x7fdcac197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53172/0 (socket says 172.21.15.39:53172) 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdca9d9b700 1 -- 172.21.15.39:0/2145052824 learned_addr learned my addr 172.21.15.39:0/2145052824 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcaad9d700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcac106c40 0x7fdcac19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcaa59c700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcac1019e0 0x7fdcac196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdca9d9b700 1 -- 172.21.15.39:0/2145052824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcac1019e0 msgr2=0x7fdcac196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdca9d9b700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcac1019e0 0x7fdcac196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdca9d9b700 1 -- 172.21.15.39:0/2145052824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcac106c40 msgr2=0x7fdcac19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdca9d9b700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcac106c40 0x7fdcac19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdca9d9b700 1 -- 172.21.15.39:0/2145052824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c0095f0 con 0x7fdcac104310 2024-05-24T11:37:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcaa59c700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcac1019e0 0x7fdcac196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdca9d9b700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac104310 0x7fdcac197380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdca000d950 tx=0x7fdca000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdc9b7fe700 1 -- 172.21.15.39:0/2145052824 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0009940 con 0x7fdcac104310 2024-05-24T11:37:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdc9b7fe700 1 -- 172.21.15.39:0/2145052824 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdca0010460 con 0x7fdcac104310 2024-05-24T11:37:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcac19ef20 con 0x7fdcac104310 2024-05-24T11:37:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdc9b7fe700 1 -- 172.21.15.39:0/2145052824 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000f620 con 0x7fdcac104310 2024-05-24T11:37:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.703+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac19f3f0 con 0x7fdcac104310 2024-05-24T11:37:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.704+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcac0fcf70 con 0x7fdcac104310 2024-05-24T11:37:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.706+0000 7fdc9b7fe700 1 -- 172.21.15.39:0/2145052824 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdca00105d0 con 0x7fdcac104310 2024-05-24T11:37:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.706+0000 7fdc9b7fe700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdc940a1230 0x7fdc940a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.706+0000 7fdcaa59c700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdc940a1230 0x7fdc940a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.706+0000 7fdc9b7fe700 1 -- 172.21.15.39:0/2145052824 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdca00cc660 con 0x7fdcac104310 2024-05-24T11:37:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.707+0000 7fdcaa59c700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdc940a1230 0x7fdc940a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c003780 tx=0x7fdc9c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.710+0000 7fdc9b7fe700 1 -- 172.21.15.39:0/2145052824 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdca0095e40 con 0x7fdcac104310 2024-05-24T11:37:36.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:36 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1779922167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:36.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:36 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/231745927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.919+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcac10c640 con 0x7fdcac104310 2024-05-24T11:37:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.920+0000 7fdc9b7fe700 1 -- 172.21.15.39:0/2145052824 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdca00160a0 con 0x7fdcac104310 2024-05-24T11:37:36.925 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdc940a1230 msgr2=0x7fdc940a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdc940a1230 0x7fdc940a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c003780 tx=0x7fdc9c005fb0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac104310 msgr2=0x7fdcac197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac104310 0x7fdcac197380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdca000d950 tx=0x7fdca000dc60 comp rx=0 tx=0).stop 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 shutdown_connections 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdc940a1230 0x7fdc940a36f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcac1019e0 0x7fdcac196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcac106c40 0x7fdcac19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 --2- 172.21.15.39:0/2145052824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac104310 0x7fdcac197380 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 >> 172.21.15.39:0/2145052824 conn(0x7fdcac0fb3f0 msgr2=0x7fdcac103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 shutdown_connections 2024-05-24T11:37:36.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:36.922+0000 7fdcb0bb6700 1 -- 172.21.15.39:0/2145052824 wait complete. 2024-05-24T11:37:36.939 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-24T11:37:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.064+0000 7f0dcd42c700 1 -- 172.21.15.39:0/323184868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc8100f20 msgr2=0x7f0dc8101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.064+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/323184868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc8100f20 0x7f0dc8101360 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009990 tx=0x7f0db8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.064+0000 7f0dcd42c700 1 -- 172.21.15.39:0/323184868 shutdown_connections 2024-05-24T11:37:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.064+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/323184868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc8102ab0 0x7f0dc81093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.064+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/323184868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc8100f20 0x7f0dc8101360 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.064+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/323184868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0dc81020f0 0x7f0dc8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.064+0000 7f0dcd42c700 1 -- 172.21.15.39:0/323184868 >> 172.21.15.39:0/323184868 conn(0x7f0dc80fc4e0 msgr2=0x7f0dc80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.064+0000 7f0dcd42c700 1 -- 172.21.15.39:0/323184868 shutdown_connections 2024-05-24T11:37:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.064+0000 7f0dcd42c700 1 -- 172.21.15.39:0/323184868 wait complete. 2024-05-24T11:37:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dcd42c700 1 Processor -- start 2024-05-24T11:37:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dcd42c700 1 -- start start 2024-05-24T11:37:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dcd42c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc8100f20 0x7f0dc8196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dcd42c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc81020f0 0x7f0dc81974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dcd42c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0dc8102ab0 0x7f0dc819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dcd42c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc819eab0 con 0x7f0dc81020f0 2024-05-24T11:37:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dcd42c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc819ebf0 con 0x7f0dc8100f20 2024-05-24T11:37:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dcd42c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc819ed80 con 0x7f0dc8102ab0 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dc67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc81020f0 0x7f0dc81974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dc67fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc81020f0 0x7f0dc81974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53190/0 (socket says 172.21.15.39:53190) 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.065+0000 7f0dc67fc700 1 -- 172.21.15.39:0/2200015464 learned_addr learned my addr 172.21.15.39:0/2200015464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dc6ffd700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc8100f20 0x7f0dc8196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dc77fe700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0dc8102ab0 0x7f0dc819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dc67fc700 1 -- 172.21.15.39:0/2200015464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0dc8102ab0 msgr2=0x7f0dc819e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dc67fc700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0dc8102ab0 0x7f0dc819e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dc67fc700 1 -- 172.21.15.39:0/2200015464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc8100f20 msgr2=0x7f0dc8196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dc67fc700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc8100f20 0x7f0dc8196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dc67fc700 1 -- 172.21.15.39:0/2200015464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db80095f0 con 0x7f0dc81020f0 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dc77fe700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0dc8102ab0 0x7f0dc819e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dc67fc700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc81020f0 0x7f0dc81974c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0db000b530 tx=0x7f0db000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dbffff700 1 -- 172.21.15.39:0/2200015464 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0011830 con 0x7f0dc81020f0 2024-05-24T11:37:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dbffff700 1 -- 172.21.15.39:0/2200015464 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0db0011e20 con 0x7f0dc81020f0 2024-05-24T11:37:37.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc819f060 con 0x7f0dc81020f0 2024-05-24T11:37:37.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dbffff700 1 -- 172.21.15.39:0/2200015464 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db000f2e0 con 0x7f0dc81020f0 2024-05-24T11:37:37.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.066+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc819f570 con 0x7f0dc81020f0 2024-05-24T11:37:37.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.067+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc8108480 con 0x7f0dc81020f0 2024-05-24T11:37:37.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.068+0000 7f0dbffff700 1 -- 172.21.15.39:0/2200015464 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0db0011990 con 0x7f0dc81020f0 2024-05-24T11:37:37.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.069+0000 7f0dbffff700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0db40a11b0 0x7f0db40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.069+0000 7f0dbffff700 1 -- 172.21.15.39:0/2200015464 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0db00cc810 con 0x7f0dc81020f0 2024-05-24T11:37:37.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.069+0000 7f0dc6ffd700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0db40a11b0 0x7f0db40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.069+0000 7f0dc6ffd700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0db40a11b0 0x7f0db40a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b5c0 tx=0x7f0db8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:37.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.073+0000 7f0dbffff700 1 -- 172.21.15.39:0/2200015464 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0db0096000 con 0x7f0dc81020f0 2024-05-24T11:37:37.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:36 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1779922167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:37.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:36 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/231745927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:37.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:36 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1779922167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:37.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:36 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/231745927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:37.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.278+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0dc810c780 con 0x7f0dc81020f0 2024-05-24T11:37:37.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.279+0000 7f0dbffff700 1 -- 172.21.15.39:0/2200015464 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0db0099620 con 0x7f0dc81020f0 2024-05-24T11:37:37.284 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-05-24T11:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0db40a11b0 msgr2=0x7f0db40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0db40a11b0 0x7f0db40a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b5c0 tx=0x7f0db8005fd0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc81020f0 msgr2=0x7f0dc81974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc81020f0 0x7f0dc81974c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0db000b530 tx=0x7f0db000b840 comp rx=0 tx=0).stop 2024-05-24T11:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 shutdown_connections 2024-05-24T11:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0db40a11b0 0x7f0db40a3670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0dc8102ab0 0x7f0dc819e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc8100f20 0x7f0dc8196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 --2- 172.21.15.39:0/2200015464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc81020f0 0x7f0dc81974c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 >> 172.21.15.39:0/2200015464 conn(0x7f0dc80fc4e0 msgr2=0x7f0dc81055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 shutdown_connections 2024-05-24T11:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.281+0000 7f0dcd42c700 1 -- 172.21.15.39:0/2200015464 wait complete. 2024-05-24T11:37:37.299 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:37:37.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.420+0000 7fb34b781700 1 -- 172.21.15.39:0/4118648359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 msgr2=0x7fb344103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.420+0000 7fb34b781700 1 --2- 172.21.15.39:0/4118648359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 0x7fb344103d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb340009990 tx=0x7fb340009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.420+0000 7fb34b781700 1 -- 172.21.15.39:0/4118648359 shutdown_connections 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.420+0000 7fb34b781700 1 --2- 172.21.15.39:0/4118648359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb344106c00 0x7fb344108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.420+0000 7fb34b781700 1 --2- 172.21.15.39:0/4118648359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 0x7fb344103d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.420+0000 7fb34b781700 1 --2- 172.21.15.39:0/4118648359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3441042d0 0x7fb3441066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.420+0000 7fb34b781700 1 -- 172.21.15.39:0/4118648359 >> 172.21.15.39:0/4118648359 conn(0x7fb3440fb2d0 msgr2=0x7fb3440fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.420+0000 7fb34b781700 1 -- 172.21.15.39:0/4118648359 shutdown_connections 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.420+0000 7fb34b781700 1 -- 172.21.15.39:0/4118648359 wait complete. 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34b781700 1 Processor -- start 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34b781700 1 -- start start 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34b781700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 0x7fb344196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34b781700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3441042d0 0x7fb344197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34b781700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb344106c00 0x7fb34419e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34b781700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34419e960 con 0x7fb3441019a0 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34b781700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34419eaa0 con 0x7fb344106c00 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34b781700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34419ec30 con 0x7fb3441042d0 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34951d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 0x7fb344196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34951d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 0x7fb344196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49636/0 (socket says 172.21.15.39:49636) 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34951d700 1 -- 172.21.15.39:0/618085726 learned_addr learned my addr 172.21.15.39:0/618085726 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb349d1e700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb344106c00 0x7fb34419e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb348d1c700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3441042d0 0x7fb344197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34951d700 1 -- 172.21.15.39:0/618085726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3441042d0 msgr2=0x7fb344197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34951d700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3441042d0 0x7fb344197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.421+0000 7fb34951d700 1 -- 172.21.15.39:0/618085726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb344106c00 msgr2=0x7fb34419e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.422+0000 7fb34951d700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb344106c00 0x7fb34419e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.422+0000 7fb34951d700 1 -- 172.21.15.39:0/618085726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400095f0 con 0x7fb3441019a0 2024-05-24T11:37:37.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.422+0000 7fb34951d700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 0x7fb344196e30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb3400098e0 tx=0x7fb340003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:37.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.422+0000 7fb3367fc700 1 -- 172.21.15.39:0/618085726 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34001d070 con 0x7fb3441019a0 2024-05-24T11:37:37.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.422+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34419eeb0 con 0x7fb3441019a0 2024-05-24T11:37:37.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.422+0000 7fb3367fc700 1 -- 172.21.15.39:0/618085726 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb34000bd90 con 0x7fb3441019a0 2024-05-24T11:37:37.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.422+0000 7fb3367fc700 1 -- 172.21.15.39:0/618085726 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340022e30 con 0x7fb3441019a0 2024-05-24T11:37:37.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.422+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34419f2a0 con 0x7fb3441019a0 2024-05-24T11:37:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.423+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3440fce50 con 0x7fb3441019a0 2024-05-24T11:37:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.424+0000 7fb3367fc700 1 -- 172.21.15.39:0/618085726 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb340022600 con 0x7fb3441019a0 2024-05-24T11:37:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.424+0000 7fb3367fc700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb3300a11b0 0x7fb3300a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.425+0000 7fb348d1c700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb3300a11b0 0x7fb3300a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.425+0000 7fb3367fc700 1 -- 172.21.15.39:0/618085726 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3400cdf90 con 0x7fb3441019a0 2024-05-24T11:37:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.425+0000 7fb348d1c700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb3300a11b0 0x7fb3300a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb338009a10 tx=0x7fb338009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:37.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.428+0000 7fb3367fc700 1 -- 172.21.15.39:0/618085726 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb340097800 con 0x7fb3441019a0 2024-05-24T11:37:37.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.692+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb344068060 con 0x7fb3441019a0 2024-05-24T11:37:37.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.692+0000 7fb3367fc700 1 -- 172.21.15.39:0/618085726 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10906 (secure 0 0 0) 0x7fb34009ae20 con 0x7fb3441019a0 2024-05-24T11:37:37.698 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:37.698 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-24T11:36:48.500682+0000","modified":"2024-05-24T11:36:55.584150+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24511,"mds_2":24506,"mds_3":24490,"mds_4":14631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/3006971240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3006971240},{"type":"v1","addr":"172.21.15.39:6839","nonce":3006971240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/3024712815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3024712815},{"type":"v1","addr":"172.21.15.39:6841","nonce":3024712815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/218688120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":218688120},{"type":"v1","addr":"172.21.15.137:6839","nonce":218688120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3196235839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3196235839},{"type":"v1","addr":"172.21.15.53:6839","nonce":3196235839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-24T11:37:37.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb3300a11b0 msgr2=0x7fb3300a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb3300a11b0 0x7fb3300a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb338009a10 tx=0x7fb338009190 comp rx=0 tx=0).stop 2024-05-24T11:37:37.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 msgr2=0x7fb344196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 0x7fb344196e30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb3400098e0 tx=0x7fb340003a50 comp rx=0 tx=0).stop 2024-05-24T11:37:37.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 shutdown_connections 2024-05-24T11:37:37.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb3300a11b0 0x7fb3300a3670 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb344106c00 0x7fb34419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3441019a0 0x7fb344196e30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 --2- 172.21.15.39:0/618085726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3441042d0 0x7fb344197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 >> 172.21.15.39:0/618085726 conn(0x7fb3440fb2d0 msgr2=0x7fb344103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:37.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 shutdown_connections 2024-05-24T11:37:37.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.695+0000 7fb34b781700 1 -- 172.21.15.39:0/618085726 wait complete. 2024-05-24T11:37:37.701 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-24T11:37:37.712 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:37:37.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.836+0000 7f1073c63700 1 -- 172.21.15.39:0/1255386747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c106bb0 msgr2=0x7f106c108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.836+0000 7f1073c63700 1 --2- 172.21.15.39:0/1255386747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c106bb0 0x7f106c108fc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f106400b0a0 tx=0x7f106400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.836+0000 7f1073c63700 1 -- 172.21.15.39:0/1255386747 shutdown_connections 2024-05-24T11:37:37.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.836+0000 7f1073c63700 1 --2- 172.21.15.39:0/1255386747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f106c101950 0x7f106c103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.836+0000 7f1073c63700 1 --2- 172.21.15.39:0/1255386747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c106bb0 0x7f106c108fc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.836+0000 7f1073c63700 1 --2- 172.21.15.39:0/1255386747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f106c104280 0x7f106c106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.836+0000 7f1073c63700 1 -- 172.21.15.39:0/1255386747 >> 172.21.15.39:0/1255386747 conn(0x7f106c0fb2e0 msgr2=0x7f106c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.836+0000 7f1073c63700 1 -- 172.21.15.39:0/1255386747 shutdown_connections 2024-05-24T11:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.836+0000 7f1073c63700 1 -- 172.21.15.39:0/1255386747 wait complete. 2024-05-24T11:37:37.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f1073c63700 1 Processor -- start 2024-05-24T11:37:37.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f1073c63700 1 -- start start 2024-05-24T11:37:37.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f1073c63700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f106c101950 0x7f106c196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f1073c63700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c104280 0x7f106c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f1073c63700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f106c106bb0 0x7f106c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f1073c63700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c19e8e0 con 0x7f106c104280 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f1073c63700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c19ea20 con 0x7f106c106bb0 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f1073c63700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c19ebb0 con 0x7f106c101950 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f10711fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c104280 0x7f106c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f10711fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c104280 0x7f106c197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49658/0 (socket says 172.21.15.39:49658) 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f10711fe700 1 -- 172.21.15.39:0/2532943427 learned_addr learned my addr 172.21.15.39:0/2532943427 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.837+0000 7f1072200700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f106c106bb0 0x7f106c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f10719ff700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f106c101950 0x7f106c196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f10711fe700 1 -- 172.21.15.39:0/2532943427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f106c101950 msgr2=0x7f106c196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f10711fe700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f106c101950 0x7f106c196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f10711fe700 1 -- 172.21.15.39:0/2532943427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f106c106bb0 msgr2=0x7f106c19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f10711fe700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f106c106bb0 0x7f106c19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f10711fe700 1 -- 172.21.15.39:0/2532943427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064009d00 con 0x7f106c104280 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f10719ff700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f106c101950 0x7f106c196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:37.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f10711fe700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c104280 0x7f106c197340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f106800e9a0 tx=0x7f106800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:37.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f105effd700 1 -- 172.21.15.39:0/2532943427 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068009790 con 0x7f106c104280 2024-05-24T11:37:37.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f105effd700 1 -- 172.21.15.39:0/2532943427 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f106800cbd0 con 0x7f106c104280 2024-05-24T11:37:37.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c19ee90 con 0x7f106c104280 2024-05-24T11:37:37.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f105effd700 1 -- 172.21.15.39:0/2532943427 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068018680 con 0x7f106c104280 2024-05-24T11:37:37.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.838+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c19f360 con 0x7f106c104280 2024-05-24T11:37:37.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.839+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c0fce60 con 0x7f106c104280 2024-05-24T11:37:37.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.840+0000 7f105effd700 1 -- 172.21.15.39:0/2532943427 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f106800cd40 con 0x7f106c104280 2024-05-24T11:37:37.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.841+0000 7f105effd700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10580a1230 0x7f10580a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:37.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.841+0000 7f10719ff700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10580a1230 0x7f10580a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:37.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.841+0000 7f105effd700 1 -- 172.21.15.39:0/2532943427 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1068014070 con 0x7f106c104280 2024-05-24T11:37:37.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.841+0000 7f10719ff700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10580a1230 0x7f10580a36f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1060005d90 tx=0x7f1060005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:37.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:37.844+0000 7f105effd700 1 -- 172.21.15.39:0/2532943427 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f10680965c0 con 0x7f106c104280 2024-05-24T11:37:38.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.055+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f106c067380 con 0x7f106c104280 2024-05-24T11:37:38.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.057+0000 7f105effd700 1 -- 172.21.15.39:0/2532943427 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+22061 (secure 0 0 0) 0x7f1068026020 con 0x7f106c104280 2024-05-24T11:37:38.061 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:38.062 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:49.502598+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10580a1230 msgr2=0x7f10580a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10580a1230 0x7f10580a36f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1060005d90 tx=0x7f1060005d20 comp rx=0 tx=0).stop 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c104280 msgr2=0x7f106c197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c104280 0x7f106c197340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f106800e9a0 tx=0x7f106800ecb0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 shutdown_connections 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f10580a1230 0x7f10580a36f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f106c101950 0x7f106c196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f106c106bb0 0x7f106c19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 --2- 172.21.15.39:0/2532943427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f106c104280 0x7f106c197340 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 >> 172.21.15.39:0/2532943427 conn(0x7f106c0fb2e0 msgr2=0x7f106c1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:38.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 shutdown_connections 2024-05-24T11:37:38.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.059+0000 7f1073c63700 1 -- 172.21.15.39:0/2532943427 wait complete. 2024-05-24T11:37:38.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:37 smithi039 ceph-mon[42919]: pgmap v244: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-24T11:37:38.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:37 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2145052824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:38.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:37 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2200015464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:38.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:37 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/618085726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:38.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-24T11:36:46.509049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-24T11:36:47.866055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:37:38.077 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-24T11:37:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 -- 172.21.15.39:0/403005731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 msgr2=0x7f3d04108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 --2- 172.21.15.39:0/403005731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 0x7f3d04108290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00b0a0 tx=0x7f3cfc00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 -- 172.21.15.39:0/403005731 shutdown_connections 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 --2- 172.21.15.39:0/403005731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d040ffe60 0x7f3d04100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 --2- 172.21.15.39:0/403005731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 0x7f3d04108290 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 --2- 172.21.15.39:0/403005731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04100fb0 0x7f3d04101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 -- 172.21.15.39:0/403005731 >> 172.21.15.39:0/403005731 conn(0x7f3d040fb3f0 msgr2=0x7f3d040fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 -- 172.21.15.39:0/403005731 shutdown_connections 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 -- 172.21.15.39:0/403005731 wait complete. 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 Processor -- start 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.202+0000 7f3d0b932700 1 -- start start 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d0b932700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d040ffe60 0x7f3d04196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d0b932700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04100fb0 0x7f3d04197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d0b932700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 0x7f3d0419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d0b932700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0419ea20 con 0x7f3d04101970 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d0b932700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0419eb60 con 0x7f3d04100fb0 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d0b932700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0419ecf0 con 0x7f3d040ffe60 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d09ecf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 0x7f3d0419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d09ecf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 0x7f3d0419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49678/0 (socket says 172.21.15.39:49678) 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d09ecf700 1 -- 172.21.15.39:0/4047190765 learned_addr learned my addr 172.21.15.39:0/4047190765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d096ce700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d040ffe60 0x7f3d04196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d08ecd700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04100fb0 0x7f3d04197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d09ecf700 1 -- 172.21.15.39:0/4047190765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d040ffe60 msgr2=0x7f3d04196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d09ecf700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d040ffe60 0x7f3d04196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d09ecf700 1 -- 172.21.15.39:0/4047190765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04100fb0 msgr2=0x7f3d04197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d09ecf700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04100fb0 0x7f3d04197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d09ecf700 1 -- 172.21.15.39:0/4047190765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc009d00 con 0x7f3d04101970 2024-05-24T11:37:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d09ecf700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 0x7f3d0419e450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009570 tx=0x7f3cfc0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3cfa7fc700 1 -- 172.21.15.39:0/4047190765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00e040 con 0x7f3d04101970 2024-05-24T11:37:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3cfa7fc700 1 -- 172.21.15.39:0/4047190765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3cfc01d470 con 0x7f3d04101970 2024-05-24T11:37:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0419ef70 con 0x7f3d04101970 2024-05-24T11:37:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.203+0000 7f3cfa7fc700 1 -- 172.21.15.39:0/4047190765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0052a0 con 0x7f3d04101970 2024-05-24T11:37:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.204+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0419f390 con 0x7f3d04101970 2024-05-24T11:37:38.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.204+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d04067cf0 con 0x7f3d04101970 2024-05-24T11:37:38.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.207+0000 7f3cfa7fc700 1 -- 172.21.15.39:0/4047190765 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3cfc01d5e0 con 0x7f3d04101970 2024-05-24T11:37:38.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.207+0000 7f3cfa7fc700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3cf00a11b0 0x7f3cf00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.207+0000 7f3cfa7fc700 1 -- 172.21.15.39:0/4047190765 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3cfc0ce4a0 con 0x7f3d04101970 2024-05-24T11:37:38.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.207+0000 7f3d096ce700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3cf00a11b0 0x7f3cf00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.208+0000 7f3d096ce700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3cf00a11b0 0x7f3cf00a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009c60 tx=0x7f3d00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:38.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.210+0000 7f3cfa7fc700 1 -- 172.21.15.39:0/4047190765 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3cfc097d10 con 0x7f3d04101970 2024-05-24T11:37:38.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:37 smithi053 ceph-mon[47448]: pgmap v244: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-24T11:37:38.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:37 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2145052824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:38.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:37 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2200015464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:38.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:37 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/618085726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:38.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:37 smithi137 ceph-mon[47907]: pgmap v244: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-24T11:37:38.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:37 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2145052824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:38.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:37 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2200015464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:38.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:37 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/618085726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:38.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.416+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d0419f640 con 0x7f3d04101970 2024-05-24T11:37:38.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.416+0000 7f3cfa7fc700 1 -- 172.21.15.39:0/4047190765 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3cfc09b330 con 0x7f3d04101970 2024-05-24T11:37:38.421 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:37:38.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3cf00a11b0 msgr2=0x7f3cf00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3cf00a11b0 0x7f3cf00a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009c60 tx=0x7f3d00009260 comp rx=0 tx=0).stop 2024-05-24T11:37:38.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 msgr2=0x7f3d0419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 0x7f3d0419e450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009570 tx=0x7f3cfc0095a0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 shutdown_connections 2024-05-24T11:37:38.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f3cf00a11b0 0x7f3cf00a3670 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d040ffe60 0x7f3d04196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d04101970 0x7f3d0419e450 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 --2- 172.21.15.39:0/4047190765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d04100fb0 0x7f3d04197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 >> 172.21.15.39:0/4047190765 conn(0x7f3d040fb3f0 msgr2=0x7f3d04067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:38.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 shutdown_connections 2024-05-24T11:37:38.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.419+0000 7f3d0b932700 1 -- 172.21.15.39:0/4047190765 wait complete. 2024-05-24T11:37:38.436 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-24T11:37:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 -- 172.21.15.39:0/1902517159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c781020f0 msgr2=0x7f9c78102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 --2- 172.21.15.39:0/1902517159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c781020f0 0x7f9c78102570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009990 tx=0x7f9c60009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 -- 172.21.15.39:0/1902517159 shutdown_connections 2024-05-24T11:37:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 --2- 172.21.15.39:0/1902517159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78102ab0 0x7f9c781093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 --2- 172.21.15.39:0/1902517159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c78100f20 0x7f9c78101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 --2- 172.21.15.39:0/1902517159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c781020f0 0x7f9c78102570 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 -- 172.21.15.39:0/1902517159 >> 172.21.15.39:0/1902517159 conn(0x7f9c780fc4e0 msgr2=0x7f9c780fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 -- 172.21.15.39:0/1902517159 shutdown_connections 2024-05-24T11:37:38.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 -- 172.21.15.39:0/1902517159 wait complete. 2024-05-24T11:37:38.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 Processor -- start 2024-05-24T11:37:38.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.561+0000 7f9c7e921700 1 -- start start 2024-05-24T11:37:38.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c7e921700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c78100f20 0x7f9c78074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c7e921700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c781020f0 0x7f9c78073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c7e921700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78102ab0 0x7f9c780738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c7e921700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78074070 con 0x7f9c78100f20 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c7e921700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780741e0 con 0x7f9c781020f0 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c7e921700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c781a0e10 con 0x7f9c78102ab0 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c78100f20 0x7f9c78074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c78100f20 0x7f9c78074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49692/0 (socket says 172.21.15.39:49692) 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c77fff700 1 -- 172.21.15.39:0/3523931103 learned_addr learned my addr 172.21.15.39:0/3523931103 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c7cebe700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78102ab0 0x7f9c780738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c777fe700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c781020f0 0x7f9c78073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c77fff700 1 -- 172.21.15.39:0/3523931103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78102ab0 msgr2=0x7f9c780738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c77fff700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78102ab0 0x7f9c780738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c77fff700 1 -- 172.21.15.39:0/3523931103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c781020f0 msgr2=0x7f9c78073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c77fff700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c781020f0 0x7f9c78073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c77fff700 1 -- 172.21.15.39:0/3523931103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600095f0 con 0x7f9c78100f20 2024-05-24T11:37:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c77fff700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c78100f20 0x7f9c78074d00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800c990 tx=0x7f9c6800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c757fa700 1 -- 172.21.15.39:0/3523931103 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c680078e0 con 0x7f9c78100f20 2024-05-24T11:37:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c781a10a0 con 0x7f9c78100f20 2024-05-24T11:37:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c757fa700 1 -- 172.21.15.39:0/3523931103 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c6800f450 con 0x7f9c78100f20 2024-05-24T11:37:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.562+0000 7f9c757fa700 1 -- 172.21.15.39:0/3523931103 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68018660 con 0x7f9c78100f20 2024-05-24T11:37:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.563+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c781a1600 con 0x7f9c78100f20 2024-05-24T11:37:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.563+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c781083b0 con 0x7f9c78100f20 2024-05-24T11:37:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.565+0000 7f9c757fa700 1 -- 172.21.15.39:0/3523931103 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9c68007a40 con 0x7f9c78100f20 2024-05-24T11:37:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.565+0000 7f9c757fa700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c640a11b0 0x7f9c640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.565+0000 7f9c777fe700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c640a11b0 0x7f9c640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.565+0000 7f9c757fa700 1 -- 172.21.15.39:0/3523931103 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9c680cc880 con 0x7f9c78100f20 2024-05-24T11:37:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.566+0000 7f9c777fe700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c640a11b0 0x7f9c640a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c010 tx=0x7f9c60003720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.569+0000 7f9c757fa700 1 -- 172.21.15.39:0/3523931103 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9c680960f0 con 0x7f9c78100f20 2024-05-24T11:37:38.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.774+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c7810c910 con 0x7f9c78100f20 2024-05-24T11:37:38.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.774+0000 7f9c757fa700 1 -- 172.21.15.39:0/3523931103 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c6800f5c0 con 0x7f9c78100f20 2024-05-24T11:37:38.779 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-05-24T11:37:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c640a11b0 msgr2=0x7f9c640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c640a11b0 0x7f9c640a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c010 tx=0x7f9c60003720 comp rx=0 tx=0).stop 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c78100f20 msgr2=0x7f9c78074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c78100f20 0x7f9c78074d00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800c990 tx=0x7f9c6800cca0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 shutdown_connections 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f9c640a11b0 0x7f9c640a3670 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78102ab0 0x7f9c780738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c78100f20 0x7f9c78074d00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 --2- 172.21.15.39:0/3523931103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c781020f0 0x7f9c78073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 >> 172.21.15.39:0/3523931103 conn(0x7f9c780fc4e0 msgr2=0x7f9c781054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 shutdown_connections 2024-05-24T11:37:38.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.777+0000 7f9c7e921700 1 -- 172.21.15.39:0/3523931103 wait complete. 2024-05-24T11:37:38.794 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-24T11:37:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.920+0000 7f5144f7f700 1 -- 172.21.15.39:0/633466444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5140101390 msgr2=0x7f51401017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.920+0000 7f5144f7f700 1 --2- 172.21.15.39:0/633466444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5140101390 0x7f51401017b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5130009990 tx=0x7f5130009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.920+0000 7f5144f7f700 1 -- 172.21.15.39:0/633466444 shutdown_connections 2024-05-24T11:37:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.920+0000 7f5144f7f700 1 --2- 172.21.15.39:0/633466444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51401009a0 0x7f51401082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.920+0000 7f5144f7f700 1 --2- 172.21.15.39:0/633466444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5140101390 0x7f51401017b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.920+0000 7f5144f7f700 1 --2- 172.21.15.39:0/633466444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51400fffe0 0x7f5140100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.920+0000 7f5144f7f700 1 -- 172.21.15.39:0/633466444 >> 172.21.15.39:0/633466444 conn(0x7f51400fab50 msgr2=0x7f51400fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.920+0000 7f5144f7f700 1 -- 172.21.15.39:0/633466444 shutdown_connections 2024-05-24T11:37:38.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.920+0000 7f5144f7f700 1 -- 172.21.15.39:0/633466444 wait complete. 2024-05-24T11:37:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f5144f7f700 1 Processor -- start 2024-05-24T11:37:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f5144f7f700 1 -- start start 2024-05-24T11:37:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f5144f7f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51400fffe0 0x7f5140195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f5144f7f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51401009a0 0x7f5140196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f5144f7f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140101390 0x7f514019d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f5144f7f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514019d840 con 0x7f51401009a0 2024-05-24T11:37:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f5144f7f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514019d9b0 con 0x7f5140101390 2024-05-24T11:37:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f5144f7f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514019db70 con 0x7f51400fffe0 2024-05-24T11:37:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f513dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51401009a0 0x7f5140196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f513dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51401009a0 0x7f5140196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49710/0 (socket says 172.21.15.39:49710) 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f513dd9b700 1 -- 172.21.15.39:0/2786695312 learned_addr learned my addr 172.21.15.39:0/2786695312 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.921+0000 7f513ed9d700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140101390 0x7f514019d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f513e59c700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51400fffe0 0x7f5140195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f513dd9b700 1 -- 172.21.15.39:0/2786695312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51400fffe0 msgr2=0x7f5140195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f513dd9b700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51400fffe0 0x7f5140195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f513dd9b700 1 -- 172.21.15.39:0/2786695312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140101390 msgr2=0x7f514019d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f513dd9b700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140101390 0x7f514019d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f513dd9b700 1 -- 172.21.15.39:0/2786695312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300095f0 con 0x7f51401009a0 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f513e59c700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51400fffe0 0x7f5140195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f513dd9b700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51401009a0 0x7f5140196290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f513400c970 tx=0x7f513400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f512f7fe700 1 -- 172.21.15.39:0/2786695312 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134004220 con 0x7f51401009a0 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f512f7fe700 1 -- 172.21.15.39:0/2786695312 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5134007d30 con 0x7f51401009a0 2024-05-24T11:37:38.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514019de50 con 0x7f51401009a0 2024-05-24T11:37:38.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f512f7fe700 1 -- 172.21.15.39:0/2786695312 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134003b80 con 0x7f51401009a0 2024-05-24T11:37:38.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.922+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514019e270 con 0x7f51401009a0 2024-05-24T11:37:38.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.923+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51401073b0 con 0x7f51401009a0 2024-05-24T11:37:38.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.924+0000 7f512f7fe700 1 -- 172.21.15.39:0/2786695312 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5134003ce0 con 0x7f51401009a0 2024-05-24T11:37:38.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.925+0000 7f512f7fe700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51280a11b0 0x7f51280a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:38.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.925+0000 7f513e59c700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51280a11b0 0x7f51280a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:38.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.925+0000 7f512f7fe700 1 -- 172.21.15.39:0/2786695312 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f51340cc4e0 con 0x7f51401009a0 2024-05-24T11:37:38.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.925+0000 7f513e59c700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51280a11b0 0x7f51280a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5130006010 tx=0x7f5130009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:38.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:38.929+0000 7f512f7fe700 1 -- 172.21.15.39:0/2786695312 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5134095cc0 con 0x7f51401009a0 2024-05-24T11:37:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:38 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2532943427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:38 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4047190765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:39.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:38 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3523931103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:39.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.135+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5140106d90 con 0x7f51401009a0 2024-05-24T11:37:39.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.136+0000 7f512f7fe700 1 -- 172.21.15.39:0/2786695312 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51340992e0 con 0x7f51401009a0 2024-05-24T11:37:39.141 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-05-24T11:37:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51280a11b0 msgr2=0x7f51280a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51280a11b0 0x7f51280a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5130006010 tx=0x7f5130009510 comp rx=0 tx=0).stop 2024-05-24T11:37:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51401009a0 msgr2=0x7f5140196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51401009a0 0x7f5140196290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f513400c970 tx=0x7f513400cc80 comp rx=0 tx=0).stop 2024-05-24T11:37:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 shutdown_connections 2024-05-24T11:37:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f51280a11b0 0x7f51280a3670 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51400fffe0 0x7f5140195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5140101390 0x7f514019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 --2- 172.21.15.39:0/2786695312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51401009a0 0x7f5140196290 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 >> 172.21.15.39:0/2786695312 conn(0x7f51400fab50 msgr2=0x7f51401044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:39.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 shutdown_connections 2024-05-24T11:37:39.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.138+0000 7f5144f7f700 1 -- 172.21.15.39:0/2786695312 wait complete. 2024-05-24T11:37:39.155 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:37:39.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:38 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2532943427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:39.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:38 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4047190765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:39.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:38 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3523931103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:38 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2532943427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:38 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4047190765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:39.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:38 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3523931103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:39.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 -- 172.21.15.39:0/3035548821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 msgr2=0x7f7b5c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3035548821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 0x7f7b5c102440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009990 tx=0x7f7b4c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 -- 172.21.15.39:0/3035548821 shutdown_connections 2024-05-24T11:37:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3035548821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c102980 0x7f7b5c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3035548821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c100e70 0x7f7b5c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3035548821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 0x7f7b5c102440 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 -- 172.21.15.39:0/3035548821 >> 172.21.15.39:0/3035548821 conn(0x7f7b5c0fc400 msgr2=0x7f7b5c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 -- 172.21.15.39:0/3035548821 shutdown_connections 2024-05-24T11:37:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 -- 172.21.15.39:0/3035548821 wait complete. 2024-05-24T11:37:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 Processor -- start 2024-05-24T11:37:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.277+0000 7f7b61bad700 1 -- start start 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b61bad700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c100e70 0x7f7b5c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b61bad700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 0x7f7b5c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b61bad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c102980 0x7f7b5c1a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b61bad700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c1a0a40 con 0x7f7b5c101fc0 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b61bad700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c1a0b80 con 0x7f7b5c100e70 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b61bad700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c1a0d10 con 0x7f7b5c102980 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 0x7f7b5c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 0x7f7b5c199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49720/0 (socket says 172.21.15.39:49720) 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5affd700 1 -- 172.21.15.39:0/3047811024 learned_addr learned my addr 172.21.15.39:0/3047811024 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c102980 0x7f7b5c1a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5b7fe700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c100e70 0x7f7b5c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5affd700 1 -- 172.21.15.39:0/3047811024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c102980 msgr2=0x7f7b5c1a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5affd700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c102980 0x7f7b5c1a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5affd700 1 -- 172.21.15.39:0/3047811024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c100e70 msgr2=0x7f7b5c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5affd700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c100e70 0x7f7b5c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5affd700 1 -- 172.21.15.39:0/3047811024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c0095f0 con 0x7f7b5c101fc0 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5b7fe700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c100e70 0x7f7b5c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b5affd700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 0x7f7b5c199450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c006010 tx=0x7f7b4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b58ff9700 1 -- 172.21.15.39:0/3047811024 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c01d070 con 0x7f7b5c101fc0 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5c1a0f90 con 0x7f7b5c101fc0 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b58ff9700 1 -- 172.21.15.39:0/3047811024 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b4c00bd50 con 0x7f7b5c101fc0 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.278+0000 7f7b58ff9700 1 -- 172.21.15.39:0/3047811024 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00fd90 con 0x7f7b5c101fc0 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.279+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c1a1350 con 0x7f7b5c101fc0 2024-05-24T11:37:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.279+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5c067380 con 0x7f7b5c101fc0 2024-05-24T11:37:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.280+0000 7f7b58ff9700 1 -- 172.21.15.39:0/3047811024 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7b4c00f4f0 con 0x7f7b5c101fc0 2024-05-24T11:37:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.281+0000 7f7b58ff9700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b480a11b0 0x7f7b480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.281+0000 7f7b5b7fe700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b480a11b0 0x7f7b480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.281+0000 7f7b58ff9700 1 -- 172.21.15.39:0/3047811024 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7b4c0cdfb0 con 0x7f7b5c101fc0 2024-05-24T11:37:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.282+0000 7f7b5b7fe700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b480a11b0 0x7f7b480a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009cb0 tx=0x7f7b44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.284+0000 7f7b58ff9700 1 -- 172.21.15.39:0/3047811024 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b4c097820 con 0x7f7b5c101fc0 2024-05-24T11:37:39.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.537+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b5c10c790 con 0x7f7b5c101fc0 2024-05-24T11:37:39.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.538+0000 7f7b58ff9700 1 -- 172.21.15.39:0/3047811024 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10906 (secure 0 0 0) 0x7f7b4c09ae40 con 0x7f7b5c101fc0 2024-05-24T11:37:39.544 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:39.544 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-24T11:36:48.500682+0000","modified":"2024-05-24T11:36:55.584150+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24511,"mds_2":24506,"mds_3":24490,"mds_4":14631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/3006971240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3006971240},{"type":"v1","addr":"172.21.15.39:6839","nonce":3006971240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/3024712815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3024712815},{"type":"v1","addr":"172.21.15.39:6841","nonce":3024712815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/218688120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":218688120},{"type":"v1","addr":"172.21.15.137:6839","nonce":218688120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3196235839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3196235839},{"type":"v1","addr":"172.21.15.53:6839","nonce":3196235839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-24T11:37:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b480a11b0 msgr2=0x7f7b480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b480a11b0 0x7f7b480a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009cb0 tx=0x7f7b44009260 comp rx=0 tx=0).stop 2024-05-24T11:37:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 msgr2=0x7f7b5c199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 0x7f7b5c199450 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c006010 tx=0x7f7b4c004730 comp rx=0 tx=0).stop 2024-05-24T11:37:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 shutdown_connections 2024-05-24T11:37:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b480a11b0 0x7f7b480a3670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c102980 0x7f7b5c1a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c100e70 0x7f7b5c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 --2- 172.21.15.39:0/3047811024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c101fc0 0x7f7b5c199450 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.540+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 >> 172.21.15.39:0/3047811024 conn(0x7f7b5c0fc400 msgr2=0x7f7b5c107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.541+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 shutdown_connections 2024-05-24T11:37:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.541+0000 7f7b61bad700 1 -- 172.21.15.39:0/3047811024 wait complete. 2024-05-24T11:37:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-24T11:37:39.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:37:39.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 -- 172.21.15.39:0/2227528740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 msgr2=0x7f2dbc108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/2227528740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 0x7f2dbc108fd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2db400b0a0 tx=0x7f2db400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 -- 172.21.15.39:0/2227528740 shutdown_connections 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/2227528740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dbc101900 0x7f2dbc103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/2227528740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 0x7f2dbc108fd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/2227528740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dbc104230 0x7f2dbc106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 -- 172.21.15.39:0/2227528740 >> 172.21.15.39:0/2227528740 conn(0x7f2dbc0fb2f0 msgr2=0x7f2dbc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 -- 172.21.15.39:0/2227528740 shutdown_connections 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 -- 172.21.15.39:0/2227528740 wait complete. 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 Processor -- start 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.681+0000 7f2dc28cb700 1 -- start start 2024-05-24T11:37:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc28cb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dbc101900 0x7f2dbc196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc28cb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dbc104230 0x7f2dbc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc28cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 0x7f2dbc19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc28cb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc19e8e0 con 0x7f2dbc106b60 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc28cb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc19ea20 con 0x7f2dbc101900 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc28cb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc19ebb0 con 0x7f2dbc104230 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc0e68700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 0x7f2dbc19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc0e68700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 0x7f2dbc19e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49730/0 (socket says 172.21.15.39:49730) 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc0e68700 1 -- 172.21.15.39:0/4014520419 learned_addr learned my addr 172.21.15.39:0/4014520419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dbbfff700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dbc101900 0x7f2dbc196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dbb7fe700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dbc104230 0x7f2dbc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc0e68700 1 -- 172.21.15.39:0/4014520419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dbc104230 msgr2=0x7f2dbc1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc0e68700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dbc104230 0x7f2dbc1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc0e68700 1 -- 172.21.15.39:0/4014520419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dbc101900 msgr2=0x7f2dbc196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc0e68700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dbc101900 0x7f2dbc196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc0e68700 1 -- 172.21.15.39:0/4014520419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4009d00 con 0x7f2dbc106b60 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.682+0000 7f2dc0e68700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 0x7f2dbc19e310 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2db400b780 tx=0x7f2db400bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.683+0000 7f2db97fa700 1 -- 172.21.15.39:0/4014520419 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db400e040 con 0x7f2dbc106b60 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.683+0000 7f2db97fa700 1 -- 172.21.15.39:0/4014520419 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2db4009480 con 0x7f2dbc106b60 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.683+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc19ee30 con 0x7f2dbc106b60 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.683+0000 7f2db97fa700 1 -- 172.21.15.39:0/4014520419 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40052c0 con 0x7f2dbc106b60 2024-05-24T11:37:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.683+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc19f1f0 con 0x7f2dbc106b60 2024-05-24T11:37:39.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.684+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc0fce70 con 0x7f2dbc106b60 2024-05-24T11:37:39.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.685+0000 7f2db97fa700 1 -- 172.21.15.39:0/4014520419 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2db4005420 con 0x7f2dbc106b60 2024-05-24T11:37:39.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.685+0000 7f2db97fa700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2da40a11b0 0x7f2da40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:39.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.685+0000 7f2dbbfff700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2da40a11b0 0x7f2da40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:39.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.685+0000 7f2db97fa700 1 -- 172.21.15.39:0/4014520419 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2db40cd9f0 con 0x7f2dbc106b60 2024-05-24T11:37:39.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.686+0000 7f2dbbfff700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2da40a11b0 0x7f2da40a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009c60 tx=0x7f2db0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:39.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.690+0000 7f2db97fa700 1 -- 172.21.15.39:0/4014520419 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2db40cfe60 con 0x7f2dbc106b60 2024-05-24T11:37:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:39 smithi039 ceph-mon[42919]: pgmap v245: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-24T11:37:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:39 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2786695312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:39.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:39 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3047811024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.901+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dbc10c510 con 0x7f2dbc106b60 2024-05-24T11:37:39.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.902+0000 7f2db97fa700 1 -- 172.21.15.39:0/4014520419 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+22061 (secure 0 0 0) 0x7f2db4097260 con 0x7f2dbc106b60 2024-05-24T11:37:39.908 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:39.908 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:36:49.502598+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:37:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2da40a11b0 msgr2=0x7f2da40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2da40a11b0 0x7f2da40a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009c60 tx=0x7f2db0009260 comp rx=0 tx=0).stop 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 msgr2=0x7f2dbc19e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 0x7f2dbc19e310 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2db400b780 tx=0x7f2db400bcc0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 shutdown_connections 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f2da40a11b0 0x7f2da40a3670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dbc106b60 0x7f2dbc19e310 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dbc101900 0x7f2dbc196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 --2- 172.21.15.39:0/4014520419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2dbc104230 0x7f2dbc1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 >> 172.21.15.39:0/4014520419 conn(0x7f2dbc0fb2f0 msgr2=0x7f2dbc1021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 shutdown_connections 2024-05-24T11:37:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:39.905+0000 7f2dc28cb700 1 -- 172.21.15.39:0/4014520419 wait complete. 2024-05-24T11:37:39.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:37:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.044+0000 7faf2fb0b700 1 -- 172.21.15.39:0/4089025506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 msgr2=0x7faf281013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.044+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/4089025506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 0x7faf281013e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faf180099c0 tx=0x7faf18009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.044+0000 7faf2fb0b700 1 -- 172.21.15.39:0/4089025506 shutdown_connections 2024-05-24T11:37:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.044+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/4089025506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf280ffd90 0x7faf281001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.044+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/4089025506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf28101920 0x7faf281081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.044+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/4089025506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 0x7faf281013e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.044+0000 7faf2fb0b700 1 -- 172.21.15.39:0/4089025506 >> 172.21.15.39:0/4089025506 conn(0x7faf280fb300 msgr2=0x7faf280fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.044+0000 7faf2fb0b700 1 -- 172.21.15.39:0/4089025506 shutdown_connections 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.044+0000 7faf2fb0b700 1 -- 172.21.15.39:0/4089025506 wait complete. 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2fb0b700 1 Processor -- start 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2fb0b700 1 -- start start 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2fb0b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf280ffd90 0x7faf28196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2fb0b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 0x7faf28197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2fb0b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf28101920 0x7faf2819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2fb0b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2819e820 con 0x7faf28100f80 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2fb0b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2819e990 con 0x7faf280ffd90 2024-05-24T11:37:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2fb0b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2819eb50 con 0x7faf28101920 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2d0a6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 0x7faf28197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2d0a6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 0x7faf28197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49756/0 (socket says 172.21.15.39:49756) 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2d0a6700 1 -- 172.21.15.39:0/1889777448 learned_addr learned my addr 172.21.15.39:0/1889777448 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2d8a7700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf280ffd90 0x7faf28196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.045+0000 7faf2e0a8700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf28101920 0x7faf2819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2d0a6700 1 -- 172.21.15.39:0/1889777448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf28101920 msgr2=0x7faf2819e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2d0a6700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf28101920 0x7faf2819e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2d0a6700 1 -- 172.21.15.39:0/1889777448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf280ffd90 msgr2=0x7faf28196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2d0a6700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf280ffd90 0x7faf28196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2d0a6700 1 -- 172.21.15.39:0/1889777448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf180095f0 con 0x7faf28100f80 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2e0a8700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf28101920 0x7faf2819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2d8a7700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf280ffd90 0x7faf28196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:40.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2d0a6700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 0x7faf28197290 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf18000c00 tx=0x7faf18003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf1effd700 1 -- 172.21.15.39:0/1889777448 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1801d070 con 0x7faf28100f80 2024-05-24T11:37:40.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf1effd700 1 -- 172.21.15.39:0/1889777448 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf1800bd30 con 0x7faf28100f80 2024-05-24T11:37:40.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2819edd0 con 0x7faf28100f80 2024-05-24T11:37:40.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf1effd700 1 -- 172.21.15.39:0/1889777448 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18022dc0 con 0x7faf28100f80 2024-05-24T11:37:40.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.046+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2819f1a0 con 0x7faf28100f80 2024-05-24T11:37:40.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.047+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf28067380 con 0x7faf28100f80 2024-05-24T11:37:40.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.048+0000 7faf1effd700 1 -- 172.21.15.39:0/1889777448 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faf18003ff0 con 0x7faf28100f80 2024-05-24T11:37:40.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.048+0000 7faf1effd700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf140a11b0 0x7faf140a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.049+0000 7faf2d8a7700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf140a11b0 0x7faf140a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.049+0000 7faf1effd700 1 -- 172.21.15.39:0/1889777448 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf180cddd0 con 0x7faf28100f80 2024-05-24T11:37:40.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.049+0000 7faf2d8a7700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf140a11b0 0x7faf140a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf28069ee0 tx=0x7faf240092a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.053+0000 7faf1effd700 1 -- 172.21.15.39:0/1889777448 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf18097640 con 0x7faf28100f80 2024-05-24T11:37:40.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:39 smithi053 ceph-mon[47448]: pgmap v245: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-24T11:37:40.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:39 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2786695312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:40.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:39 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3047811024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:39 smithi137 ceph-mon[47907]: pgmap v245: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-24T11:37:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:39 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2786695312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:39 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3047811024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.307+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf2810b6f0 con 0x7faf28100f80 2024-05-24T11:37:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.308+0000 7faf1effd700 1 -- 172.21.15.39:0/1889777448 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10906 (secure 0 0 0) 0x7faf18027090 con 0x7faf28100f80 2024-05-24T11:37:40.313 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:40.313 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-24T11:36:48.500682+0000","modified":"2024-05-24T11:36:55.584150+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24511,"mds_2":24506,"mds_3":24490,"mds_4":14631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/3006971240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3006971240},{"type":"v1","addr":"172.21.15.39:6839","nonce":3006971240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/3024712815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3024712815},{"type":"v1","addr":"172.21.15.39:6841","nonce":3024712815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/218688120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":218688120},{"type":"v1","addr":"172.21.15.137:6839","nonce":218688120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3196235839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3196235839},{"type":"v1","addr":"172.21.15.53:6839","nonce":3196235839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf140a11b0 msgr2=0x7faf140a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf140a11b0 0x7faf140a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf28069ee0 tx=0x7faf240092a0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 msgr2=0x7faf28197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 0x7faf28197290 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf18000c00 tx=0x7faf18003b00 comp rx=0 tx=0).stop 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 shutdown_connections 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7faf140a11b0 0x7faf140a3670 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf28101920 0x7faf2819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf280ffd90 0x7faf28196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 --2- 172.21.15.39:0/1889777448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf28100f80 0x7faf28197290 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 >> 172.21.15.39:0/1889777448 conn(0x7faf280fb300 msgr2=0x7faf280fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 shutdown_connections 2024-05-24T11:37:40.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.310+0000 7faf2fb0b700 1 -- 172.21.15.39:0/1889777448 wait complete. 2024-05-24T11:37:40.317 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-24T11:37:40.328 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-24T11:36:48.500682+0000', 'modified': '2024-05-24T11:36:55.584150+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24542, 'mds_1': 24511, 'mds_2': 24506, 'mds_3': 24490, 'mds_4': 14631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6837/3631576511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 3631576511}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 3631576511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6839/3006971240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 3006971240}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 3006971240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6841/3024712815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 3024712815}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 3024712815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/451866213', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 451866213}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 451866213}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24476': {'gid': 24476, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/3629179598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 3629179598}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 3629179598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/1490077205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 1490077205}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 1490077205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.137:6837/967972436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 967972436}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 967972436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.53:6837/3764768463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 3764768463}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 3764768463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.137:6839/218688120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 218688120}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 218688120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6839/3196235839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 3196235839}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 3196235839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-24T11:37:40.328 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-24T11:37:40.329 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-24T11:37:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.451+0000 7f0def8c5700 1 -- 172.21.15.39:0/2997342659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de8100fb0 msgr2=0x7f0de8101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.451+0000 7f0def8c5700 1 --2- 172.21.15.39:0/2997342659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de8100fb0 0x7f0de8101430 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009990 tx=0x7f0dd8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.451+0000 7f0def8c5700 1 -- 172.21.15.39:0/2997342659 shutdown_connections 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.451+0000 7f0def8c5700 1 --2- 172.21.15.39:0/2997342659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de80ffe60 0x7f0de8100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.451+0000 7f0def8c5700 1 --2- 172.21.15.39:0/2997342659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0de8101970 0x7f0de8108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.451+0000 7f0def8c5700 1 --2- 172.21.15.39:0/2997342659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de8100fb0 0x7f0de8101430 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.451+0000 7f0def8c5700 1 -- 172.21.15.39:0/2997342659 >> 172.21.15.39:0/2997342659 conn(0x7f0de80fb3f0 msgr2=0x7f0de80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.451+0000 7f0def8c5700 1 -- 172.21.15.39:0/2997342659 shutdown_connections 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.451+0000 7f0def8c5700 1 -- 172.21.15.39:0/2997342659 wait complete. 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0def8c5700 1 Processor -- start 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0def8c5700 1 -- start start 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0def8c5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de80ffe60 0x7f0de806a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0def8c5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0de8100fb0 0x7f0de819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0def8c5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8101970 0x7f0de819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0def8c5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de819e910 con 0x7f0de80ffe60 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0def8c5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de819ea80 con 0x7f0de8100fb0 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0def8c5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de819ec40 con 0x7f0de8101970 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0ded661700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de80ffe60 0x7f0de806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0ded661700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de80ffe60 0x7f0de806a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49772/0 (socket says 172.21.15.39:49772) 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0ded661700 1 -- 172.21.15.39:0/3619375806 learned_addr learned my addr 172.21.15.39:0/3619375806 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0dece60700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0de8100fb0 0x7f0de819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0dede62700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8101970 0x7f0de819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0ded661700 1 -- 172.21.15.39:0/3619375806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8101970 msgr2=0x7f0de819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0ded661700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8101970 0x7f0de819e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0ded661700 1 -- 172.21.15.39:0/3619375806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0de8100fb0 msgr2=0x7f0de819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0ded661700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0de8100fb0 0x7f0de819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0ded661700 1 -- 172.21.15.39:0/3619375806 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd80095f0 con 0x7f0de80ffe60 2024-05-24T11:37:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0dede62700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8101970 0x7f0de819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.452+0000 7f0ded661700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de80ffe60 0x7f0de806a010 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c920 tx=0x7f0de400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.453+0000 7f0dde7fc700 1 -- 172.21.15.39:0/3619375806 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4007b00 con 0x7f0de80ffe60 2024-05-24T11:37:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.453+0000 7f0dde7fc700 1 -- 172.21.15.39:0/3619375806 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0de400ce70 con 0x7f0de80ffe60 2024-05-24T11:37:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.453+0000 7f0def8c5700 1 -- 172.21.15.39:0/3619375806 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de819ef20 con 0x7f0de80ffe60 2024-05-24T11:37:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.453+0000 7f0dde7fc700 1 -- 172.21.15.39:0/3619375806 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4005440 con 0x7f0de80ffe60 2024-05-24T11:37:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.453+0000 7f0def8c5700 1 -- 172.21.15.39:0/3619375806 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de819f3a0 con 0x7f0de80ffe60 2024-05-24T11:37:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.455+0000 7f0dde7fc700 1 -- 172.21.15.39:0/3619375806 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0de40055a0 con 0x7f0de80ffe60 2024-05-24T11:37:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.456+0000 7f0dde7fc700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0dd40a1230 0x7f0dd40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.456+0000 7f0dece60700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0dd40a1230 0x7f0dd40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.456+0000 7f0dde7fc700 1 -- 172.21.15.39:0/3619375806 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0de40cc420 con 0x7f0de80ffe60 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.456+0000 7f0dece60700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0dd40a1230 0x7f0dd40a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8006010 tx=0x7f0dd800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dfba80b80 1 Processor -- start 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dfba80b80 1 -- start start 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 0x557f843d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6c20 0x557f843d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f84328100 con 0x557f843d6c20 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f84325ac0 con 0x557f843d5900 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f843213b0 con 0x557f843d6260 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dede62700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6c20 0x557f843d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dede62700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6c20 0x557f843d70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49784/0 (socket says 172.21.15.39:49784) 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dede62700 1 -- 172.21.15.39:0/3117248068 learned_addr learned my addr 172.21.15.39:0/3117248068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0ded661700 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dece60700 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 0x557f843d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dede62700 1 -- 172.21.15.39:0/3117248068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 msgr2=0x557f843d66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dede62700 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 0x557f843d66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dede62700 1 -- 172.21.15.39:0/3117248068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 msgr2=0x557f843d5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dede62700 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843d5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dede62700 1 -- 172.21.15.39:0/3117248068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f841a2360 con 0x557f843d6c20 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.457+0000 7f0dece60700 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 0x557f843d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dede62700 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6c20 0x557f843d70a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009440 tx=0x7f0de0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356a9fe080a2d27c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dd37fe700 1 -- 172.21.15.39:0/3117248068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de000e070 con 0x557f843d6c20 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dd37fe700 1 -- 172.21.15.39:0/3117248068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0de000f040 con 0x557f843d6c20 2024-05-24T11:37:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dd37fe700 1 -- 172.21.15.39:0/3117248068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0015b10 con 0x557f843d6c20 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dfba80b80 1 -- 172.21.15.39:0/3117248068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6c20 msgr2=0x557f843d70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6c20 0x557f843d70a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009440 tx=0x7f0de0009920 comp rx=0 tx=0).stop 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dfba80b80 1 -- 172.21.15.39:0/3117248068 shutdown_connections 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6c20 0x557f843d70a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843d5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/3117248068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 0x557f843d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dfba80b80 1 -- 172.21.15.39:0/3117248068 >> 172.21.15.39:0/3117248068 conn(0x557f8432f1c0 msgr2=0x557f843cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dfba80b80 1 -- 172.21.15.39:0/3117248068 shutdown_connections 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.458+0000 7f0dfba80b80 1 -- 172.21.15.39:0/3117248068 wait complete. 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dfba80b80 1 -- start start 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dfba80b80 1 Processor -- start 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843df180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843e6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f842bd510 con 0x557f843d6260 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f842bd1b0 con 0x557f843d5900 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f841a4650 con 0x557f843d6c20 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dfba80b80 10 client.0 _ll_register_callbacks cb 0x557f842c6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dece60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dece60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843df6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49794/0 (socket says 172.21.15.39:49794) 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.459+0000 7f0dece60700 1 -- 172.21.15.39:0/2682964958 learned_addr learned my addr 172.21.15.39:0/2682964958 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0ded661700 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dede62700 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dece60700 1 -- 172.21.15.39:0/2682964958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 msgr2=0x557f843e6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dece60700 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843e6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dece60700 1 -- 172.21.15.39:0/2682964958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 msgr2=0x557f843df180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dece60700 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843df180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dece60700 1 -- 172.21.15.39:0/2682964958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de00090f0 con 0x557f843d6260 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dede62700 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dece60700 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843df6c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8015000 tx=0x7f0dd801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bd2bee2c2bd07f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/2682964958 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8004500 con 0x557f843d6260 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/2682964958 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0dd8004660 con 0x557f843d6260 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2682964958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f8427a220 con 0x557f843d6260 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/2682964958 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dd8004820 con 0x557f843d6260 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dfba80b80 10 client.15057 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2682964958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f8414eb10 con 0x557f843d6260 2024-05-24T11:37:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.460+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/2682964958 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557f8427a220 con 0x557f843d6260 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/2682964958 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0dd800f9c0 con 0x557f843d6260 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0dfba80b80 10 client.15057 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0dfba80b80 10 client.15057 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0dfba80b80 20 client.15057 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x557f843ea190 0x557f843ef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0dfba80b80 4 client.15057 mds_command: new command op to 14631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2682964958 --> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f843a07c0 con 0x557f843ea190 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0ded661700 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x557f843ea190 0x557f843ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0dd0ff9700 10 client.15057 ms_handle_connect on v2:172.21.15.39:6838/3006971240 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.461+0000 7f0ded661700 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x557f843ea190 0x557f843ef490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6477683ea4ddccdd server_cookie=15a42bc643db92bb in_seq=0 out_seq=0 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/2682964958 <== mds.4 v2:172.21.15.39:6838/3006971240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f843a07c0 con 0x557f843ea190 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dd0ff9700 10 client.15057 handle_command_reply: tid=0 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 client.15057 shutdown 2024-05-24T11:37:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2682964958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 msgr2=0x557f843df6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843df6c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8015000 tx=0x7f0dd801a050 comp rx=0 tx=0).stop 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2682964958 shutdown_connections 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x557f843ea190 0x557f843ef490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dd0ff9700 0 client.15057 ms_handle_reset on v2:172.21.15.39:6838/3006971240 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843df180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2682964958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843df6c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2682964958 >> 172.21.15.39:0/2682964958 conn(0x557f8432f1c0 msgr2=0x557f843d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2682964958 shutdown_connections 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2682964958 wait complete. 2024-05-24T11:37:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.462+0000 7f0dfba80b80 20 client.15057 trim_cache size 0 max 16384 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.480+0000 7f0dfba80b80 1 Processor -- start 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.480+0000 7f0dfba80b80 1 -- start start 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843de700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8431a150 con 0x557f843d6260 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8431e010 con 0x557f843d5900 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f843cd520 con 0x557f843d6c20 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dece60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dece60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843de700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49798/0 (socket says 172.21.15.39:49798) 2024-05-24T11:37:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dece60700 1 -- 172.21.15.39:0/2659307473 learned_addr learned my addr 172.21.15.39:0/2659307473 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dece60700 1 -- 172.21.15.39:0/2659307473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 msgr2=0x557f843cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dede62700 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0ded661700 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dece60700 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dece60700 1 -- 172.21.15.39:0/2659307473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 msgr2=0x557f843d5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dece60700 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843d5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dece60700 1 -- 172.21.15.39:0/2659307473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f842e7510 con 0x557f843d6260 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dede62700 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dece60700 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843de700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800b2c0 tx=0x7f0dd8009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ddf34e8facdcab server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dd3fff700 1 -- 172.21.15.39:0/2659307473 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800b990 con 0x557f843d6260 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dd3fff700 1 -- 172.21.15.39:0/2659307473 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0dd8036020 con 0x557f843d6260 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.481+0000 7f0dd3fff700 1 -- 172.21.15.39:0/2659307473 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd802fb10 con 0x557f843d6260 2024-05-24T11:37:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2659307473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 msgr2=0x557f843de700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843de700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800b2c0 tx=0x7f0dd8009290 comp rx=0 tx=0).stop 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2659307473 shutdown_connections 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6c20 0x557f843cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d5900 0x557f843d5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/2659307473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d6260 0x557f843de700 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2659307473 >> 172.21.15.39:0/2659307473 conn(0x557f8432f1c0 msgr2=0x557f843c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2659307473 shutdown_connections 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 -- 172.21.15.39:0/2659307473 wait complete. 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 -- start start 2024-05-24T11:37:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.482+0000 7f0dfba80b80 1 Processor -- start 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d5900 0x557f843e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 0x557f843e1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dfba80b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d6c20 0x557f843e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f843e6470 con 0x557f843d5900 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f843e65e0 con 0x557f843d6c20 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dfba80b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f843e6750 con 0x557f843d6260 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dfba80b80 10 client.0 _ll_register_callbacks cb 0x557f842c6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0ded661700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d5900 0x557f843e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0ded661700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d5900 0x557f843e0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49806/0 (socket says 172.21.15.39:49806) 2024-05-24T11:37:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0ded661700 1 -- 172.21.15.39:0/1326596839 learned_addr learned my addr 172.21.15.39:0/1326596839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dede62700 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d6c20 0x557f843e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dece60700 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 0x557f843e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0ded661700 1 -- 172.21.15.39:0/1326596839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 msgr2=0x557f843e1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0ded661700 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 0x557f843e1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0ded661700 1 -- 172.21.15.39:0/1326596839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d6c20 msgr2=0x557f843e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0ded661700 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d6c20 0x557f843e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0ded661700 1 -- 172.21.15.39:0/1326596839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f8414eb10 con 0x557f843d5900 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dede62700 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d6c20 0x557f843e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0ded661700 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d5900 0x557f843e0c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c8f0 tx=0x7f0de409a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f964ce4eaccc16e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.483+0000 7f0dd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/1326596839 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de40b2290 con 0x557f843d5900 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/1326596839 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0de409d6f0 con 0x557f843d5900 2024-05-24T11:37:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dfba80b80 1 -- 172.21.15.39:0/1326596839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0de000f040 con 0x557f843d5900 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/1326596839 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0de40a1d40 con 0x557f843d5900 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dfba80b80 10 client.15063 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dfba80b80 1 -- 172.21.15.39:0/1326596839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f841a6c80 con 0x557f843d5900 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/1326596839 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0de000f040 con 0x557f843d5900 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/1326596839 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0de4098db0 con 0x557f843d5900 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dfba80b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dfba80b80 10 client.15063 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dfba80b80 20 client.15063 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x557f84445820 0x557f84442170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dfba80b80 4 client.15063 mds_command: new command op to 14631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dfba80b80 1 -- 172.21.15.39:0/1326596839 --> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557f84319e70 con 0x557f84445820 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.484+0000 7f0dece60700 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x557f84445820 0x557f84442170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dece60700 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x557f84445820 0x557f84442170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5094e83bfde32e99 server_cookie=a12dfe6f20810b9b in_seq=0 out_seq=0 2024-05-24T11:37:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dd0ff9700 10 client.15063 ms_handle_connect on v2:172.21.15.39:6838/3006971240 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dd0ff9700 1 -- 172.21.15.39:0/1326596839 <== mds.4 v2:172.21.15.39:6838/3006971240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557f84319e70 con 0x557f84445820 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dd0ff9700 10 client.15063 handle_command_reply: tid=0 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 client.15063 shutdown 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 -- 172.21.15.39:0/1326596839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d5900 msgr2=0x557f843e0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d5900 0x557f843e0c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c8f0 tx=0x7f0de409a070 comp rx=0 tx=0).stop 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 -- 172.21.15.39:0/1326596839 shutdown_connections 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x557f84445820 0x557f84442170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f843d6c20 0x557f843e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f843d5900 0x557f843e0c70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 --2- 172.21.15.39:0/1326596839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f843d6260 0x557f843e1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dd0ff9700 0 client.15063 ms_handle_reset on v2:172.21.15.39:6838/3006971240 2024-05-24T11:37:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 -- 172.21.15.39:0/1326596839 >> 172.21.15.39:0/1326596839 conn(0x557f8432f1c0 msgr2=0x557f843c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 -- 172.21.15.39:0/1326596839 shutdown_connections 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 1 -- 172.21.15.39:0/1326596839 wait complete. 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.485+0000 7f0dfba80b80 20 client.15063 trim_cache size 0 max 16384 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 14631, 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:40.498 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:40.499 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 45.906415301000003, 2024-05-24T11:37:40.499 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:40.499 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 77, 2024-05-24T11:37:40.499 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:40.499 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 61.335347290000001 2024-05-24T11:37:40.499 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 -- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0dd40a1230 msgr2=0x7f0dd40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0dd40a1230 0x7f0dd40a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8006010 tx=0x7f0dd800b5b0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 -- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de80ffe60 msgr2=0x7f0de806a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de80ffe60 0x7f0de806a010 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c920 tx=0x7f0de400cc30 comp rx=0 tx=0).stop 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 -- 172.21.15.39:0/3619375806 shutdown_connections 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0dd40a1230 0x7f0dd40a36f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0de8101970 0x7f0de819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0de80ffe60 0x7f0de806a010 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 --2- 172.21.15.39:0/3619375806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0de8100fb0 0x7f0de819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 -- 172.21.15.39:0/3619375806 >> 172.21.15.39:0/3619375806 conn(0x7f0de80fb3f0 msgr2=0x7f0de81044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 -- 172.21.15.39:0/3619375806 shutdown_connections 2024-05-24T11:37:40.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.486+0000 7f0def8c5700 1 -- 172.21.15.39:0/3619375806 wait complete. 2024-05-24T11:37:40.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-24T11:37:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.626+0000 7fe433046700 1 -- 172.21.15.39:0/231903792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c101970 msgr2=0x7fe42c108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.626+0000 7fe433046700 1 --2- 172.21.15.39:0/231903792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c101970 0x7fe42c108290 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe42400b0a0 tx=0x7fe42400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.626+0000 7fe433046700 1 -- 172.21.15.39:0/231903792 shutdown_connections 2024-05-24T11:37:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.626+0000 7fe433046700 1 --2- 172.21.15.39:0/231903792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe42c0ffe60 0x7fe42c100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.626+0000 7fe433046700 1 --2- 172.21.15.39:0/231903792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c101970 0x7fe42c108290 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.626+0000 7fe433046700 1 --2- 172.21.15.39:0/231903792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe42c100fb0 0x7fe42c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.626+0000 7fe433046700 1 -- 172.21.15.39:0/231903792 >> 172.21.15.39:0/231903792 conn(0x7fe42c0fb3f0 msgr2=0x7fe42c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.626+0000 7fe433046700 1 -- 172.21.15.39:0/231903792 shutdown_connections 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.626+0000 7fe433046700 1 -- 172.21.15.39:0/231903792 wait complete. 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe433046700 1 Processor -- start 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe433046700 1 -- start start 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe433046700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe42c0ffe60 0x7fe42c196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe433046700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c100fb0 0x7fe42c197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe433046700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe42c101970 0x7fe42c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe433046700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c19e9f0 con 0x7fe42c100fb0 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe433046700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c19eb30 con 0x7fe42c0ffe60 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe433046700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c19ecc0 con 0x7fe42c101970 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe42bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c100fb0 0x7fe42c197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe430de2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe42c0ffe60 0x7fe42c196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe42bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c100fb0 0x7fe42c197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49820/0 (socket says 172.21.15.39:49820) 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe42bfff700 1 -- 172.21.15.39:0/4115344790 learned_addr learned my addr 172.21.15.39:0/4115344790 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe4315e3700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe42c101970 0x7fe42c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe42bfff700 1 -- 172.21.15.39:0/4115344790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe42c101970 msgr2=0x7fe42c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe42bfff700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe42c101970 0x7fe42c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe42bfff700 1 -- 172.21.15.39:0/4115344790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe42c0ffe60 msgr2=0x7fe42c196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe42bfff700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe42c0ffe60 0x7fe42c196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe42bfff700 1 -- 172.21.15.39:0/4115344790 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424009d00 con 0x7fe42c100fb0 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe430de2700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe42c0ffe60 0x7fe42c196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.627+0000 7fe42bfff700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c100fb0 0x7fe42c197400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe41800d950 tx=0x7fe41800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.628+0000 7fe429ffb700 1 -- 172.21.15.39:0/4115344790 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41800f890 con 0x7fe42c100fb0 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.628+0000 7fe429ffb700 1 -- 172.21.15.39:0/4115344790 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe41800fe80 con 0x7fe42c100fb0 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.628+0000 7fe429ffb700 1 -- 172.21.15.39:0/4115344790 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41800e3b0 con 0x7fe42c100fb0 2024-05-24T11:37:40.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.628+0000 7fe433046700 1 -- 172.21.15.39:0/4115344790 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c19efa0 con 0x7fe42c100fb0 2024-05-24T11:37:40.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.628+0000 7fe433046700 1 -- 172.21.15.39:0/4115344790 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c19f3c0 con 0x7fe42c100fb0 2024-05-24T11:37:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.630+0000 7fe429ffb700 1 -- 172.21.15.39:0/4115344790 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe418010460 con 0x7fe42c100fb0 2024-05-24T11:37:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.631+0000 7fe429ffb700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe41c0a11b0 0x7fe41c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.631+0000 7fe429ffb700 1 -- 172.21.15.39:0/4115344790 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe4180cc770 con 0x7fe42c100fb0 2024-05-24T11:37:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.631+0000 7fe430de2700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe41c0a11b0 0x7fe41c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.631+0000 7fe430de2700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe41c0a11b0 0x7fe41c0a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe420005950 tx=0x7fe42000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe43f201b80 1 Processor -- start 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe43f201b80 1 -- start start 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12be50 0x563c6c12c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 0x563c6c12cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe43f201b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c082100 con 0x563c6c12b4f0 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe43f201b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c07fac0 con 0x563c6c12be50 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe43f201b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c07b3b0 con 0x563c6c12c810 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe430de2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe430de2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49822/0 (socket says 172.21.15.39:49822) 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe430de2700 1 -- 172.21.15.39:0/1133522774 learned_addr learned my addr 172.21.15.39:0/1133522774 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe42bfff700 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12be50 0x563c6c12c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe430de2700 1 -- 172.21.15.39:0/1133522774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 msgr2=0x563c6c12cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe4315e3700 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 0x563c6c12cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe430de2700 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 0x563c6c12cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe430de2700 1 -- 172.21.15.39:0/1133522774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12be50 msgr2=0x563c6c12c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe430de2700 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12be50 0x563c6c12c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe430de2700 1 -- 172.21.15.39:0/1133522774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c6befc360 con 0x563c6c12b4f0 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe4315e3700 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 0x563c6c12cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe42bfff700 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12be50 0x563c6c12c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.632+0000 7fe430de2700 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b910 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe420011790 tx=0x7fe420013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96f59fbf53eb1d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe416ffd700 1 -- 172.21.15.39:0/1133522774 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42001e620 con 0x563c6c12b4f0 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe416ffd700 1 -- 172.21.15.39:0/1133522774 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe420015070 con 0x563c6c12b4f0 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe416ffd700 1 -- 172.21.15.39:0/1133522774 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4200269a0 con 0x563c6c12b4f0 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe43f201b80 1 -- 172.21.15.39:0/1133522774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 msgr2=0x563c6c12b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe43f201b80 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b910 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe420011790 tx=0x7fe420013000 comp rx=0 tx=0).stop 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe43f201b80 1 -- 172.21.15.39:0/1133522774 shutdown_connections 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe43f201b80 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 0x563c6c12cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe43f201b80 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b910 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe43f201b80 1 --2- 172.21.15.39:0/1133522774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12be50 0x563c6c12c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe43f201b80 1 -- 172.21.15.39:0/1133522774 >> 172.21.15.39:0/1133522774 conn(0x563c6c0891c0 msgr2=0x563c6c125d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe43f201b80 1 -- 172.21.15.39:0/1133522774 shutdown_connections 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.633+0000 7fe43f201b80 1 -- 172.21.15.39:0/1133522774 wait complete. 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe43f201b80 1 -- start start 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe43f201b80 1 Processor -- start 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 0x563c6c134de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12be50 0x563c6c135320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12c810 0x563c6c13c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe43f201b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c017510 con 0x563c6c12c810 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe43f201b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c0171b0 con 0x563c6c12b4f0 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe43f201b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6befe650 con 0x563c6c12be50 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe43f201b80 10 client.0 _ll_register_callbacks cb 0x563c6c020ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe4315e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12c810 0x563c6c13c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe4315e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12c810 0x563c6c13c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49830/0 (socket says 172.21.15.39:49830) 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe4315e3700 1 -- 172.21.15.39:0/2808102471 learned_addr learned my addr 172.21.15.39:0/2808102471 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.634+0000 7fe430de2700 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 0x563c6c134de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe42bfff700 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12be50 0x563c6c135320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe4315e3700 1 -- 172.21.15.39:0/2808102471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12be50 msgr2=0x563c6c135320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe4315e3700 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12be50 0x563c6c135320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe4315e3700 1 -- 172.21.15.39:0/2808102471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 msgr2=0x563c6c134de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe4315e3700 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 0x563c6c134de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe4315e3700 1 -- 172.21.15.39:0/2808102471 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe420011440 con 0x563c6c12c810 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe4315e3700 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12c810 0x563c6c13c360 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe424015000 tx=0x7fe424009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6460830b66f63162 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe40bfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe40bfff700 1 -- 172.21.15.39:0/2808102471 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42400e070 con 0x563c6c12c810 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe40bfff700 1 -- 172.21.15.39:0/2808102471 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe424003e10 con 0x563c6c12c810 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe43f201b80 1 -- 172.21.15.39:0/2808102471 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c6bfd4220 con 0x563c6c12c810 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe40bfff700 1 -- 172.21.15.39:0/2808102471 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe424006000 con 0x563c6c12c810 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe43f201b80 10 client.15075 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe43f201b80 1 -- 172.21.15.39:0/2808102471 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c6bea8b10 con 0x563c6c12c810 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe40bfff700 1 -- 172.21.15.39:0/2808102471 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563c6bfd4220 con 0x563c6c12c810 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe40bfff700 1 -- 172.21.15.39:0/2808102471 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe42400b0d0 con 0x563c6c12c810 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.635+0000 7fe43f201b80 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.636+0000 7fe43f201b80 10 client.15075 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-24T11:37:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.636+0000 7fe43f201b80 20 client.15075 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.636+0000 7fe43f201b80 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x563c6c13ff90 0x563c6c145290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.636+0000 7fe43f201b80 4 client.15075 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.636+0000 7fe43f201b80 1 -- 172.21.15.39:0/2808102471 --> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c6c0fa7c0 con 0x563c6c13ff90 2024-05-24T11:37:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.636+0000 7fe430de2700 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x563c6c13ff90 0x563c6c145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.636+0000 7fe40bfff700 10 client.15075 ms_handle_connect on v2:172.21.15.137:6836/967972436 2024-05-24T11:37:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.636+0000 7fe430de2700 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x563c6c13ff90 0x563c6c145290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3bab4ae866d96136 server_cookie=15d7dc63c62d38f7 in_seq=0 out_seq=0 2024-05-24T11:37:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe40bfff700 1 -- 172.21.15.39:0/2808102471 <== mds.3 v2:172.21.15.137:6836/967972436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c6c0fa7c0 con 0x563c6c13ff90 2024-05-24T11:37:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe40bfff700 10 client.15075 handle_command_reply: tid=0 2024-05-24T11:37:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 client.15075 shutdown 2024-05-24T11:37:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 -- 172.21.15.39:0/2808102471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12c810 msgr2=0x563c6c13c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12c810 0x563c6c13c360 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe424015000 tx=0x7fe424009300 comp rx=0 tx=0).stop 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 -- 172.21.15.39:0/2808102471 shutdown_connections 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12c810 0x563c6c13c360 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 0x563c6c134de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x563c6c13ff90 0x563c6c145290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 --2- 172.21.15.39:0/2808102471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12be50 0x563c6c135320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe40bfff700 0 client.15075 ms_handle_reset on v2:172.21.15.137:6836/967972436 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 -- 172.21.15.39:0/2808102471 >> 172.21.15.39:0/2808102471 conn(0x563c6c0891c0 msgr2=0x563c6c125550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 -- 172.21.15.39:0/2808102471 shutdown_connections 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.637+0000 7fe43f201b80 1 -- 172.21.15.39:0/2808102471 wait complete. 2024-05-24T11:37:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.638+0000 7fe43f201b80 20 client.15075 trim_cache size 0 max 16384 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.656+0000 7fe43f201b80 1 Processor -- start 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.656+0000 7fe43f201b80 1 -- start start 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.656+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.656+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12be50 0x563c6c13ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.656+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12c810 0x563c6c1342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.656+0000 7fe43f201b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c078010 con 0x563c6c12b4f0 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.656+0000 7fe43f201b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c01a5a0 con 0x563c6c12c810 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.656+0000 7fe43f201b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c134950 con 0x563c6c12be50 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe430de2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe430de2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49840/0 (socket says 172.21.15.39:49840) 2024-05-24T11:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe430de2700 1 -- 172.21.15.39:0/1928464608 learned_addr learned my addr 172.21.15.39:0/1928464608 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe430de2700 1 -- 172.21.15.39:0/1928464608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12be50 msgr2=0x563c6c13ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe430de2700 1 --2- 172.21.15.39:0/1928464608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12be50 0x563c6c13ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe430de2700 1 -- 172.21.15.39:0/1928464608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12c810 msgr2=0x563c6c1342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe430de2700 1 --2- 172.21.15.39:0/1928464608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12c810 0x563c6c1342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe430de2700 1 -- 172.21.15.39:0/1928464608 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c6c041510 con 0x563c6c12b4f0 2024-05-24T11:37:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe430de2700 1 --2- 172.21.15.39:0/1928464608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b8d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe420011790 tx=0x7fe42001ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f7e49bb3a547e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe4177fe700 1 -- 172.21.15.39:0/1928464608 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42002b040 con 0x563c6c12b4f0 2024-05-24T11:37:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe4177fe700 1 -- 172.21.15.39:0/1928464608 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe42002c040 con 0x563c6c12b4f0 2024-05-24T11:37:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.657+0000 7fe4177fe700 1 -- 172.21.15.39:0/1928464608 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420034070 con 0x563c6c12b4f0 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 -- 172.21.15.39:0/1928464608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 msgr2=0x563c6c12b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 --2- 172.21.15.39:0/1928464608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b8d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe420011790 tx=0x7fe42001ee70 comp rx=0 tx=0).stop 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 -- 172.21.15.39:0/1928464608 shutdown_connections 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 --2- 172.21.15.39:0/1928464608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12c810 0x563c6c1342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 --2- 172.21.15.39:0/1928464608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12b4f0 0x563c6c12b8d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 --2- 172.21.15.39:0/1928464608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12be50 0x563c6c13ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 -- 172.21.15.39:0/1928464608 >> 172.21.15.39:0/1928464608 conn(0x563c6c0891c0 msgr2=0x563c6c12cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 -- 172.21.15.39:0/1928464608 shutdown_connections 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 -- 172.21.15.39:0/1928464608 wait complete. 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 -- start start 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.658+0000 7fe43f201b80 1 Processor -- start 2024-05-24T11:37:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 0x563c6c1320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12be50 0x563c6c13c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe43f201b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 0x563c6c1307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe43f201b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c130ce0 con 0x563c6c12be50 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe43f201b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c130e50 con 0x563c6c12b4f0 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe43f201b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6c131010 con 0x563c6c12c810 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe43f201b80 10 client.0 _ll_register_callbacks cb 0x563c6c020ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe42bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12be50 0x563c6c13c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe42bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12be50 0x563c6c13c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49846/0 (socket says 172.21.15.39:49846) 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe42bfff700 1 -- 172.21.15.39:0/4109221202 learned_addr learned my addr 172.21.15.39:0/4109221202 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe430de2700 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 0x563c6c1320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe4315e3700 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 0x563c6c1307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe42bfff700 1 -- 172.21.15.39:0/4109221202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 msgr2=0x563c6c1307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe42bfff700 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 0x563c6c1307a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe42bfff700 1 -- 172.21.15.39:0/4109221202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 msgr2=0x563c6c1320c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe42bfff700 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 0x563c6c1320c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe42bfff700 1 -- 172.21.15.39:0/4109221202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c6befc360 con 0x563c6c12be50 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe42bfff700 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12be50 0x563c6c13c280 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe41800b2f0 tx=0x7fe418010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb59bc7cf36836ae server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe40bfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe40bfff700 1 -- 172.21.15.39:0/4109221202 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418099200 con 0x563c6c12be50 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe40bfff700 1 -- 172.21.15.39:0/4109221202 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe418099b10 con 0x563c6c12be50 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe43f201b80 1 -- 172.21.15.39:0/4109221202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe420015070 con 0x563c6c12be50 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.659+0000 7fe40bfff700 1 -- 172.21.15.39:0/4109221202 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe41809d940 con 0x563c6c12be50 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe43f201b80 10 client.15081 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe43f201b80 1 -- 172.21.15.39:0/4109221202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c6bf00c80 con 0x563c6c12be50 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe40bfff700 1 -- 172.21.15.39:0/4109221202 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe420015070 con 0x563c6c12be50 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe40bfff700 1 -- 172.21.15.39:0/4109221202 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe4180a1b90 con 0x563c6c12be50 2024-05-24T11:37:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe43f201b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe43f201b80 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe43f201b80 20 client.15081 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe43f201b80 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x563c6c19b420 0x563c6c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe43f201b80 4 client.15081 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe43f201b80 1 -- 172.21.15.39:0/4109221202 --> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563c6c073e70 con 0x563c6c19b420 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.660+0000 7fe430de2700 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x563c6c19b420 0x563c6c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe40bfff700 10 client.15081 ms_handle_connect on v2:172.21.15.137:6836/967972436 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe430de2700 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x563c6c19b420 0x563c6c197e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38dffc008eb3d748 server_cookie=53096b8ed2c4a01c in_seq=0 out_seq=0 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe40bfff700 1 -- 172.21.15.39:0/4109221202 <== mds.3 v2:172.21.15.137:6836/967972436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563c6c073e70 con 0x563c6c19b420 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe40bfff700 10 client.15081 handle_command_reply: tid=0 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 client.15081 shutdown 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 -- 172.21.15.39:0/4109221202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12be50 msgr2=0x563c6c13c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12be50 0x563c6c13c280 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe41800b2f0 tx=0x7fe418010850 comp rx=0 tx=0).stop 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 -- 172.21.15.39:0/4109221202 shutdown_connections 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x563c6c19b420 0x563c6c197e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c6c12c810 0x563c6c1307a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe40bfff700 0 client.15081 ms_handle_reset on v2:172.21.15.137:6836/967972436 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563c6c12b4f0 0x563c6c1320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 --2- 172.21.15.39:0/4109221202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6c12be50 0x563c6c13c280 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 -- 172.21.15.39:0/4109221202 >> 172.21.15.39:0/4109221202 conn(0x563c6c0891c0 msgr2=0x563c6c132d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 -- 172.21.15.39:0/4109221202 shutdown_connections 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 1 -- 172.21.15.39:0/4109221202 wait complete. 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.661+0000 7fe43f201b80 20 client.15081 trim_cache size 0 max 16384 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24490, 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 47.054677986999998, 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 77, 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 67.110043407000006 2024-05-24T11:37:40.674 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 -- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe41c0a11b0 msgr2=0x7fe41c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe41c0a11b0 0x7fe41c0a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe420005950 tx=0x7fe42000b410 comp rx=0 tx=0).stop 2024-05-24T11:37:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 -- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c100fb0 msgr2=0x7fe42c197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c100fb0 0x7fe42c197400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe41800d950 tx=0x7fe41800dc60 comp rx=0 tx=0).stop 2024-05-24T11:37:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 -- 172.21.15.39:0/4115344790 shutdown_connections 2024-05-24T11:37:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe41c0a11b0 0x7fe41c0a3670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe42c101970 0x7fe42c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe42c0ffe60 0x7fe42c196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 --2- 172.21.15.39:0/4115344790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe42c100fb0 0x7fe42c197400 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 -- 172.21.15.39:0/4115344790 >> 172.21.15.39:0/4115344790 conn(0x7fe42c0fb3f0 msgr2=0x7fe42c104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 -- 172.21.15.39:0/4115344790 shutdown_connections 2024-05-24T11:37:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.662+0000 7fe433046700 1 -- 172.21.15.39:0/4115344790 wait complete. 2024-05-24T11:37:40.680 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-24T11:37:40.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 -- 172.21.15.39:0/691769905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060101990 msgr2=0x7f00601082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 --2- 172.21.15.39:0/691769905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060101990 0x7f00601082b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f005800b0a0 tx=0x7f005800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 -- 172.21.15.39:0/691769905 shutdown_connections 2024-05-24T11:37:40.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 --2- 172.21.15.39:0/691769905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00600ffe80 0x7f00601002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 --2- 172.21.15.39:0/691769905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060101990 0x7f00601082b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 --2- 172.21.15.39:0/691769905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0060100fd0 0x7f0060101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 -- 172.21.15.39:0/691769905 >> 172.21.15.39:0/691769905 conn(0x7f00600fb3f0 msgr2=0x7f00600fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 -- 172.21.15.39:0/691769905 shutdown_connections 2024-05-24T11:37:40.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 -- 172.21.15.39:0/691769905 wait complete. 2024-05-24T11:37:40.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 Processor -- start 2024-05-24T11:37:40.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.802+0000 7f0066575700 1 -- start start 2024-05-24T11:37:40.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f0066575700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00600ffe80 0x7f0060069fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f0066575700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060100fd0 0x7f006019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f0066575700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0060101990 0x7f006019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f0066575700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006019ea20 con 0x7f0060100fd0 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f0066575700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006019eb90 con 0x7f0060101990 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f0066575700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006019ed50 con 0x7f00600ffe80 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060100fd0 0x7f006019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060100fd0 0x7f006019bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49856/0 (socket says 172.21.15.39:49856) 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005f7fe700 1 -- 172.21.15.39:0/874431355 learned_addr learned my addr 172.21.15.39:0/874431355 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005ffff700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00600ffe80 0x7f0060069fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f0064b12700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0060101990 0x7f006019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005f7fe700 1 -- 172.21.15.39:0/874431355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00600ffe80 msgr2=0x7f0060069fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005f7fe700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00600ffe80 0x7f0060069fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005f7fe700 1 -- 172.21.15.39:0/874431355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0060101990 msgr2=0x7f006019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005f7fe700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0060101990 0x7f006019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005f7fe700 1 -- 172.21.15.39:0/874431355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0058009d00 con 0x7f0060100fd0 2024-05-24T11:37:40.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.803+0000 7f005f7fe700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060100fd0 0x7f006019bc30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f004800d950 tx=0x7f004800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.804+0000 7f005d7fa700 1 -- 172.21.15.39:0/874431355 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004800f890 con 0x7f0060100fd0 2024-05-24T11:37:40.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.804+0000 7f0066575700 1 -- 172.21.15.39:0/874431355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006019efd0 con 0x7f0060100fd0 2024-05-24T11:37:40.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.804+0000 7f005d7fa700 1 -- 172.21.15.39:0/874431355 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f004800fe80 con 0x7f0060100fd0 2024-05-24T11:37:40.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.804+0000 7f005d7fa700 1 -- 172.21.15.39:0/874431355 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004800e3b0 con 0x7f0060100fd0 2024-05-24T11:37:40.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.804+0000 7f0066575700 1 -- 172.21.15.39:0/874431355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006019f4b0 con 0x7f0060100fd0 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.806+0000 7f005d7fa700 1 -- 172.21.15.39:0/874431355 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0048010460 con 0x7f0060100fd0 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.806+0000 7f005d7fa700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f004c0a11b0 0x7f004c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.807+0000 7f005ffff700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f004c0a11b0 0x7f004c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.807+0000 7f005ffff700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f004c0a11b0 0x7f004c0a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0050005950 tx=0x7f005000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.807+0000 7f005d7fa700 1 -- 172.21.15.39:0/874431355 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0048010760 con 0x7f0060100fd0 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.807+0000 7f0072730b80 1 Processor -- start 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0072730b80 1 -- start start 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0072730b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b6a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0072730b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6ae50 0x557e21b6b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0072730b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6b810 0x557e21b6bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0072730b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21ac1100 con 0x557e21b6b810 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0072730b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21abeac0 con 0x557e21b6a4f0 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0072730b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21aba3b0 con 0x557e21b6ae50 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0064b12700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6b810 0x557e21b6bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f005ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b6a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0064b12700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6b810 0x557e21b6bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49866/0 (socket says 172.21.15.39:49866) 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0064b12700 1 -- 172.21.15.39:0/2505091996 learned_addr learned my addr 172.21.15.39:0/2505091996 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f005f7fe700 1 --2- 172.21.15.39:0/2505091996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6ae50 0x557e21b6b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f005ffff700 1 -- 172.21.15.39:0/2505091996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6ae50 msgr2=0x557e21b6b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f005ffff700 1 --2- 172.21.15.39:0/2505091996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6ae50 0x557e21b6b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f005ffff700 1 -- 172.21.15.39:0/2505091996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6b810 msgr2=0x557e21b6bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f005ffff700 1 --2- 172.21.15.39:0/2505091996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6b810 0x557e21b6bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f005ffff700 1 -- 172.21.15.39:0/2505091996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e2193b360 con 0x557e21b6a4f0 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.808+0000 7f0064b12700 1 --2- 172.21.15.39:0/2505091996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6b810 0x557e21b6bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:37:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.809+0000 7f005ffff700 1 --2- 172.21.15.39:0/2505091996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b6a910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f005000fcd0 tx=0x7f0050012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d5b143897ad633e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.809+0000 7f00567fc700 1 -- 172.21.15.39:0/2505091996 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00500107b0 con 0x557e21b6a4f0 2024-05-24T11:37:40.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.809+0000 7f00567fc700 1 -- 172.21.15.39:0/2505091996 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0050020070 con 0x557e21b6a4f0 2024-05-24T11:37:40.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.809+0000 7f00567fc700 1 -- 172.21.15.39:0/2505091996 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050025c00 con 0x557e21b6a4f0 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.809+0000 7f0072730b80 1 -- 172.21.15.39:0/2505091996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 msgr2=0x557e21b6a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.809+0000 7f0072730b80 1 --2- 172.21.15.39:0/2505091996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b6a910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f005000fcd0 tx=0x7f0050012000 comp rx=0 tx=0).stop 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 -- 172.21.15.39:0/2505091996 shutdown_connections 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 --2- 172.21.15.39:0/2505091996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6b810 0x557e21b6bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 --2- 172.21.15.39:0/2505091996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b6a910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 --2- 172.21.15.39:0/2505091996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6ae50 0x557e21b6b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 -- 172.21.15.39:0/2505091996 >> 172.21.15.39:0/2505091996 conn(0x557e21ac81c0 msgr2=0x557e21b64d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 -- 172.21.15.39:0/2505091996 shutdown_connections 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 -- 172.21.15.39:0/2505091996 wait complete. 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 -- start start 2024-05-24T11:37:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 Processor -- start 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 0x557e21b73df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b74330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 0x557e21b7b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21a56510 con 0x557e21b6ae50 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21a561b0 con 0x557e21b6b810 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2193d650 con 0x557e21b6a4f0 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.810+0000 7f0072730b80 10 client.0 _ll_register_callbacks cb 0x557e21a5fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b74330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b74330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49872/0 (socket says 172.21.15.39:49872) 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005f7fe700 1 -- 172.21.15.39:0/2694164800 learned_addr learned my addr 172.21.15.39:0/2694164800 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f0064b12700 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 0x557e21b7b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005ffff700 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 0x557e21b73df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005f7fe700 1 -- 172.21.15.39:0/2694164800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 msgr2=0x557e21b73df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005f7fe700 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 0x557e21b73df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005f7fe700 1 -- 172.21.15.39:0/2694164800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 msgr2=0x557e21b7b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005f7fe700 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 0x557e21b7b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005f7fe700 1 -- 172.21.15.39:0/2694164800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0060108be0 con 0x557e21b6ae50 2024-05-24T11:37:40.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f005f7fe700 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b74330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0048010710 tx=0x7f0048099be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a398d6ef802d64bb server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.811+0000 7f003f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:40.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f003f7fe700 1 -- 172.21.15.39:0/2694164800 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00480a1f90 con 0x557e21b6ae50 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f0072730b80 1 -- 172.21.15.39:0/2694164800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e21a13220 con 0x557e21b6ae50 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f003f7fe700 1 -- 172.21.15.39:0/2694164800 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f00480a1b00 con 0x557e21b6ae50 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f003f7fe700 1 -- 172.21.15.39:0/2694164800 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0048099470 con 0x557e21b6ae50 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f0072730b80 10 client.15093 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f0072730b80 1 -- 172.21.15.39:0/2694164800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f005000f980 con 0x557e21b6ae50 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f003f7fe700 1 -- 172.21.15.39:0/2694164800 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557e21a13220 con 0x557e21b6ae50 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f003f7fe700 1 -- 172.21.15.39:0/2694164800 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f004809ac40 con 0x557e21b6ae50 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f0072730b80 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f0072730b80 10 client.15093 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f0072730b80 20 client.15093 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f0072730b80 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x557e21b7efb0 0x557e21b84290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f0072730b80 4 client.15093 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:40.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.814+0000 7f0072730b80 1 -- 172.21.15.39:0/2694164800 --> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e21b397c0 con 0x557e21b7efb0 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.815+0000 7f005ffff700 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x557e21b7efb0 0x557e21b84290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.815+0000 7f003f7fe700 10 client.15093 ms_handle_connect on v2:172.21.15.53:6836/3764768463 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.815+0000 7f005ffff700 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x557e21b7efb0 0x557e21b84290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10714f603c5f6444 server_cookie=9c7d303b4be0cfc3 in_seq=0 out_seq=0 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f003f7fe700 1 -- 172.21.15.39:0/2694164800 <== mds.2 v2:172.21.15.53:6836/3764768463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e21b397c0 con 0x557e21b7efb0 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f003f7fe700 10 client.15093 handle_command_reply: tid=0 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 client.15093 shutdown 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 -- 172.21.15.39:0/2694164800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 msgr2=0x557e21b74330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b74330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0048010710 tx=0x7f0048099be0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 -- 172.21.15.39:0/2694164800 shutdown_connections 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 0x557e21b73df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x557e21b7efb0 0x557e21b84290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 0x557e21b7b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 --2- 172.21.15.39:0/2694164800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b74330 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f003f7fe700 0 client.15093 ms_handle_reset on v2:172.21.15.53:6836/3764768463 2024-05-24T11:37:40.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 -- 172.21.15.39:0/2694164800 >> 172.21.15.39:0/2694164800 conn(0x557e21ac81c0 msgr2=0x557e21b64550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 -- 172.21.15.39:0/2694164800 shutdown_connections 2024-05-24T11:37:40.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.816+0000 7f0072730b80 1 -- 172.21.15.39:0/2694164800 wait complete. 2024-05-24T11:37:40.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.817+0000 7f0072730b80 20 client.15093 trim_cache size 0 max 16384 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f0072730b80 1 Processor -- start 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f0072730b80 1 -- start start 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f0072730b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 0x557e21b6a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f0072730b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f0072730b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 0x557e21b73300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f0072730b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21b397c0 con 0x557e21b6ae50 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f0072730b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21a13220 con 0x557e21b6b810 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f0072730b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21b65b80 con 0x557e21b6a4f0 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49884/0 (socket says 172.21.15.39:49884) 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005f7fe700 1 -- 172.21.15.39:0/1467275494 learned_addr learned my addr 172.21.15.39:0/1467275494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005ffff700 1 --2- 172.21.15.39:0/1467275494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 0x557e21b6a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005f7fe700 1 -- 172.21.15.39:0/1467275494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 msgr2=0x557e21b6a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f0064b12700 1 --2- 172.21.15.39:0/1467275494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 0x557e21b73300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005f7fe700 1 --2- 172.21.15.39:0/1467275494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 0x557e21b6a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005f7fe700 1 -- 172.21.15.39:0/1467275494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 msgr2=0x557e21b73300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005f7fe700 1 --2- 172.21.15.39:0/1467275494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 0x557e21b73300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005f7fe700 1 -- 172.21.15.39:0/1467275494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e218e7b10 con 0x557e21b6ae50 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.835+0000 7f005f7fe700 1 --2- 172.21.15.39:0/1467275494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0048096580 tx=0x7f0048099a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bc2e63142476c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0056ffd700 1 -- 172.21.15.39:0/1467275494 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048014070 con 0x557e21b6ae50 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0056ffd700 1 -- 172.21.15.39:0/1467275494 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00480a1f90 con 0x557e21b6ae50 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0056ffd700 1 -- 172.21.15.39:0/1467275494 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00480a1d60 con 0x557e21b6ae50 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0072730b80 1 -- 172.21.15.39:0/1467275494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 msgr2=0x557e21b7b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0072730b80 1 --2- 172.21.15.39:0/1467275494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0048096580 tx=0x7f0048099a10 comp rx=0 tx=0).stop 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0072730b80 1 -- 172.21.15.39:0/1467275494 shutdown_connections 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0072730b80 1 --2- 172.21.15.39:0/1467275494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6a4f0 0x557e21b6a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0072730b80 1 --2- 172.21.15.39:0/1467275494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6b810 0x557e21b73300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0072730b80 1 --2- 172.21.15.39:0/1467275494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0072730b80 1 -- 172.21.15.39:0/1467275494 >> 172.21.15.39:0/1467275494 conn(0x557e21ac81c0 msgr2=0x557e21b6bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0072730b80 1 -- 172.21.15.39:0/1467275494 shutdown_connections 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.836+0000 7f0072730b80 1 -- 172.21.15.39:0/1467275494 wait complete. 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0072730b80 1 -- start start 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0072730b80 1 Processor -- start 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0072730b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b88c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0072730b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0072730b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6b810 0x557e21b63d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0072730b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21b64410 con 0x557e21b6ae50 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0072730b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21b64580 con 0x557e21b6a4f0 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0072730b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e21b64740 con 0x557e21b6b810 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0072730b80 10 client.0 _ll_register_callbacks cb 0x557e21a5fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49892/0 (socket says 172.21.15.39:49892) 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005f7fe700 1 -- 172.21.15.39:0/1748635880 learned_addr learned my addr 172.21.15.39:0/1748635880 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005ffff700 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b88c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0064b12700 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6b810 0x557e21b63d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005f7fe700 1 -- 172.21.15.39:0/1748635880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6b810 msgr2=0x557e21b63d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005f7fe700 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6b810 0x557e21b63d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005f7fe700 1 -- 172.21.15.39:0/1748635880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 msgr2=0x557e21b88c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005f7fe700 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b88c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005f7fe700 1 -- 172.21.15.39:0/1748635880 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005000f980 con 0x557e21b6ae50 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f005ffff700 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b88c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.837+0000 7f0064b12700 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6b810 0x557e21b63d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f005f7fe700 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0048099810 tx=0x7f0048099650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ae81b9a6d4b92b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f003f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f003f7fe700 1 -- 172.21.15.39:0/1748635880 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048014040 con 0x557e21b6ae50 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f003f7fe700 1 -- 172.21.15.39:0/1748635880 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f004809ab20 con 0x557e21b6ae50 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f0072730b80 1 -- 172.21.15.39:0/1748635880 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0050020070 con 0x557e21b6ae50 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f003f7fe700 1 -- 172.21.15.39:0/1748635880 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f004809ad20 con 0x557e21b6ae50 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f0072730b80 10 client.15099 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f0072730b80 1 -- 172.21.15.39:0/1748635880 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557e21a80510 con 0x557e21b6ae50 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f003f7fe700 1 -- 172.21.15.39:0/1748635880 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0050020070 con 0x557e21b6ae50 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f003f7fe700 1 -- 172.21.15.39:0/1748635880 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f004809e860 con 0x557e21b6ae50 2024-05-24T11:37:40.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f0072730b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f0072730b80 10 client.15099 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f0072730b80 20 client.15099 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f0072730b80 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x557e21bda2d0 0x557e21bd6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f0072730b80 4 client.15099 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.838+0000 7f0072730b80 1 -- 172.21.15.39:0/1748635880 --> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557e21ab2e70 con 0x557e21bda2d0 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f005ffff700 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x557e21bda2d0 0x557e21bd6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f005ffff700 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x557e21bda2d0 0x557e21bd6cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc93287dd04810e server_cookie=15cfabe312fd30a4 in_seq=0 out_seq=0 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f003f7fe700 10 client.15099 ms_handle_connect on v2:172.21.15.53:6836/3764768463 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f003f7fe700 1 -- 172.21.15.39:0/1748635880 <== mds.2 v2:172.21.15.53:6836/3764768463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557e21ab2e70 con 0x557e21bda2d0 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f003f7fe700 10 client.15099 handle_command_reply: tid=0 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f0072730b80 1 client.15099 shutdown 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f0072730b80 1 -- 172.21.15.39:0/1748635880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 msgr2=0x557e21b7b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f0072730b80 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0048099810 tx=0x7f0048099650 comp rx=0 tx=0).stop 2024-05-24T11:37:40.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f0072730b80 1 -- 172.21.15.39:0/1748635880 shutdown_connections 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f0072730b80 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557e21b6b810 0x557e21b63d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.839+0000 7f0072730b80 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x557e21bda2d0 0x557e21bd6cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0072730b80 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e21b6a4f0 0x557e21b88c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f003f7fe700 0 client.15099 ms_handle_reset on v2:172.21.15.53:6836/3764768463 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0072730b80 1 --2- 172.21.15.39:0/1748635880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e21b6ae50 0x557e21b7b330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0072730b80 1 -- 172.21.15.39:0/1748635880 >> 172.21.15.39:0/1748635880 conn(0x557e21ac81c0 msgr2=0x557e21b6e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0072730b80 1 -- 172.21.15.39:0/1748635880 shutdown_connections 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0072730b80 1 -- 172.21.15.39:0/1748635880 wait complete. 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0072730b80 20 client.15099 trim_cache size 0 max 16384 2024-05-24T11:37:40.852 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24506, 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 48.238836493999997, 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 77, 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 65.322187253999999 2024-05-24T11:37:40.853 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:40.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 -- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f004c0a11b0 msgr2=0x7f004c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f004c0a11b0 0x7f004c0a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0050005950 tx=0x7f005000a300 comp rx=0 tx=0).stop 2024-05-24T11:37:40.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 -- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060100fd0 msgr2=0x7f006019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060100fd0 0x7f006019bc30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f004800d950 tx=0x7f004800dc60 comp rx=0 tx=0).stop 2024-05-24T11:37:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 -- 172.21.15.39:0/874431355 shutdown_connections 2024-05-24T11:37:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f004c0a11b0 0x7f004c0a3670 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00600ffe80 0x7f0060069fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0060101990 0x7f006019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 --2- 172.21.15.39:0/874431355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060100fd0 0x7f006019bc30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 -- 172.21.15.39:0/874431355 >> 172.21.15.39:0/874431355 conn(0x7f00600fb3f0 msgr2=0x7f00601043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 -- 172.21.15.39:0/874431355 shutdown_connections 2024-05-24T11:37:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.840+0000 7f0066575700 1 -- 172.21.15.39:0/874431355 wait complete. 2024-05-24T11:37:40.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-24T11:37:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.981+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/1328161545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d340ffe60 msgr2=0x7f6d34100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.981+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/1328161545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d340ffe60 0x7f6d34100280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009990 tx=0x7f6d30009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.981+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/1328161545 shutdown_connections 2024-05-24T11:37:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.981+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/1328161545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d34101970 0x7f6d34108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.981+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/1328161545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d340ffe60 0x7f6d34100280 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.981+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/1328161545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d34100fb0 0x7f6d34101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.981+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/1328161545 >> 172.21.15.39:0/1328161545 conn(0x7f6d340fb3f0 msgr2=0x7f6d340fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:40.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.981+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/1328161545 shutdown_connections 2024-05-24T11:37:40.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.981+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/1328161545 wait complete. 2024-05-24T11:37:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3bc1d700 1 Processor -- start 2024-05-24T11:37:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3bc1d700 1 -- start start 2024-05-24T11:37:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3bc1d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d340ffe60 0x7f6d34195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3bc1d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d34100fb0 0x7f6d34196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3bc1d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d34101970 0x7f6d3419d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3bc1d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3419d830 con 0x7f6d34101970 2024-05-24T11:37:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3bc1d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3419d9a0 con 0x7f6d34100fb0 2024-05-24T11:37:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3bc1d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3419db60 con 0x7f6d340ffe60 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3a1ba700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d34101970 0x7f6d3419d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3a1ba700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d34101970 0x7f6d3419d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49910/0 (socket says 172.21.15.39:49910) 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d3a1ba700 1 -- 172.21.15.39:0/217312463 learned_addr learned my addr 172.21.15.39:0/217312463 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.982+0000 7f6d399b9700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d340ffe60 0x7f6d34195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d391b8700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d34100fb0 0x7f6d34196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d3a1ba700 1 -- 172.21.15.39:0/217312463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d340ffe60 msgr2=0x7f6d34195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d340ffe60 0x7f6d34195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d3a1ba700 1 -- 172.21.15.39:0/217312463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d34100fb0 msgr2=0x7f6d34196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d34100fb0 0x7f6d34196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d3a1ba700 1 -- 172.21.15.39:0/217312463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d300095f0 con 0x7f6d34101970 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d34101970 0x7f6d3419d260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c007ee0 tx=0x7f6d2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d2affd700 1 -- 172.21.15.39:0/217312463 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00ddd0 con 0x7f6d34101970 2024-05-24T11:37:40.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d2affd700 1 -- 172.21.15.39:0/217312463 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d2c00f040 con 0x7f6d34101970 2024-05-24T11:37:40.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/217312463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3419de40 con 0x7f6d34101970 2024-05-24T11:37:40.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d2affd700 1 -- 172.21.15.39:0/217312463 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0047d0 con 0x7f6d34101970 2024-05-24T11:37:40.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.983+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/217312463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3419e210 con 0x7f6d34101970 2024-05-24T11:37:40.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.985+0000 7f6d2affd700 1 -- 172.21.15.39:0/217312463 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6d2c003a00 con 0x7f6d34101970 2024-05-24T11:37:40.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.986+0000 7f6d2affd700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6d200a11b0 0x7f6d200a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.986+0000 7f6d399b9700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6d200a11b0 0x7f6d200a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.986+0000 7f6d2affd700 1 -- 172.21.15.39:0/217312463 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d2c0cc230 con 0x7f6d34101970 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.986+0000 7f6d399b9700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6d200a11b0 0x7f6d200a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000c010 tx=0x7f6d30003720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.986+0000 7f6d47dd8b80 1 Processor -- start 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d47dd8b80 1 -- start start 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca61951910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca61952c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca618a8100 con 0x55ca61951e50 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca618a5ac0 con 0x55ca61952810 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca618a13b0 con 0x55ca619514f0 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d391b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:40.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d391b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49918/0 (socket says 172.21.15.39:49918) 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d391b8700 1 -- 172.21.15.39:0/3233254739 learned_addr learned my addr 172.21.15.39:0/3233254739 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca61952c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d399b9700 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca61951910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d391b8700 1 -- 172.21.15.39:0/3233254739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 msgr2=0x55ca61951910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d391b8700 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca61951910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d391b8700 1 -- 172.21.15.39:0/3233254739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 msgr2=0x55ca61952c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d391b8700 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca61952c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d391b8700 1 -- 172.21.15.39:0/3233254739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca61722360 con 0x55ca61951e50 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca61952c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.987+0000 7f6d391b8700 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619522d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400bb10 tx=0x7f6d2400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804258320651638f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d1ffff700 1 -- 172.21.15.39:0/3233254739 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400dd00 con 0x55ca61951e50 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d1ffff700 1 -- 172.21.15.39:0/3233254739 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d2400de60 con 0x55ca61951e50 2024-05-24T11:37:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d1ffff700 1 -- 172.21.15.39:0/3233254739 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400f710 con 0x55ca61951e50 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3233254739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 msgr2=0x55ca619522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619522d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400bb10 tx=0x7f6d2400be20 comp rx=0 tx=0).stop 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3233254739 shutdown_connections 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca61951910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca61952c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3233254739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619522d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3233254739 >> 172.21.15.39:0/3233254739 conn(0x55ca618af1c0 msgr2=0x55ca6194bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3233254739 shutdown_connections 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.988+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3233254739 wait complete. 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d47dd8b80 1 -- start start 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d47dd8b80 1 Processor -- start 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca619514f0 0x55ca6195f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca61951e50 0x55ca6195f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61952810 0x55ca61966710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6183d510 con 0x55ca61952810 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6183d1b0 con 0x55ca619514f0 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca61724650 con 0x55ca61951e50 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d47dd8b80 10 client.0 _ll_register_callbacks cb 0x55ca61846ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d3a1ba700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61952810 0x55ca61966710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d3a1ba700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61952810 0x55ca61966710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49934/0 (socket says 172.21.15.39:49934) 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d3a1ba700 1 -- 172.21.15.39:0/3765886128 learned_addr learned my addr 172.21.15.39:0/3765886128 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.989+0000 7f6d399b9700 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca619514f0 0x55ca6195f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d391b8700 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca61951e50 0x55ca6195f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d3a1ba700 1 -- 172.21.15.39:0/3765886128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca61951e50 msgr2=0x55ca6195f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca61951e50 0x55ca6195f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d3a1ba700 1 -- 172.21.15.39:0/3765886128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca619514f0 msgr2=0x55ca6195f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca619514f0 0x55ca6195f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d3a1ba700 1 -- 172.21.15.39:0/3765886128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2400b770 con 0x55ca61952810 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d399b9700 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca619514f0 0x55ca6195f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61952810 0x55ca61966710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c099660 tx=0x7f6d2c099dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea1240ef00be03f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/3765886128 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0a1b50 con 0x55ca61952810 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/3765886128 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d2c003f20 con 0x55ca61952810 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3765886128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca617fa220 con 0x55ca61952810 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/3765886128 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d2c0b2020 con 0x55ca61952810 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d47dd8b80 10 client.15111 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3765886128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ca616ceb10 con 0x55ca61952810 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/3765886128 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ca617fa220 con 0x55ca61952810 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/3765886128 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d2c099930 con 0x55ca61952810 2024-05-24T11:37:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d47dd8b80 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.990+0000 7f6d47dd8b80 10 client.15111 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.991+0000 7f6d47dd8b80 20 client.15111 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.991+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x55ca61969d80 0x55ca6196f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.991+0000 7f6d47dd8b80 4 client.15111 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.991+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3765886128 --> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca619207c0 con 0x55ca61969d80 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.991+0000 7f6d399b9700 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x55ca61969d80 0x55ca6196f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.991+0000 7f6d1d7fa700 10 client.15111 ms_handle_connect on v2:172.21.15.137:6838/218688120 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.991+0000 7f6d399b9700 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x55ca61969d80 0x55ca6196f670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d383eb49a5ada13 server_cookie=80081e3929d23659 in_seq=0 out_seq=0 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/3765886128 <== mds.1 v2:172.21.15.137:6838/218688120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca619207c0 con 0x55ca61969d80 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d1d7fa700 10 client.15111 handle_command_reply: tid=0 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 client.15111 shutdown 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3765886128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61952810 msgr2=0x55ca61966710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61952810 0x55ca61966710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c099660 tx=0x7f6d2c099dd0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3765886128 shutdown_connections 2024-05-24T11:37:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x55ca61969d80 0x55ca6196f670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61952810 0x55ca61966710 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d1d7fa700 0 client.15111 ms_handle_reset on v2:172.21.15.137:6838/218688120 2024-05-24T11:37:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca619514f0 0x55ca6195f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3765886128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca61951e50 0x55ca6195f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3765886128 >> 172.21.15.39:0/3765886128 conn(0x55ca618af1c0 msgr2=0x55ca6194b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3765886128 shutdown_connections 2024-05-24T11:37:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3765886128 wait complete. 2024-05-24T11:37:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:40.992+0000 7f6d47dd8b80 20 client.15111 trim_cache size 0 max 16384 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d47dd8b80 1 Processor -- start 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d47dd8b80 1 -- start start 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca6195f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca6195f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6185f2b0 con 0x55ca61951e50 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6189a150 con 0x55ca61952810 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6189e010 con 0x55ca619514f0 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d391b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d391b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619666d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49936/0 (socket says 172.21.15.39:49936) 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d391b8700 1 -- 172.21.15.39:0/3882669530 learned_addr learned my addr 172.21.15.39:0/3882669530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d399b9700 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca6195f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca6195f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d391b8700 1 -- 172.21.15.39:0/3882669530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 msgr2=0x55ca6195f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d391b8700 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca6195f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d391b8700 1 -- 172.21.15.39:0/3882669530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 msgr2=0x55ca6195f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d391b8700 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca6195f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d391b8700 1 -- 172.21.15.39:0/3882669530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca61867510 con 0x55ca61951e50 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca6195f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.011+0000 7f6d391b8700 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619666d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400bb10 tx=0x7f6d2400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa48da65a0d5265a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d28da5700 1 -- 172.21.15.39:0/3882669530 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400dd00 con 0x55ca61951e50 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d28da5700 1 -- 172.21.15.39:0/3882669530 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d240054d0 con 0x55ca61951e50 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d28da5700 1 -- 172.21.15.39:0/3882669530 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24004e50 con 0x55ca61951e50 2024-05-24T11:37:41.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3882669530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 msgr2=0x55ca619666d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619666d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400bb10 tx=0x7f6d2400b610 comp rx=0 tx=0).stop 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3882669530 shutdown_connections 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca6195f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca6195f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/3882669530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca619666d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3882669530 >> 172.21.15.39:0/3882669530 conn(0x55ca618af1c0 msgr2=0x55ca6195e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3882669530 shutdown_connections 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/3882669530 wait complete. 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.012+0000 7f6d47dd8b80 1 -- start start 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d47dd8b80 1 Processor -- start 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca6195b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca6195bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d47dd8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca619c1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca61966760 con 0x55ca61951e50 2024-05-24T11:37:41.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca619668d0 con 0x55ca61952810 2024-05-24T11:37:41.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d47dd8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca61966a40 con 0x55ca619514f0 2024-05-24T11:37:41.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d47dd8b80 10 client.0 _ll_register_callbacks cb 0x55ca61846ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:41.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d391b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca6195bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d391b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca6195bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49944/0 (socket says 172.21.15.39:49944) 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d391b8700 1 -- 172.21.15.39:0/1843418002 learned_addr learned my addr 172.21.15.39:0/1843418002 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d3a1ba700 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca619c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d399b9700 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca6195b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d391b8700 1 -- 172.21.15.39:0/1843418002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 msgr2=0x55ca6195b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d391b8700 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca6195b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d391b8700 1 -- 172.21.15.39:0/1843418002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 msgr2=0x55ca619c1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d391b8700 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca619c1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d391b8700 1 -- 172.21.15.39:0/1843418002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca61722360 con 0x55ca61951e50 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.013+0000 7f6d399b9700 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca6195b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d391b8700 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca6195bd10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400d620 tx=0x7f6d24004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3c85f62e0d5053 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/1843418002 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d240050a0 con 0x55ca61951e50 2024-05-24T11:37:41.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/1843418002 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d24013070 con 0x55ca61951e50 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/1843418002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d2400de60 con 0x55ca61951e50 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/1843418002 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d24005200 con 0x55ca61951e50 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d47dd8b80 10 client.15117 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/1843418002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ca61726c80 con 0x55ca61951e50 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/1843418002 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d2400de60 con 0x55ca61951e50 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/1843418002 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d24034580 con 0x55ca61951e50 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d47dd8b80 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d47dd8b80 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d47dd8b80 20 client.15117 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x55ca619c5f50 0x55ca619c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d47dd8b80 4 client.15117 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.014+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/1843418002 --> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ca61899e70 con 0x55ca619c5f50 2024-05-24T11:37:41.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.015+0000 7f6d399b9700 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x55ca619c5f50 0x55ca619c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.015+0000 7f6d399b9700 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x55ca619c5f50 0x55ca619c28f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70958d7892a14a0f server_cookie=8fd382a44f005020 in_seq=0 out_seq=0 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.015+0000 7f6d1d7fa700 10 client.15117 ms_handle_connect on v2:172.21.15.137:6838/218688120 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.015+0000 7f6d1d7fa700 1 -- 172.21.15.39:0/1843418002 <== mds.1 v2:172.21.15.137:6838/218688120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ca61899e70 con 0x55ca619c5f50 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.015+0000 7f6d1d7fa700 10 client.15117 handle_command_reply: tid=0 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.015+0000 7f6d47dd8b80 1 client.15117 shutdown 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.015+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/1843418002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 msgr2=0x55ca6195bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.015+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca6195bd10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400d620 tx=0x7f6d24004920 comp rx=0 tx=0).stop 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/1843418002 shutdown_connections 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x55ca619c5f50 0x55ca619c28f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ca619514f0 0x55ca6195b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca61952810 0x55ca619c1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d47dd8b80 1 --2- 172.21.15.39:0/1843418002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ca61951e50 0x55ca6195bd10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d1d7fa700 0 client.15117 ms_handle_reset on v2:172.21.15.137:6838/218688120 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/1843418002 >> 172.21.15.39:0/1843418002 conn(0x55ca618af1c0 msgr2=0x55ca6195e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/1843418002 shutdown_connections 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d47dd8b80 1 -- 172.21.15.39:0/1843418002 wait complete. 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d47dd8b80 20 client.15117 trim_cache size 0 max 16384 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24511, 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:41.029 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 49.426621445000002, 2024-05-24T11:37:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 77, 2024-05-24T11:37:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:41.030 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 63.781494109999997 2024-05-24T11:37:41.030 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:41.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6d200a11b0 msgr2=0x7f6d200a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6d200a11b0 0x7f6d200a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000c010 tx=0x7f6d30003720 comp rx=0 tx=0).stop 2024-05-24T11:37:41.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d34101970 msgr2=0x7f6d3419d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d34101970 0x7f6d3419d260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c007ee0 tx=0x7f6d2c00d3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/217312463 shutdown_connections 2024-05-24T11:37:41.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6d200a11b0 0x7f6d200a3670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d340ffe60 0x7f6d34195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d34101970 0x7f6d3419d260 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 --2- 172.21.15.39:0/217312463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d34100fb0 0x7f6d34196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/217312463 >> 172.21.15.39:0/217312463 conn(0x7f6d340fb3f0 msgr2=0x7f6d34104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/217312463 shutdown_connections 2024-05-24T11:37:41.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.016+0000 7f6d3bc1d700 1 -- 172.21.15.39:0/217312463 wait complete. 2024-05-24T11:37:41.033 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:40 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4014520419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:41.033 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:40 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1889777448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:41.035 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-24T11:37:41.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.156+0000 7fec89052700 1 -- 172.21.15.39:0/2657521646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84104390 msgr2=0x7fec84106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.156+0000 7fec89052700 1 --2- 172.21.15.39:0/2657521646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84104390 0x7fec84106780 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fec78009990 tx=0x7fec78009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.156+0000 7fec89052700 1 -- 172.21.15.39:0/2657521646 shutdown_connections 2024-05-24T11:37:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.156+0000 7fec89052700 1 --2- 172.21.15.39:0/2657521646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec84106cc0 0x7fec841090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.156+0000 7fec89052700 1 --2- 172.21.15.39:0/2657521646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec84101a60 0x7fec84103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.156+0000 7fec89052700 1 --2- 172.21.15.39:0/2657521646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84104390 0x7fec84106780 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.156+0000 7fec89052700 1 -- 172.21.15.39:0/2657521646 >> 172.21.15.39:0/2657521646 conn(0x7fec840fb3f0 msgr2=0x7fec840fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.156+0000 7fec89052700 1 -- 172.21.15.39:0/2657521646 shutdown_connections 2024-05-24T11:37:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.156+0000 7fec89052700 1 -- 172.21.15.39:0/2657521646 wait complete. 2024-05-24T11:37:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.157+0000 7fec89052700 1 Processor -- start 2024-05-24T11:37:41.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.157+0000 7fec89052700 1 -- start start 2024-05-24T11:37:41.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.157+0000 7fec89052700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84101a60 0x7fec84196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.157+0000 7fec89052700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec84104390 0x7fec841973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.157+0000 7fec89052700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec84106cc0 0x7fec8419e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.157+0000 7fec89052700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8419e9a0 con 0x7fec84101a60 2024-05-24T11:37:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.157+0000 7fec89052700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8419eae0 con 0x7fec84106cc0 2024-05-24T11:37:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec89052700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8419ec70 con 0x7fec84104390 2024-05-24T11:37:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec82d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84101a60 0x7fec84196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec82d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84101a60 0x7fec84196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49956/0 (socket says 172.21.15.39:49956) 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec82d9d700 1 -- 172.21.15.39:0/2217219117 learned_addr learned my addr 172.21.15.39:0/2217219117 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec8259c700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec84104390 0x7fec841973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec8359e700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec84106cc0 0x7fec8419e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec82d9d700 1 -- 172.21.15.39:0/2217219117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec84104390 msgr2=0x7fec841973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec82d9d700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec84104390 0x7fec841973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec82d9d700 1 -- 172.21.15.39:0/2217219117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec84106cc0 msgr2=0x7fec8419e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec82d9d700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec84106cc0 0x7fec8419e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec82d9d700 1 -- 172.21.15.39:0/2217219117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec780095f0 con 0x7fec84101a60 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec8359e700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec84106cc0 0x7fec8419e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec82d9d700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84101a60 0x7fec84196e70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fec7400e9a0 tx=0x7fec7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec6bfff700 1 -- 172.21.15.39:0/2217219117 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74009790 con 0x7fec84101a60 2024-05-24T11:37:41.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec6bfff700 1 -- 172.21.15.39:0/2217219117 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec7400cbd0 con 0x7fec84101a60 2024-05-24T11:37:41.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec89052700 1 -- 172.21.15.39:0/2217219117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8419ef50 con 0x7fec84101a60 2024-05-24T11:37:41.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec6bfff700 1 -- 172.21.15.39:0/2217219117 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74018680 con 0x7fec84101a60 2024-05-24T11:37:41.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.158+0000 7fec89052700 1 -- 172.21.15.39:0/2217219117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8419f370 con 0x7fec84101a60 2024-05-24T11:37:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.160+0000 7fec6bfff700 1 -- 172.21.15.39:0/2217219117 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fec7400cd40 con 0x7fec84101a60 2024-05-24T11:37:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.161+0000 7fec6bfff700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec6c0a11b0 0x7fec6c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.161+0000 7fec8259c700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec6c0a11b0 0x7fec6c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.161+0000 7fec6bfff700 1 -- 172.21.15.39:0/2217219117 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fec74014070 con 0x7fec84101a60 2024-05-24T11:37:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.162+0000 7fec8259c700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec6c0a11b0 0x7fec6c0a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec78006010 tx=0x7fec7800b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.162+0000 7fec9520db80 1 Processor -- start 2024-05-24T11:37:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.162+0000 7fec9520db80 1 -- start start 2024-05-24T11:37:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec9520db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa973910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec9520db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 0x558efa9742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec9520db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa974c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec9520db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa8ca100 con 0x558efa9734f0 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec9520db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa8c7ac0 con 0x558efa973e50 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec9520db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa8c33b0 con 0x558efa974810 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec82d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa973910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec82d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa973910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49970/0 (socket says 172.21.15.39:49970) 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec82d9d700 1 -- 172.21.15.39:0/748352003 learned_addr learned my addr 172.21.15.39:0/748352003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec8359e700 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa974c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec8259c700 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 0x558efa9742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec82d9d700 1 -- 172.21.15.39:0/748352003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 msgr2=0x558efa974c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec82d9d700 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa974c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec82d9d700 1 -- 172.21.15.39:0/748352003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 msgr2=0x558efa9742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec82d9d700 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 0x558efa9742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec82d9d700 1 -- 172.21.15.39:0/748352003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558efa744360 con 0x558efa9734f0 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec8359e700 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa974c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec82d9d700 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa973910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fec740a1f10 tx=0x7fec74099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4840eadc1b3153f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec697fa700 1 -- 172.21.15.39:0/748352003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7409ddf0 con 0x558efa9734f0 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec697fa700 1 -- 172.21.15.39:0/748352003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec740ade90 con 0x558efa9734f0 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.163+0000 7fec697fa700 1 -- 172.21.15.39:0/748352003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74099780 con 0x558efa9734f0 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 -- 172.21.15.39:0/748352003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 msgr2=0x558efa973910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa973910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fec740a1f10 tx=0x7fec74099bd0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 -- 172.21.15.39:0/748352003 shutdown_connections 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa974c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa973910 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 --2- 172.21.15.39:0/748352003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 0x558efa9742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 -- 172.21.15.39:0/748352003 >> 172.21.15.39:0/748352003 conn(0x558efa8d11c0 msgr2=0x558efa96dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 -- 172.21.15.39:0/748352003 shutdown_connections 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 -- 172.21.15.39:0/748352003 wait complete. 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 -- start start 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.164+0000 7fec9520db80 1 Processor -- start 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec9520db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa97cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec9520db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 0x558efa97d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec9520db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa984300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec9520db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa85f510 con 0x558efa9734f0 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec9520db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa85f1b0 con 0x558efa973e50 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec9520db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa746650 con 0x558efa974810 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec9520db80 10 client.0 _ll_register_callbacks cb 0x558efa868ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec82d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa97cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec82d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa97cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49980/0 (socket says 172.21.15.39:49980) 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec82d9d700 1 -- 172.21.15.39:0/1688051547 learned_addr learned my addr 172.21.15.39:0/1688051547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec8259c700 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 0x558efa97d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec8359e700 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa984300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec82d9d700 1 -- 172.21.15.39:0/1688051547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 msgr2=0x558efa984300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec82d9d700 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa984300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec82d9d700 1 -- 172.21.15.39:0/1688051547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 msgr2=0x558efa97d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec82d9d700 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 0x558efa97d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec82d9d700 1 -- 172.21.15.39:0/1688051547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8419f370 con 0x558efa9734f0 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec82d9d700 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa97cd80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fec740964c0 tx=0x7fec74099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591aa6d0e5128d69 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.165+0000 7fec66ff5700 1 -- 172.21.15.39:0/1688051547 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740a2440 con 0x558efa9734f0 2024-05-24T11:37:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec66ff5700 1 -- 172.21.15.39:0/1688051547 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fec7409ddf0 con 0x558efa9734f0 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec9520db80 1 -- 172.21.15.39:0/1688051547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558efa81c220 con 0x558efa9734f0 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec66ff5700 1 -- 172.21.15.39:0/1688051547 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec740b82b0 con 0x558efa9734f0 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec9520db80 10 client.15129 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec9520db80 1 -- 172.21.15.39:0/1688051547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558efa6f0b10 con 0x558efa9734f0 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec66ff5700 1 -- 172.21.15.39:0/1688051547 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558efa81c220 con 0x558efa9734f0 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec66ff5700 1 -- 172.21.15.39:0/1688051547 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fec74095db0 con 0x558efa9734f0 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec9520db80 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec9520db80 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec9520db80 20 client.15129 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec9520db80 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558efa987c60 0x558efa98cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec9520db80 4 client.15129 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec9520db80 1 -- 172.21.15.39:0/1688051547 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558efa9427c0 con 0x558efa987c60 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.166+0000 7fec8259c700 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558efa987c60 0x558efa98cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.167+0000 7fec8259c700 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558efa987c60 0x558efa98cf00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bb93876eb6fe78c server_cookie=62b919b741abc168 in_seq=0 out_seq=0 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.167+0000 7fec66ff5700 10 client.15129 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec66ff5700 1 -- 172.21.15.39:0/1688051547 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558efa9427c0 con 0x558efa987c60 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec66ff5700 10 client.15129 handle_command_reply: tid=0 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 client.15129 shutdown 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 -- 172.21.15.39:0/1688051547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 msgr2=0x558efa97cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa97cd80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fec740964c0 tx=0x7fec74099a50 comp rx=0 tx=0).stop 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 -- 172.21.15.39:0/1688051547 shutdown_connections 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa984300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558efa987c60 0x558efa98cf00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa9734f0 0x558efa97cd80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec66ff5700 0 client.15129 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 --2- 172.21.15.39:0/1688051547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa973e50 0x558efa97d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 -- 172.21.15.39:0/1688051547 >> 172.21.15.39:0/1688051547 conn(0x558efa8d11c0 msgr2=0x558efa96d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 -- 172.21.15.39:0/1688051547 shutdown_connections 2024-05-24T11:37:41.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 1 -- 172.21.15.39:0/1688051547 wait complete. 2024-05-24T11:37:41.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.168+0000 7fec9520db80 20 client.15129 trim_cache size 0 max 16384 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.186+0000 7fec9520db80 1 Processor -- start 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec9520db80 1 -- start start 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec9520db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 0x558efa97d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec9520db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa973e50 0x558efa9738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec9520db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa974810 0x558efa9849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec9520db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa96ebb0 con 0x558efa974810 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec9520db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa8812e0 con 0x558efa9734f0 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec9520db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa8c0010 con 0x558efa973e50 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa974810 0x558efa9849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa974810 0x558efa9849b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49996/0 (socket says 172.21.15.39:49996) 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8359e700 1 -- 172.21.15.39:0/3166002434 learned_addr learned my addr 172.21.15.39:0/3166002434 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8259c700 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa973e50 0x558efa9738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec82d9d700 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 0x558efa97d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8359e700 1 -- 172.21.15.39:0/3166002434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa973e50 msgr2=0x558efa9738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8359e700 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa973e50 0x558efa9738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8359e700 1 -- 172.21.15.39:0/3166002434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 msgr2=0x558efa97d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8359e700 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 0x558efa97d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8359e700 1 -- 172.21.15.39:0/3166002434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558efa889510 con 0x558efa974810 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec82d9d700 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 0x558efa97d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec8359e700 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa974810 0x558efa9849b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0060b0 tx=0x7fec7c0076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3816847d3c4f419c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.187+0000 7fec69ffb700 1 -- 172.21.15.39:0/3166002434 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c004bb0 con 0x558efa974810 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec69ffb700 1 -- 172.21.15.39:0/3166002434 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec7c00f040 con 0x558efa974810 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec69ffb700 1 -- 172.21.15.39:0/3166002434 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c017cc0 con 0x558efa974810 2024-05-24T11:37:41.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 -- 172.21.15.39:0/3166002434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa974810 msgr2=0x558efa9849b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa974810 0x558efa9849b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0060b0 tx=0x7fec7c0076c0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 -- 172.21.15.39:0/3166002434 shutdown_connections 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa974810 0x558efa9849b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 0x558efa97d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 --2- 172.21.15.39:0/3166002434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa973e50 0x558efa9738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 -- 172.21.15.39:0/3166002434 >> 172.21.15.39:0/3166002434 conn(0x558efa8d11c0 msgr2=0x558efa974bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 -- 172.21.15.39:0/3166002434 shutdown_connections 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 -- 172.21.15.39:0/3166002434 wait complete. 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.188+0000 7fec9520db80 1 -- start start 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec9520db80 1 Processor -- start 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec9520db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 0x558efa979c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec9520db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa973e50 0x558efa97a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec9520db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa984380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec9520db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa9788b0 con 0x558efa973e50 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec9520db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa978a20 con 0x558efa9734f0 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec9520db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558efa978be0 con 0x558efa974810 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec9520db80 10 client.0 _ll_register_callbacks cb 0x558efa868ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa973e50 0x558efa97a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa973e50 0x558efa97a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50012/0 (socket says 172.21.15.39:50012) 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8259c700 1 -- 172.21.15.39:0/19706954 learned_addr learned my addr 172.21.15.39:0/19706954 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec82d9d700 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 0x558efa979c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8359e700 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa984380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8259c700 1 -- 172.21.15.39:0/19706954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 msgr2=0x558efa984380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8259c700 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa984380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8259c700 1 -- 172.21.15.39:0/19706954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 msgr2=0x558efa979c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8259c700 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 0x558efa979c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8259c700 1 -- 172.21.15.39:0/19706954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c009ab0 con 0x558efa973e50 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec8259c700 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa973e50 0x558efa97a1d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec78003f60 tx=0x7fec78009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a73531b68dede4c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.189+0000 7fec66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec66ff5700 1 -- 172.21.15.39:0/19706954 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7800b7d0 con 0x558efa973e50 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec66ff5700 1 -- 172.21.15.39:0/19706954 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fec7800b930 con 0x558efa973e50 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec9520db80 1 -- 172.21.15.39:0/19706954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec740ade90 con 0x558efa973e50 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec66ff5700 1 -- 172.21.15.39:0/19706954 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec7800baf0 con 0x558efa973e50 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec9520db80 10 client.15135 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec9520db80 1 -- 172.21.15.39:0/19706954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558efa748c80 con 0x558efa973e50 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec66ff5700 1 -- 172.21.15.39:0/19706954 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec740ade90 con 0x558efa973e50 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec66ff5700 1 -- 172.21.15.39:0/19706954 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fec78048070 con 0x558efa973e50 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec9520db80 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec9520db80 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec9520db80 20 client.15135 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec9520db80 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558efa9e2ea0 0x558efa9df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec9520db80 4 client.15135 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec9520db80 1 -- 172.21.15.39:0/19706954 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558efa8bbe70 con 0x558efa9e2ea0 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.190+0000 7fec82d9d700 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558efa9e2ea0 0x558efa9df7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec82d9d700 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558efa9e2ea0 0x558efa9df7d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f18b490c18a80b5e server_cookie=a626d7b5e7a2481 in_seq=0 out_seq=0 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec66ff5700 10 client.15135 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec66ff5700 1 -- 172.21.15.39:0/19706954 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558efa8bbe70 con 0x558efa9e2ea0 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec66ff5700 10 client.15135 handle_command_reply: tid=0 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 client.15135 shutdown 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 -- 172.21.15.39:0/19706954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa973e50 msgr2=0x558efa97a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa973e50 0x558efa97a1d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec78003f60 tx=0x7fec78009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 -- 172.21.15.39:0/19706954 shutdown_connections 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558efa974810 0x558efa984380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558efa9e2ea0 0x558efa9df7d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558efa9734f0 0x558efa979c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec66ff5700 0 client.15135 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:37:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 --2- 172.21.15.39:0/19706954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558efa973e50 0x558efa97a1d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 -- 172.21.15.39:0/19706954 >> 172.21.15.39:0/19706954 conn(0x558efa8d11c0 msgr2=0x558efa974bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 -- 172.21.15.39:0/19706954 shutdown_connections 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 1 -- 172.21.15.39:0/19706954 wait complete. 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.191+0000 7fec9520db80 20 client.15135 trim_cache size 0 max 16384 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24542, 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:41.204 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 52.688862389999997, 2024-05-24T11:37:41.205 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:41.205 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 77, 2024-05-24T11:37:41.205 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:41.205 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 58.321718214999997 2024-05-24T11:37:41.205 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 -- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec6c0a11b0 msgr2=0x7fec6c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec6c0a11b0 0x7fec6c0a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec78006010 tx=0x7fec7800b610 comp rx=0 tx=0).stop 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 -- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84101a60 msgr2=0x7fec84196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84101a60 0x7fec84196e70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fec7400e9a0 tx=0x7fec7400ecb0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 -- 172.21.15.39:0/2217219117 shutdown_connections 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fec6c0a11b0 0x7fec6c0a3670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec84106cc0 0x7fec8419e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec84101a60 0x7fec84196e70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 --2- 172.21.15.39:0/2217219117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec84104390 0x7fec841973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 -- 172.21.15.39:0/2217219117 >> 172.21.15.39:0/2217219117 conn(0x7fec840fb3f0 msgr2=0x7fec841039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 -- 172.21.15.39:0/2217219117 shutdown_connections 2024-05-24T11:37:41.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:41.192+0000 7fec89052700 1 -- 172.21.15.39:0/2217219117 wait complete. 2024-05-24T11:37:41.210 INFO:tasks.cephadm:Setup complete, yielding 2024-05-24T11:37:41.210 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-24T11:37:41.220 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:37:41.221 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-24T11:37:41.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:40 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4014520419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:41.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:40 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1889777448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:41.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:40 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4014520419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:41.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:40 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1889777448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:41.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:41 smithi039 ceph-mon[42919]: pgmap v246: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:37:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:41 smithi053 ceph-mon[47448]: pgmap v246: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:37:42.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:41 smithi137 ceph-mon[47907]: pgmap v246: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:37:42.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.506+0000 7fddb330d700 1 -- 172.21.15.39:0/3445570771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac106db0 msgr2=0x7fddac1091a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:42.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.506+0000 7fddb330d700 1 --2- 172.21.15.39:0/3445570771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac106db0 0x7fddac1091a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdda400b0a0 tx=0x7fdda400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.506+0000 7fddb330d700 1 -- 172.21.15.39:0/3445570771 shutdown_connections 2024-05-24T11:37:42.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.506+0000 7fddb330d700 1 --2- 172.21.15.39:0/3445570771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac106db0 0x7fddac1091a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.506+0000 7fddb330d700 1 --2- 172.21.15.39:0/3445570771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac101b50 0x7fddac103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.506+0000 7fddb330d700 1 --2- 172.21.15.39:0/3445570771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddac104480 0x7fddac106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.506+0000 7fddb330d700 1 -- 172.21.15.39:0/3445570771 >> 172.21.15.39:0/3445570771 conn(0x7fddac0fb4c0 msgr2=0x7fddac0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:42.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.506+0000 7fddb330d700 1 -- 172.21.15.39:0/3445570771 shutdown_connections 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.506+0000 7fddb330d700 1 -- 172.21.15.39:0/3445570771 wait complete. 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb330d700 1 Processor -- start 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb330d700 1 -- start start 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb330d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddac101b50 0x7fddac197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb330d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac104480 0x7fddac197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb330d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac106db0 0x7fddac19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb330d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac19eb60 con 0x7fddac104480 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb330d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac19eca0 con 0x7fddac106db0 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb330d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac19ee30 con 0x7fddac101b50 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb08a8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac104480 0x7fddac197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb08a8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac104480 0x7fddac197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50034/0 (socket says 172.21.15.39:50034) 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb08a8700 1 -- 172.21.15.39:0/3173033565 learned_addr learned my addr 172.21.15.39:0/3173033565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:42.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb18aa700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac106db0 0x7fddac19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.507+0000 7fddb10a9700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddac101b50 0x7fddac197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fddb08a8700 1 -- 172.21.15.39:0/3173033565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddac101b50 msgr2=0x7fddac197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fddb08a8700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddac101b50 0x7fddac197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fddb08a8700 1 -- 172.21.15.39:0/3173033565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac106db0 msgr2=0x7fddac19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fddb08a8700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac106db0 0x7fddac19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fddb08a8700 1 -- 172.21.15.39:0/3173033565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda4009d00 con 0x7fddac104480 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fddb10a9700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddac101b50 0x7fddac197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fddb08a8700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac104480 0x7fddac197560 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdda000b560 tx=0x7fdda000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fdd9e7fc700 1 -- 172.21.15.39:0/3173033565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0011830 con 0x7fddac104480 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddac19f110 con 0x7fddac104480 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fdd9e7fc700 1 -- 172.21.15.39:0/3173033565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdda0011e20 con 0x7fddac104480 2024-05-24T11:37:42.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fdd9e7fc700 1 -- 172.21.15.39:0/3173033565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0010dd0 con 0x7fddac104480 2024-05-24T11:37:42.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.508+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddac19f6e0 con 0x7fddac104480 2024-05-24T11:37:42.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.509+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddac0fd040 con 0x7fddac104480 2024-05-24T11:37:42.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.511+0000 7fdd9e7fc700 1 -- 172.21.15.39:0/3173033565 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdda0011990 con 0x7fddac104480 2024-05-24T11:37:42.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.511+0000 7fdd9e7fc700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdd980a11b0 0x7fdd980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:42.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.512+0000 7fdd9e7fc700 1 -- 172.21.15.39:0/3173033565 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdda00cc720 con 0x7fddac104480 2024-05-24T11:37:42.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.512+0000 7fddb10a9700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdd980a11b0 0x7fdd980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:42.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.512+0000 7fddb10a9700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdd980a11b0 0x7fdd980a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007900 tx=0x7fdda8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:42.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.515+0000 7fdd9e7fc700 1 -- 172.21.15.39:0/3173033565 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdda0095f10 con 0x7fddac104480 2024-05-24T11:37:42.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.738+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fddac04ec60 con 0x7fdd980a11b0 2024-05-24T11:37:42.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.739+0000 7fdd9e7fc700 1 -- 172.21.15.39:0/3173033565 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fddac04ec60 con 0x7fdd980a11b0 2024-05-24T11:37:42.913 INFO:teuthology.orchestra.run.smithi039.stdout:Backend: cephadm 2024-05-24T11:37:42.913 INFO:teuthology.orchestra.run.smithi039.stdout:Available: Yes 2024-05-24T11:37:42.913 INFO:teuthology.orchestra.run.smithi039.stdout:Paused: No 2024-05-24T11:37:42.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdd980a11b0 msgr2=0x7fdd980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:42.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdd980a11b0 0x7fdd980a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007900 tx=0x7fdda8008040 comp rx=0 tx=0).stop 2024-05-24T11:37:42.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac104480 msgr2=0x7fddac197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:42.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac104480 0x7fddac197560 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdda000b560 tx=0x7fdda000b870 comp rx=0 tx=0).stop 2024-05-24T11:37:42.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 shutdown_connections 2024-05-24T11:37:42.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fdd980a11b0 0x7fdd980a3670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddac101b50 0x7fddac197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddac106db0 0x7fddac19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 --2- 172.21.15.39:0/3173033565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddac104480 0x7fddac197560 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:42.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 >> 172.21.15.39:0/3173033565 conn(0x7fddac0fb4c0 msgr2=0x7fddac102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:42.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 shutdown_connections 2024-05-24T11:37:42.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:42.742+0000 7fddb330d700 1 -- 172.21.15.39:0/3173033565 wait complete. 2024-05-24T11:37:43.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-24T11:37:43.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-24T11:37:43.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-24T11:37:43.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-24T11:37:43.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-24T11:37:43.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:42 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-24T11:37:43.169 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-24T11:37:43.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:42 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-24T11:37:43.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:42 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-24T11:37:43.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:42 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-24T11:37:43.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:42 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-24T11:37:43.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:42 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-24T11:37:43.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:42 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-24T11:37:43.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:42 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-24T11:37:43.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:42 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-24T11:37:43.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:42 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-24T11:37:43.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:42 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-24T11:37:43.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:42 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-24T11:37:43.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:42 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-24T11:37:43.399 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:43 smithi053 ceph-mon[47448]: pgmap v247: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-24T11:37:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:43 smithi053 ceph-mon[47448]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-24T11:37:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-24T11:37:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-24T11:37:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-24T11:37:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-24T11:37:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:43 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-24T11:37:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:43 smithi053 ceph-mon[47448]: osdmap e78: 12 total, 12 up, 12 in 2024-05-24T11:37:44.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:43 smithi137 ceph-mon[47907]: pgmap v247: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-24T11:37:44.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:43 smithi137 ceph-mon[47907]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:44.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-24T11:37:44.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-24T11:37:44.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-24T11:37:44.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-24T11:37:44.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-24T11:37:44.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:43 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-24T11:37:44.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:43 smithi137 ceph-mon[47907]: osdmap e78: 12 total, 12 up, 12 in 2024-05-24T11:37:44.248 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:43 smithi039 ceph-mon[42919]: pgmap v247: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-24T11:37:44.248 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:43 smithi039 ceph-mon[42919]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:44.248 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-24T11:37:44.249 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-24T11:37:44.249 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-24T11:37:44.249 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-24T11:37:44.249 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-24T11:37:44.249 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:43 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-24T11:37:44.249 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:43 smithi039 ceph-mon[42919]: osdmap e78: 12 total, 12 up, 12 in 2024-05-24T11:37:44.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.422+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3638445993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 msgr2=0x7fe5cc109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:44.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.422+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3638445993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 0x7fe5cc109450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400b0d0 tx=0x7fe5c400b3e0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.422+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3638445993 shutdown_connections 2024-05-24T11:37:44.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.422+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3638445993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc101000 0x7fe5cc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.422+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3638445993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 0x7fe5cc109450 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.422+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3638445993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc102200 0x7fe5cc102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.422+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3638445993 >> 172.21.15.39:0/3638445993 conn(0x7fe5cc0fc5c0 msgr2=0x7fe5cc0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:44.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.422+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3638445993 shutdown_connections 2024-05-24T11:37:44.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.422+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3638445993 wait complete. 2024-05-24T11:37:44.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d31d6700 1 Processor -- start 2024-05-24T11:37:44.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d31d6700 1 -- start start 2024-05-24T11:37:44.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d31d6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101000 0x7fe5cc196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:44.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d31d6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc102200 0x7fe5cc197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:44.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d31d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 0x7fe5cc19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d31d6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc19ea80 con 0x7fe5cc102ba0 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d31d6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc19ebc0 con 0x7fe5cc101000 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d31d6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc19ed50 con 0x7fe5cc102200 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d1773700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 0x7fe5cc19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d1773700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 0x7fe5cc19e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50058/0 (socket says 172.21.15.39:50058) 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d1773700 1 -- 172.21.15.39:0/3737281803 learned_addr learned my addr 172.21.15.39:0/3737281803 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.423+0000 7fe5d0f72700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101000 0x7fe5cc196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5cbfff700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc102200 0x7fe5cc197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5d1773700 1 -- 172.21.15.39:0/3737281803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc102200 msgr2=0x7fe5cc197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5d1773700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc102200 0x7fe5cc197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5d1773700 1 -- 172.21.15.39:0/3737281803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101000 msgr2=0x7fe5cc196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5d1773700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101000 0x7fe5cc196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5d1773700 1 -- 172.21.15.39:0/3737281803 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c4009d00 con 0x7fe5cc102ba0 2024-05-24T11:37:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5d1773700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 0x7fe5cc19e4b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4007990 tx=0x7fe5c40094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5c9ffb700 1 -- 172.21.15.39:0/3737281803 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400e040 con 0x7fe5cc102ba0 2024-05-24T11:37:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5c9ffb700 1 -- 172.21.15.39:0/3737281803 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5c4012d90 con 0x7fe5cc102ba0 2024-05-24T11:37:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5cc19efd0 con 0x7fe5cc102ba0 2024-05-24T11:37:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5c9ffb700 1 -- 172.21.15.39:0/3737281803 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c401dea0 con 0x7fe5cc102ba0 2024-05-24T11:37:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.424+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc19f3e0 con 0x7fe5cc102ba0 2024-05-24T11:37:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.425+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc108490 con 0x7fe5cc102ba0 2024-05-24T11:37:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.426+0000 7fe5c9ffb700 1 -- 172.21.15.39:0/3737281803 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe5c4012900 con 0x7fe5cc102ba0 2024-05-24T11:37:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.427+0000 7fe5c9ffb700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe5bc0a11b0 0x7fe5bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.427+0000 7fe5c9ffb700 1 -- 172.21.15.39:0/3737281803 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe5c40cebb0 con 0x7fe5cc102ba0 2024-05-24T11:37:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.427+0000 7fe5d0f72700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe5bc0a11b0 0x7fe5bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.427+0000 7fe5d0f72700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe5bc0a11b0 0x7fe5bc0a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc102060 tx=0x7fe5c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:44.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.430+0000 7fe5c9ffb700 1 -- 172.21.15.39:0/3737281803 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe5c4098270 con 0x7fe5cc102ba0 2024-05-24T11:37:44.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.644+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5cc02ce10 con 0x7fe5bc0a11b0 2024-05-24T11:37:44.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.658+0000 7fe5c9ffb700 1 -- 172.21.15.39:0/3737281803 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fe5cc02ce10 con 0x7fe5bc0a11b0 2024-05-24T11:37:44.665 INFO:teuthology.orchestra.run.smithi039.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-24T11:37:44.665 INFO:teuthology.orchestra.run.smithi039.stdout:mds.a smithi039 running (63s) 58s ago 63s 16.7M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 0ecd24dfb557 2024-05-24T11:37:44.665 INFO:teuthology.orchestra.run.smithi039.stdout:mds.b smithi053 running (62s) 58s ago 61s 15.7M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 48a982d25118 2024-05-24T11:37:44.665 INFO:teuthology.orchestra.run.smithi039.stdout:mds.c smithi137 running (71s) 58s ago 71s 14.3M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 1c8aca4d78e6 2024-05-24T11:37:44.665 INFO:teuthology.orchestra.run.smithi039.stdout:mds.d smithi039 running (76s) 58s ago 76s 14.1M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 dc554467db18 2024-05-24T11:37:44.666 INFO:teuthology.orchestra.run.smithi039.stdout:mds.e smithi053 running (74s) 58s ago 74s 15.8M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 ec08fa38bfa1 2024-05-24T11:37:44.666 INFO:teuthology.orchestra.run.smithi039.stdout:mds.f smithi137 running (73s) 58s ago 72s 15.8M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 b4da550cea9b 2024-05-24T11:37:44.666 INFO:teuthology.orchestra.run.smithi039.stdout:mds.g smithi039 running (65s) 58s ago 65s 16.8M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 ee672293445d 2024-05-24T11:37:44.666 INFO:teuthology.orchestra.run.smithi039.stdout:mds.h smithi053 running (69s) 58s ago 69s 15.7M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 03f129f03a66 2024-05-24T11:37:44.666 INFO:teuthology.orchestra.run.smithi039.stdout:mds.i smithi137 running (67s) 58s ago 67s 13.4M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 1a56c5736833 2024-05-24T11:37:44.666 INFO:teuthology.orchestra.run.smithi039.stdout:mds.j smithi039 running (78s) 58s ago 78s 14.3M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 015c100436f6 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:mds.k smithi053 running (80s) 58s ago 79s 14.3M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 3db246f96622 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:mds.l smithi137 running (81s) 58s ago 81s 14.4M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 a3624f5e4de2 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.x smithi039 *:9283,8765 running (6m) 58s ago 6m 525M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 e7663b17d798 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.y smithi053 *:8443,8765 running (5m) 58s ago 5m 457M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 ba706faa775d 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.z smithi137 *:8443,8765 running (5m) 58s ago 5m 459M - 18.2.2-1349-gb9b067bc 5a1940d32fa0 897822afa79b 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a smithi039 running (6m) 58s ago 6m 82.2M 2048M 18.2.2-1349-gb9b067bc 5a1940d32fa0 861f9e130e3c 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b smithi053 running (5m) 58s ago 5m 51.7M 2048M 18.2.2-1349-gb9b067bc 5a1940d32fa0 e3f722b85654 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c smithi137 running (5m) 58s ago 5m 55.6M 2048M 18.2.2-1349-gb9b067bc 5a1940d32fa0 7bec3616b680 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 smithi039 running (4m) 58s ago 4m 62.9M 4278M 18.2.2-1349-gb9b067bc 5a1940d32fa0 4a7a2634b7b6 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 smithi053 running (4m) 58s ago 4m 67.7M 4279M 18.2.2-1349-gb9b067bc 5a1940d32fa0 77cc0c443a25 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 smithi137 running (4m) 58s ago 4m 53.0M 4278M 18.2.2-1349-gb9b067bc 5a1940d32fa0 c0acacf4ac28 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 smithi039 running (4m) 58s ago 3m 60.8M 4278M 18.2.2-1349-gb9b067bc 5a1940d32fa0 178a402cdc41 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 smithi053 running (3m) 58s ago 3m 46.6M 4279M 18.2.2-1349-gb9b067bc 5a1940d32fa0 aa2fe20f43d2 2024-05-24T11:37:44.667 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 smithi137 running (3m) 58s ago 3m 59.4M 4278M 18.2.2-1349-gb9b067bc 5a1940d32fa0 25f0d2f6193e 2024-05-24T11:37:44.668 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 smithi039 running (3m) 58s ago 3m 59.3M 4278M 18.2.2-1349-gb9b067bc 5a1940d32fa0 748ce9db2894 2024-05-24T11:37:44.668 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 smithi053 running (2m) 58s ago 2m 53.1M 4279M 18.2.2-1349-gb9b067bc 5a1940d32fa0 e9421842c599 2024-05-24T11:37:44.668 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 smithi137 running (2m) 58s ago 2m 49.2M 4278M 18.2.2-1349-gb9b067bc 5a1940d32fa0 0081d9051628 2024-05-24T11:37:44.668 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 smithi039 running (2m) 58s ago 2m 55.2M 4278M 18.2.2-1349-gb9b067bc 5a1940d32fa0 107183c34eaa 2024-05-24T11:37:44.668 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 smithi053 running (113s) 58s ago 113s 57.4M 4279M 18.2.2-1349-gb9b067bc 5a1940d32fa0 f5eddb27331c 2024-05-24T11:37:44.668 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 smithi137 running (96s) 58s ago 96s 43.9M 4278M 18.2.2-1349-gb9b067bc 5a1940d32fa0 2a1e3c6ae060 2024-05-24T11:37:44.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.660+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe5bc0a11b0 msgr2=0x7fe5bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.660+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe5bc0a11b0 0x7fe5bc0a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc102060 tx=0x7fe5c000b410 comp rx=0 tx=0).stop 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.660+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 msgr2=0x7fe5cc19e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.660+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 0x7fe5cc19e4b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4007990 tx=0x7fe5c40094c0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.661+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 shutdown_connections 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.661+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe5bc0a11b0 0x7fe5bc0a3670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.661+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc102ba0 0x7fe5cc19e4b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.661+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5cc101000 0x7fe5cc196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.661+0000 7fe5d31d6700 1 --2- 172.21.15.39:0/3737281803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5cc102200 0x7fe5cc197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.661+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 >> 172.21.15.39:0/3737281803 conn(0x7fe5cc0fc5c0 msgr2=0x7fe5cc1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.661+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 shutdown_connections 2024-05-24T11:37:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:44.661+0000 7fe5d31d6700 1 -- 172.21.15.39:0/3737281803 wait complete. 2024-05-24T11:37:44.986 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:44 smithi039 ceph-mon[42919]: osdmap e79: 12 total, 12 up, 12 in 2024-05-24T11:37:45.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-24T11:37:45.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:44 smithi053 ceph-mon[47448]: osdmap e79: 12 total, 12 up, 12 in 2024-05-24T11:37:45.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:44 smithi137 ceph-mon[47907]: osdmap e79: 12 total, 12 up, 12 in 2024-05-24T11:37:45.283 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:45 smithi039 ceph-mon[42919]: pgmap v250: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-24T11:37:46.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:45 smithi039 ceph-mon[42919]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:46.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:45 smithi053 ceph-mon[47448]: pgmap v250: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-24T11:37:46.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:45 smithi053 ceph-mon[47448]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:46.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:45 smithi137 ceph-mon[47907]: pgmap v250: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-24T11:37:46.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:45 smithi137 ceph-mon[47907]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:46.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.317+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/4016394126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 msgr2=0x7ff1c8100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:46.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.317+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/4016394126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 0x7ff1c8100320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009990 tx=0x7ff1b8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.317+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/4016394126 shutdown_connections 2024-05-24T11:37:46.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.317+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/4016394126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c8101a20 0x7ff1c8108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.317+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/4016394126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 0x7ff1c8100320 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.317+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/4016394126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c8101060 0x7ff1c81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.317+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/4016394126 >> 172.21.15.39:0/4016394126 conn(0x7ff1c80fb4a0 msgr2=0x7ff1c80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:46.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.317+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/4016394126 shutdown_connections 2024-05-24T11:37:46.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.317+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/4016394126 wait complete. 2024-05-24T11:37:46.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cf9f5700 1 Processor -- start 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cf9f5700 1 -- start start 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cf9f5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 0x7ff1c8199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cf9f5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c8101060 0x7ff1c8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cf9f5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c8101a20 0x7ff1c81a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cf9f5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81a0c90 con 0x7ff1c80fff00 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cf9f5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81a0dd0 con 0x7ff1c8101060 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cf9f5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81a0f60 con 0x7ff1c8101a20 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cd791700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 0x7ff1c8199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cd791700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 0x7ff1c8199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50082/0 (socket says 172.21.15.39:50082) 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cd791700 1 -- 172.21.15.39:0/1226653435 learned_addr learned my addr 172.21.15.39:0/1226653435 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:46.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cdf92700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c8101a20 0x7ff1c81a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1ccf90700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c8101060 0x7ff1c8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cd791700 1 -- 172.21.15.39:0/1226653435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c8101a20 msgr2=0x7ff1c81a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cd791700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c8101a20 0x7ff1c81a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cd791700 1 -- 172.21.15.39:0/1226653435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c8101060 msgr2=0x7ff1c8199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cd791700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c8101060 0x7ff1c8199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cd791700 1 -- 172.21.15.39:0/1226653435 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b80095f0 con 0x7ff1c80fff00 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1ccf90700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c8101060 0x7ff1c8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.318+0000 7ff1cdf92700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c8101a20 0x7ff1c81a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.319+0000 7ff1cd791700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 0x7ff1c8199150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80059d0 tx=0x7ff1b8003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.319+0000 7ff1be7fc700 1 -- 172.21.15.39:0/1226653435 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b801d070 con 0x7ff1c80fff00 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.319+0000 7ff1be7fc700 1 -- 172.21.15.39:0/1226653435 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1b800bd10 con 0x7ff1c80fff00 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.319+0000 7ff1be7fc700 1 -- 172.21.15.39:0/1226653435 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800f460 con 0x7ff1c80fff00 2024-05-24T11:37:46.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.319+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c81a11e0 con 0x7ff1c80fff00 2024-05-24T11:37:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.319+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c81a1620 con 0x7ff1c80fff00 2024-05-24T11:37:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.320+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c804ec60 con 0x7ff1c80fff00 2024-05-24T11:37:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.321+0000 7ff1be7fc700 1 -- 172.21.15.39:0/1226653435 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff1b8022470 con 0x7ff1c80fff00 2024-05-24T11:37:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.321+0000 7ff1be7fc700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff1b40a11b0 0x7ff1b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:46.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.322+0000 7ff1ccf90700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff1b40a11b0 0x7ff1b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.322+0000 7ff1be7fc700 1 -- 172.21.15.39:0/1226653435 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff1b80ce380 con 0x7ff1c80fff00 2024-05-24T11:37:46.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.322+0000 7ff1ccf90700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff1b40a11b0 0x7ff1b40a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4007900 tx=0x7ff1c4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.325+0000 7ff1be7fc700 1 -- 172.21.15.39:0/1226653435 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff1b8097b40 con 0x7ff1c80fff00 2024-05-24T11:37:46.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.538+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1c8069090 con 0x7ff1b40a11b0 2024-05-24T11:37:46.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.543+0000 7ff1be7fc700 1 -- 172.21.15.39:0/1226653435 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7ff1c8069090 con 0x7ff1b40a11b0 2024-05-24T11:37:46.549 INFO:teuthology.orchestra.run.smithi039.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-24T11:37:46.549 INFO:teuthology.orchestra.run.smithi039.stdout:mds.all 12/12 60s ago 85s smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l;count:12 2024-05-24T11:37:46.549 INFO:teuthology.orchestra.run.smithi039.stdout:mgr 3/3 60s ago 5m smithi039=x;smithi053=y;smithi137=z;count:3 2024-05-24T11:37:46.549 INFO:teuthology.orchestra.run.smithi039.stdout:mon 3/3 60s ago 5m smithi039:172.21.15.39=a;smithi053:172.21.15.53=b;smithi137:172.21.15.137=c;count:3 2024-05-24T11:37:46.549 INFO:teuthology.orchestra.run.smithi039.stdout:osd 12 60s ago - 2024-05-24T11:37:46.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff1b40a11b0 msgr2=0x7ff1b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:46.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff1b40a11b0 0x7ff1b40a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4007900 tx=0x7ff1c4008040 comp rx=0 tx=0).stop 2024-05-24T11:37:46.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 msgr2=0x7ff1c8199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:46.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 0x7ff1c8199150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80059d0 tx=0x7ff1b8003b00 comp rx=0 tx=0).stop 2024-05-24T11:37:46.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 shutdown_connections 2024-05-24T11:37:46.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff1b40a11b0 0x7ff1b40a3670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c8101a20 0x7ff1c81a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1c80fff00 0x7ff1c8199150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 --2- 172.21.15.39:0/1226653435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1c8101060 0x7ff1c8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:46.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 >> 172.21.15.39:0/1226653435 conn(0x7ff1c80fb4a0 msgr2=0x7ff1c8068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:46.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 shutdown_connections 2024-05-24T11:37:46.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:46.545+0000 7ff1cf9f5700 1 -- 172.21.15.39:0/1226653435 wait complete. 2024-05-24T11:37:46.934 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-24T11:37:47.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:46 smithi053 ceph-mon[47448]: pgmap v251: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:47.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:46 smithi053 ceph-mon[47448]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:47.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:46 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:37:47.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:46 smithi137 ceph-mon[47907]: pgmap v251: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:47.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:46 smithi137 ceph-mon[47907]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:47.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:46 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:37:47.308 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:46 smithi039 ceph-mon[42919]: pgmap v251: 97 pgs: 97 active+clean; 639 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:46 smithi039 ceph-mon[42919]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:47.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:46 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:37:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.601+0000 7f6c5a669700 1 -- 172.21.15.39:0/1054201313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54101af0 msgr2=0x7f6c54103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.601+0000 7f6c5a669700 1 --2- 172.21.15.39:0/1054201313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54101af0 0x7f6c54103ee0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009990 tx=0x7f6c44009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.602+0000 7f6c5a669700 1 -- 172.21.15.39:0/1054201313 shutdown_connections 2024-05-24T11:37:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.602+0000 7f6c5a669700 1 --2- 172.21.15.39:0/1054201313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54106d50 0x7f6c54109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.602+0000 7f6c5a669700 1 --2- 172.21.15.39:0/1054201313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54101af0 0x7f6c54103ee0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.602+0000 7f6c5a669700 1 --2- 172.21.15.39:0/1054201313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c54104420 0x7f6c54106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.602+0000 7f6c5a669700 1 -- 172.21.15.39:0/1054201313 >> 172.21.15.39:0/1054201313 conn(0x7f6c540fb4a0 msgr2=0x7f6c540fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.602+0000 7f6c5a669700 1 -- 172.21.15.39:0/1054201313 shutdown_connections 2024-05-24T11:37:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.602+0000 7f6c5a669700 1 -- 172.21.15.39:0/1054201313 wait complete. 2024-05-24T11:37:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c5a669700 1 Processor -- start 2024-05-24T11:37:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c5a669700 1 -- start start 2024-05-24T11:37:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c5a669700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c54101af0 0x7f6c54196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c5a669700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54104420 0x7f6c541973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c5a669700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54106d50 0x7f6c5419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c5a669700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419e9f0 con 0x7f6c54106d50 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c5a669700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419eb30 con 0x7f6c54104420 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c5a669700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419ecc0 con 0x7f6c54101af0 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c58c06700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54106d50 0x7f6c5419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c58c06700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54106d50 0x7f6c5419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42300/0 (socket says 172.21.15.39:42300) 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c58c06700 1 -- 172.21.15.39:0/2602189372 learned_addr learned my addr 172.21.15.39:0/2602189372 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c4bfff700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54104420 0x7f6c541973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c53fff700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c54101af0 0x7f6c54196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c58c06700 1 -- 172.21.15.39:0/2602189372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c54101af0 msgr2=0x7f6c54196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c58c06700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c54101af0 0x7f6c54196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c58c06700 1 -- 172.21.15.39:0/2602189372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54104420 msgr2=0x7f6c541973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c58c06700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54104420 0x7f6c541973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.603+0000 7f6c58c06700 1 -- 172.21.15.39:0/2602189372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440095f0 con 0x7f6c54106d50 2024-05-24T11:37:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.604+0000 7f6c58c06700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54106d50 0x7f6c5419e420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007ef0 tx=0x7f6c4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.604+0000 7f6c51ffb700 1 -- 172.21.15.39:0/2602189372 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c017070 con 0x7f6c54106d50 2024-05-24T11:37:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.604+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5419efa0 con 0x7f6c54106d50 2024-05-24T11:37:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.604+0000 7f6c51ffb700 1 -- 172.21.15.39:0/2602189372 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c4c00f040 con 0x7f6c54106d50 2024-05-24T11:37:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.604+0000 7f6c51ffb700 1 -- 172.21.15.39:0/2602189372 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c004020 con 0x7f6c54106d50 2024-05-24T11:37:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.604+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5419f410 con 0x7f6c54106d50 2024-05-24T11:37:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.605+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c540fd020 con 0x7f6c54106d50 2024-05-24T11:37:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.606+0000 7f6c51ffb700 1 -- 172.21.15.39:0/2602189372 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6c4c00db20 con 0x7f6c54106d50 2024-05-24T11:37:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.607+0000 7f6c51ffb700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6c400a11b0 0x7f6c400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.607+0000 7f6c53fff700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6c400a11b0 0x7f6c400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.607+0000 7f6c51ffb700 1 -- 172.21.15.39:0/2602189372 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6c4c0cc4e0 con 0x7f6c54106d50 2024-05-24T11:37:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.608+0000 7f6c53fff700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6c400a11b0 0x7f6c400a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6c44006010 tx=0x7f6c4400b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.611+0000 7f6c51ffb700 1 -- 172.21.15.39:0/2602189372 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6c4c0996b0 con 0x7f6c54106d50 2024-05-24T11:37:48.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.820+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c5402ce10 con 0x7f6c400a11b0 2024-05-24T11:37:48.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.822+0000 7f6c51ffb700 1 -- 172.21.15.39:0/2602189372 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f6c5402ce10 con 0x7f6c400a11b0 2024-05-24T11:37:48.827 INFO:teuthology.orchestra.run.smithi039.stdout:HOST ADDR LABELS STATUS 2024-05-24T11:37:48.828 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 172.21.15.39 2024-05-24T11:37:48.828 INFO:teuthology.orchestra.run.smithi039.stdout:smithi053 172.21.15.53 2024-05-24T11:37:48.828 INFO:teuthology.orchestra.run.smithi039.stdout:smithi137 172.21.15.137 2024-05-24T11:37:48.828 INFO:teuthology.orchestra.run.smithi039.stdout:3 hosts in cluster 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6c400a11b0 msgr2=0x7f6c400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6c400a11b0 0x7f6c400a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6c44006010 tx=0x7f6c4400b610 comp rx=0 tx=0).stop 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54106d50 msgr2=0x7f6c5419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54106d50 0x7f6c5419e420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007ef0 tx=0x7f6c4c00d3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 shutdown_connections 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6c400a11b0 0x7f6c400a3670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c54101af0 0x7f6c54196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c54106d50 0x7f6c5419e420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 --2- 172.21.15.39:0/2602189372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c54104420 0x7f6c541973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 >> 172.21.15.39:0/2602189372 conn(0x7f6c540fb4a0 msgr2=0x7f6c541025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:48.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 shutdown_connections 2024-05-24T11:37:48.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:48.824+0000 7f6c5a669700 1 -- 172.21.15.39:0/2602189372 wait complete. 2024-05-24T11:37:49.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:48 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:48 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:48 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:48 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:48 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:48 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-24T11:37:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:48 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:48 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:48 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:48 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:48 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:48 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:48 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:48 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:48 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:48 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:48 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:48 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:49.448 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:49.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:49 smithi039 ceph-mon[42919]: pgmap v252: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:49.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:49 smithi039 ceph-mon[42919]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:49.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:49 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:37:49.998 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:49 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:37:49.998 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:49 smithi039 ceph-mon[42919]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:49 smithi053 ceph-mon[47448]: pgmap v252: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:49 smithi053 ceph-mon[47448]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:49 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:37:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:49 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:37:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:49 smithi053 ceph-mon[47448]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:50.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:49 smithi137 ceph-mon[47907]: pgmap v252: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:50.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:49 smithi137 ceph-mon[47907]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:50.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:49 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:37:50.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:49 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:37:50.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:49 smithi137 ceph-mon[47907]: from='mgr.14150 172.21.15.39:0/3733443041' entity='mgr.x' 2024-05-24T11:37:50.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.573+0000 7f73c2980700 1 -- 172.21.15.39:0/3678717806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 msgr2=0x7f73bc1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:50.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.573+0000 7f73c2980700 1 --2- 172.21.15.39:0/3678717806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 0x7f73bc1014e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009990 tx=0x7f73a4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.573+0000 7f73c2980700 1 -- 172.21.15.39:0/3678717806 shutdown_connections 2024-05-24T11:37:50.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.573+0000 7f73c2980700 1 --2- 172.21.15.39:0/3678717806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73bc0fff00 0x7f73bc100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.573+0000 7f73c2980700 1 --2- 172.21.15.39:0/3678717806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73bc101a20 0x7f73bc108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.573+0000 7f73c2980700 1 --2- 172.21.15.39:0/3678717806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 0x7f73bc1014e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.573+0000 7f73c2980700 1 -- 172.21.15.39:0/3678717806 >> 172.21.15.39:0/3678717806 conn(0x7f73bc0fb4a0 msgr2=0x7f73bc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:50.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.573+0000 7f73c2980700 1 -- 172.21.15.39:0/3678717806 shutdown_connections 2024-05-24T11:37:50.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.573+0000 7f73c2980700 1 -- 172.21.15.39:0/3678717806 wait complete. 2024-05-24T11:37:50.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73c2980700 1 Processor -- start 2024-05-24T11:37:50.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73c2980700 1 -- start start 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73c2980700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73bc0fff00 0x7f73bc103e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73c2980700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 0x7f73bc1024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73c2980700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73bc101a20 0x7f73bc102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73c2980700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc1031b0 con 0x7f73bc101060 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73c2980700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc103320 con 0x7f73bc0fff00 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73c2980700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc1a0df0 con 0x7f73bc101a20 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 0x7f73bc1024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 0x7f73bc1024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42326/0 (socket says 172.21.15.39:42326) 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73bb7fe700 1 -- 172.21.15.39:0/96238301 learned_addr learned my addr 172.21.15.39:0/96238301 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.574+0000 7f73bbfff700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73bc0fff00 0x7f73bc103e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73c0f1d700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73bc101a20 0x7f73bc102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73bb7fe700 1 -- 172.21.15.39:0/96238301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73bc101a20 msgr2=0x7f73bc102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73bb7fe700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73bc101a20 0x7f73bc102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73bb7fe700 1 -- 172.21.15.39:0/96238301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73bc0fff00 msgr2=0x7f73bc103e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73bb7fe700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73bc0fff00 0x7f73bc103e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73bb7fe700 1 -- 172.21.15.39:0/96238301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a40095f0 con 0x7f73bc101060 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73bb7fe700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 0x7f73bc1024d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73a400b5c0 tx=0x7f73a4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73b97fa700 1 -- 172.21.15.39:0/96238301 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a401d070 con 0x7f73bc101060 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73b97fa700 1 -- 172.21.15.39:0/96238301 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73a4004000 con 0x7f73bc101060 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc1a1020 con 0x7f73bc101060 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73b97fa700 1 -- 172.21.15.39:0/96238301 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4022c30 con 0x7f73bc101060 2024-05-24T11:37:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.575+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc1a1430 con 0x7f73bc101060 2024-05-24T11:37:50.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.576+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc04ec60 con 0x7f73bc101060 2024-05-24T11:37:50.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.577+0000 7f73b97fa700 1 -- 172.21.15.39:0/96238301 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f73a4022470 con 0x7f73bc101060 2024-05-24T11:37:50.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.578+0000 7f73b97fa700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73a80a11b0 0x7f73a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:50.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.578+0000 7f73bbfff700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73a80a11b0 0x7f73a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:50.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.578+0000 7f73b97fa700 1 -- 172.21.15.39:0/96238301 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f73a40cdfc0 con 0x7f73bc101060 2024-05-24T11:37:50.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.579+0000 7f73bbfff700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73a80a11b0 0x7f73a80a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00ac50 tx=0x7f73ac00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:50.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.583+0000 7f73b97fa700 1 -- 172.21.15.39:0/96238301 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f73a4097780 con 0x7f73bc101060 2024-05-24T11:37:50.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.794+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f73bc0fd6d0 con 0x7f73a80a11b0 2024-05-24T11:37:50.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.796+0000 7f73b97fa700 1 -- 172.21.15.39:0/96238301 <== mgr.14150 v2:172.21.15.39:6800/2090515453 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f73bc0fd6d0 con 0x7f73a80a11b0 2024-05-24T11:37:50.802 INFO:teuthology.orchestra.run.smithi039.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-24T11:37:50.802 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102QX4P0KGN 3726G No 2m ago Has a FileSystem, LVM detected 2024-05-24T11:37:50.802 INFO:teuthology.orchestra.run.smithi039.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 114s ago Has a FileSystem, LVM detected 2024-05-24T11:37:50.802 INFO:teuthology.orchestra.run.smithi039.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 98s ago Has a FileSystem, LVM detected 2024-05-24T11:37:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73a80a11b0 msgr2=0x7f73a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73a80a11b0 0x7f73a80a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00ac50 tx=0x7f73ac00a380 comp rx=0 tx=0).stop 2024-05-24T11:37:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 msgr2=0x7f73bc1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 0x7f73bc1024d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73a400b5c0 tx=0x7f73a4005740 comp rx=0 tx=0).stop 2024-05-24T11:37:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 shutdown_connections 2024-05-24T11:37:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73a80a11b0 0x7f73a80a3670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73bc101a20 0x7f73bc102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73bc0fff00 0x7f73bc103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 --2- 172.21.15.39:0/96238301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73bc101060 0x7f73bc1024d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:50.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.798+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 >> 172.21.15.39:0/96238301 conn(0x7f73bc0fb4a0 msgr2=0x7f73bc0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:50.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.799+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 shutdown_connections 2024-05-24T11:37:50.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:50.799+0000 7f73c2980700 1 -- 172.21.15.39:0/96238301 wait complete. 2024-05-24T11:37:51.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:50 smithi039 ceph-mon[42919]: pgmap v253: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:51.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:50 smithi039 ceph-mon[42919]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:51.177 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-24T11:37:51.187 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:37:51.187 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-24T11:37:51.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:50 smithi053 ceph-mon[47448]: pgmap v253: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:51.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:50 smithi053 ceph-mon[47448]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:51.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:50 smithi137 ceph-mon[47907]: pgmap v253: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:51.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:50 smithi137 ceph-mon[47907]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:37:51.418 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.478+0000 7f38af9bd700 1 -- 172.21.15.39:0/3489374358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102200 msgr2=0x7f38a8102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.478+0000 7f38af9bd700 1 --2- 172.21.15.39:0/3489374358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102200 0x7f38a8102660 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f38980099c0 tx=0x7f3898009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.478+0000 7f38af9bd700 1 -- 172.21.15.39:0/3489374358 shutdown_connections 2024-05-24T11:37:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.478+0000 7f38af9bd700 1 --2- 172.21.15.39:0/3489374358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38a8101000 0x7f38a8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.478+0000 7f38af9bd700 1 --2- 172.21.15.39:0/3489374358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8102ba0 0x7f38a8109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.478+0000 7f38af9bd700 1 --2- 172.21.15.39:0/3489374358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102200 0x7f38a8102660 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.478+0000 7f38af9bd700 1 -- 172.21.15.39:0/3489374358 >> 172.21.15.39:0/3489374358 conn(0x7f38a80fc5c0 msgr2=0x7f38a80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:52.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.478+0000 7f38af9bd700 1 -- 172.21.15.39:0/3489374358 shutdown_connections 2024-05-24T11:37:52.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.478+0000 7f38af9bd700 1 -- 172.21.15.39:0/3489374358 wait complete. 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38af9bd700 1 Processor -- start 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38af9bd700 1 -- start start 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38af9bd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38a8101000 0x7f38a8196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38af9bd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8102200 0x7f38a81973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38af9bd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102ba0 0x7f38a819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38af9bd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a819e9c0 con 0x7f38a8102ba0 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38af9bd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a819eb00 con 0x7f38a8102200 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38af9bd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a819ec90 con 0x7f38a8101000 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38adf5a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102ba0 0x7f38a819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:52.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38adf5a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102ba0 0x7f38a819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42356/0 (socket says 172.21.15.39:42356) 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38adf5a700 1 -- 172.21.15.39:0/1602967623 learned_addr learned my addr 172.21.15.39:0/1602967623 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.479+0000 7f38acf58700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8102200 0x7f38a81973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38ad759700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38a8101000 0x7f38a8196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38adf5a700 1 -- 172.21.15.39:0/1602967623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38a8101000 msgr2=0x7f38a8196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38adf5a700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38a8101000 0x7f38a8196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38adf5a700 1 -- 172.21.15.39:0/1602967623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8102200 msgr2=0x7f38a81973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38adf5a700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8102200 0x7f38a81973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38adf5a700 1 -- 172.21.15.39:0/1602967623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980095f0 con 0x7f38a8102ba0 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38ad759700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38a8101000 0x7f38a8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38adf5a700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102ba0 0x7f38a819e3f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f38a0007f10 tx=0x7f38a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f389e7fc700 1 -- 172.21.15.39:0/1602967623 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0017070 con 0x7f38a8102ba0 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f389e7fc700 1 -- 172.21.15.39:0/1602967623 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38a000f040 con 0x7f38a8102ba0 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a819ef70 con 0x7f38a8102ba0 2024-05-24T11:37:52.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f389e7fc700 1 -- 172.21.15.39:0/1602967623 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0005710 con 0x7f38a8102ba0 2024-05-24T11:37:52.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.480+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a819f3e0 con 0x7f38a8102ba0 2024-05-24T11:37:52.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.481+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a8108490 con 0x7f38a8102ba0 2024-05-24T11:37:52.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.482+0000 7f389e7fc700 1 -- 172.21.15.39:0/1602967623 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f38a0004ad0 con 0x7f38a8102ba0 2024-05-24T11:37:52.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.483+0000 7f389e7fc700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f38940a1230 0x7f38940a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:52.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.483+0000 7f389e7fc700 1 -- 172.21.15.39:0/1602967623 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f38a00cc800 con 0x7f38a8102ba0 2024-05-24T11:37:52.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.483+0000 7f38ad759700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f38940a1230 0x7f38940a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:52.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.483+0000 7f38ad759700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f38940a1230 0x7f38940a36f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38a8102060 tx=0x7f38a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:52.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.486+0000 7f389e7fc700 1 -- 172.21.15.39:0/1602967623 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f38a0095e30 con 0x7f38a8102ba0 2024-05-24T11:37:52.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.738+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f38a810c710 con 0x7f38a8102ba0 2024-05-24T11:37:52.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.738+0000 7f389e7fc700 1 -- 172.21.15.39:0/1602967623 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2919 (secure 0 0 0) 0x7f38a0099450 con 0x7f38a8102ba0 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:e27 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:legacy client fscid: 1 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:Filesystem 'cephfs' (1) 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:fs_name cephfs 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 27 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-05-24T11:36:48.500682+0000 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-05-24T11:36:55.584150+0000 2024-05-24T11:37:52.745 INFO:teuthology.orchestra.run.smithi039.stdout:tableserver 0 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:root 0 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:session_timeout 300 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:session_autoclose 300 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:max_file_size 1099511627776 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:max_xattr_size 65536 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:required_client_features {} 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure 0 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure_osd_epoch 0 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:max_mds 5 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:in 0,1,2,3,4 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:up {0=24542,1=24511,2=24506,3=24490,4=14631} 2024-05-24T11:37:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:failed 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:damaged 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:stopped 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:data_pools [3] 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:metadata_pool 2 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:inline_data disabled 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:balancer 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:bal_rank_mask -1 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:standby_count_wanted 0 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b{0:24542} state up:active seq 3 addr [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.f{0:24478} state up:standby-replay seq 1 addr [v2:172.21.15.137:6834/1490077205,v1:172.21.15.137:6835/1490077205] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.i{1:24511} state up:active seq 5 addr [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.e{1:24476} state up:standby-replay seq 1 addr [v2:172.21.15.53:6834/3629179598,v1:172.21.15.53:6835/3629179598] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.h{2:24506} state up:active seq 6 addr [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.l{2:24442} state up:standby-replay seq 1 addr [v2:172.21.15.137:6832/451866213,v1:172.21.15.137:6833/451866213] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c{3:24490} state up:active seq 7 addr [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a{3:14643} state up:standby-replay seq 1 addr [v2:172.21.15.39:6840/3024712815,v1:172.21.15.39:6841/3024712815] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.g{4:14631} state up:active seq 5 addr [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d{4:14571} state up:standby-replay seq 1 addr [v2:172.21.15.39:6836/3631576511,v1:172.21.15.39:6837/3631576511] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout:Standby daemons: 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.k{-1:14553} state up:standby seq 1 addr [v2:172.21.15.53:6832/2983171503,v1:172.21.15.53:6833/2983171503] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.748 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.j{-1:14559} state up:standby seq 1 addr [v2:172.21.15.39:6834/1458882734,v1:172.21.15.39:6835/1458882734] compat {c=[1],r=[1],i=[7ff]}] 2024-05-24T11:37:52.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.740+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f38940a1230 msgr2=0x7f38940a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:52.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.740+0000 7f38af9bd700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f38940a1230 0x7f38940a36f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38a8102060 tx=0x7f38a4005f90 comp rx=0 tx=0).stop 2024-05-24T11:37:52.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.740+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102ba0 msgr2=0x7f38a819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:52.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.740+0000 7f38af9bd700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102ba0 0x7f38a819e3f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f38a0007f10 tx=0x7f38a000d3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.741+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 shutdown_connections 2024-05-24T11:37:52.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.741+0000 7f38af9bd700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f38940a1230 0x7f38940a36f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.741+0000 7f38af9bd700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38a8101000 0x7f38a8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.741+0000 7f38af9bd700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38a8102ba0 0x7f38a819e3f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.741+0000 7f38af9bd700 1 --2- 172.21.15.39:0/1602967623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a8102200 0x7f38a81973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:52.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.741+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 >> 172.21.15.39:0/1602967623 conn(0x7f38a80fc5c0 msgr2=0x7f38a81055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:52.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.741+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 shutdown_connections 2024-05-24T11:37:52.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:52.741+0000 7f38af9bd700 1 -- 172.21.15.39:0/1602967623 wait complete. 2024-05-24T11:37:52.750 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-24T11:37:52.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:52 smithi039 ceph-mon[42919]: pgmap v254: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:52.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:52 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1602967623' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-24T11:37:53.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-24T11:37:53.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:52 smithi053 ceph-mon[47448]: pgmap v254: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:53.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:52 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1602967623' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-24T11:37:53.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:52 smithi137 ceph-mon[47907]: pgmap v254: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:53.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:52 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1602967623' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-24T11:37:53.397 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:37:54.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.387+0000 7fa2a754e700 1 -- 172.21.15.39:0/3727475367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 msgr2=0x7fa2a0108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:54.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.387+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3727475367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 0x7fa2a0108340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa29800b0a0 tx=0x7fa29800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.387+0000 7fa2a754e700 1 -- 172.21.15.39:0/3727475367 shutdown_connections 2024-05-24T11:37:54.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.387+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3727475367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a00fff00 0x7fa2a0100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.387+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3727475367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 0x7fa2a0108340 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.387+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3727475367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2a0101060 0x7fa2a01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.387+0000 7fa2a754e700 1 -- 172.21.15.39:0/3727475367 >> 172.21.15.39:0/3727475367 conn(0x7fa2a00fb4a0 msgr2=0x7fa2a00fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 -- 172.21.15.39:0/3727475367 shutdown_connections 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 -- 172.21.15.39:0/3727475367 wait complete. 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 Processor -- start 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 -- start start 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a00fff00 0x7fa2a0196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2a0101060 0x7fa2a0197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:54.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 0x7fa2a019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a019eac0 con 0x7fa2a0101a20 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a019ec00 con 0x7fa2a0101060 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a754e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a019ed90 con 0x7fa2a00fff00 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a5aeb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 0x7fa2a019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.388+0000 7fa2a5aeb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 0x7fa2a019e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42376/0 (socket says 172.21.15.39:42376) 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a5aeb700 1 -- 172.21.15.39:0/3460116171 learned_addr learned my addr 172.21.15.39:0/3460116171 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a4ae9700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2a0101060 0x7fa2a0197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a52ea700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a00fff00 0x7fa2a0196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a5aeb700 1 -- 172.21.15.39:0/3460116171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a00fff00 msgr2=0x7fa2a0196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a5aeb700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a00fff00 0x7fa2a0196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a5aeb700 1 -- 172.21.15.39:0/3460116171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2a0101060 msgr2=0x7fa2a0197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:54.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a5aeb700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2a0101060 0x7fa2a0197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a5aeb700 1 -- 172.21.15.39:0/3460116171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa298009d00 con 0x7fa2a0101a20 2024-05-24T11:37:54.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a4ae9700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2a0101060 0x7fa2a0197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:54.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a52ea700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a00fff00 0x7fa2a0196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:54.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a5aeb700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 0x7fa2a019e4f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa298004d40 tx=0x7fa298004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:54.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2927fc700 1 -- 172.21.15.39:0/3460116171 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29800e040 con 0x7fa2a0101a20 2024-05-24T11:37:54.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2927fc700 1 -- 172.21.15.39:0/3460116171 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2980058e0 con 0x7fa2a0101a20 2024-05-24T11:37:54.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a019f010 con 0x7fa2a0101a20 2024-05-24T11:37:54.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2927fc700 1 -- 172.21.15.39:0/3460116171 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29801db90 con 0x7fa2a0101a20 2024-05-24T11:37:54.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.389+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a019f420 con 0x7fa2a0101a20 2024-05-24T11:37:54.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.390+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a004ec60 con 0x7fa2a0101a20 2024-05-24T11:37:54.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.391+0000 7fa2927fc700 1 -- 172.21.15.39:0/3460116171 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa298005450 con 0x7fa2a0101a20 2024-05-24T11:37:54.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.392+0000 7fa2927fc700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa28c0a11b0 0x7fa28c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:54.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.392+0000 7fa2927fc700 1 -- 172.21.15.39:0/3460116171 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa2980ceca0 con 0x7fa2a0101a20 2024-05-24T11:37:54.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.392+0000 7fa2a52ea700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa28c0a11b0 0x7fa28c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:54.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.392+0000 7fa2a52ea700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa28c0a11b0 0x7fa28c0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa29c007900 tx=0x7fa29c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:54.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.395+0000 7fa2927fc700 1 -- 172.21.15.39:0/3460116171 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa298098460 con 0x7fa2a0101a20 2024-05-24T11:37:54.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.598+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fa2a010b840 con 0x7fa2a0101a20 2024-05-24T11:37:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.599+0000 7fa2927fc700 1 -- 172.21.15.39:0/3460116171 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v79) v1 ==== 56+0+5212 (secure 0 0 0) 0x7fa29809ba80 con 0x7fa2a0101a20 2024-05-24T11:37:54.605 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 79 2024-05-24T11:37:54.605 INFO:teuthology.orchestra.run.smithi039.stdout:fsid f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:37:54.605 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-05-24T11:30:55.204289+0000 2024-05-24T11:37:54.605 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-05-24T11:37:43.820939+0000 2024-05-24T11:37:54.605 INFO:teuthology.orchestra.run.smithi039.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:crush_version 26 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:full_ratio 0.95 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:backfillfull_ratio 0.9 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:nearfull_ratio 0.85 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:require_min_compat_client luminous 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:min_compat_client luminous 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:require_osd_release reef 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:stretch_mode_enabled false 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 76 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-24T11:37:54.606 INFO:teuthology.orchestra.run.smithi039.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:max_osd 12 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 up in weight 1 up_from 8 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6802/618291952,v1:172.21.15.39:6803/618291952] [v2:172.21.15.39:6804/618291952,v1:172.21.15.39:6805/618291952] exists,up 010b8819-8872-412c-b489-b1b1389d3e7f 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 up in weight 1 up_from 12 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/3467330431,v1:172.21.15.53:6801/3467330431] [v2:172.21.15.53:6802/3467330431,v1:172.21.15.53:6803/3467330431] exists,up 9572166c-feb0-4545-b576-9909a66bcdee 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 up in weight 1 up_from 17 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/3821366387,v1:172.21.15.137:6801/3821366387] [v2:172.21.15.137:6802/3821366387,v1:172.21.15.137:6803/3821366387] exists,up 7e133e51-8deb-40e3-b5ee-813fd6dfb344 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 up in weight 1 up_from 25 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6810/445163923,v1:172.21.15.39:6811/445163923] [v2:172.21.15.39:6812/445163923,v1:172.21.15.39:6813/445163923] exists,up 29168261-54e3-4898-8f65-9b35e0e4a247 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 up in weight 1 up_from 32 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/2421884743,v1:172.21.15.53:6809/2421884743] [v2:172.21.15.53:6810/2421884743,v1:172.21.15.53:6811/2421884743] exists,up cc68e143-c059-4fae-9d40-326fbd44dd8c 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 up in weight 1 up_from 36 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6808/173299621,v1:172.21.15.137:6809/173299621] [v2:172.21.15.137:6810/173299621,v1:172.21.15.137:6811/173299621] exists,up d9f89542-51c6-4d02-a245-c6fb39f4d4e0 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 up in weight 1 up_from 43 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6818/2437170915,v1:172.21.15.39:6819/2437170915] [v2:172.21.15.39:6820/2437170915,v1:172.21.15.39:6821/2437170915] exists,up eb26433d-c0d3-4cbe-ba66-2b1a27944123 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 up in weight 1 up_from 47 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/1522585706,v1:172.21.15.53:6817/1522585706] [v2:172.21.15.53:6818/1522585706,v1:172.21.15.53:6819/1522585706] exists,up ab3690ac-bcb8-4bee-93b1-d2c729919262 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 up in weight 1 up_from 54 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/753285325,v1:172.21.15.137:6817/753285325] [v2:172.21.15.137:6818/753285325,v1:172.21.15.137:6819/753285325] exists,up 7ef14bbb-dbe7-497f-b170-89772a0b6ec0 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 up in weight 1 up_from 61 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6826/1424013773,v1:172.21.15.39:6827/1424013773] [v2:172.21.15.39:6828/1424013773,v1:172.21.15.39:6829/1424013773] exists,up 32dc829d-7871-4c18-890f-82f263600944 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 up in weight 1 up_from 65 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/3257679839,v1:172.21.15.53:6825/3257679839] [v2:172.21.15.53:6826/3257679839,v1:172.21.15.53:6827/3257679839] exists,up f1b20b6f-c19f-4c58-aafa-feb7cc027de3 2024-05-24T11:37:54.607 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 up in weight 1 up_from 72 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/2768517457,v1:172.21.15.137:6825/2768517457] [v2:172.21.15.137:6826/2768517457,v1:172.21.15.137:6827/2768517457] exists,up 78bc93a0-953b-478a-9d7e-25a4005fd6a7 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.6 [8,11] 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.8 [2,11] 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.1 [4,9] 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.11 [5,9] 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.1d [4,8] 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.24 [4,9] 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3982984387 expires 2024-05-25T11:31:41.814659+0000 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/1790202809 expires 2024-05-25T11:31:20.894336+0000 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/3960296835 expires 2024-05-25T11:31:41.814659+0000 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1012810399 expires 2024-05-25T11:31:20.894336+0000 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1559144405 expires 2024-05-25T11:31:41.814659+0000 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/1790202809 expires 2024-05-25T11:31:20.894336+0000 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/2910856381 expires 2024-05-25T11:31:20.894336+0000 2024-05-24T11:37:54.608 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3563042541 expires 2024-05-25T11:31:41.814659+0000 2024-05-24T11:37:54.609 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/3960296835 expires 2024-05-25T11:31:41.814659+0000 2024-05-24T11:37:54.609 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/2243717102 expires 2024-05-25T11:31:20.894336+0000 2024-05-24T11:37:54.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa28c0a11b0 msgr2=0x7fa28c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:54.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa28c0a11b0 0x7fa28c0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa29c007900 tx=0x7fa29c008040 comp rx=0 tx=0).stop 2024-05-24T11:37:54.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 msgr2=0x7fa2a019e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 0x7fa2a019e4f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa298004d40 tx=0x7fa298004e20 comp rx=0 tx=0).stop 2024-05-24T11:37:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 shutdown_connections 2024-05-24T11:37:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa28c0a11b0 0x7fa28c0a3670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a00fff00 0x7fa2a0196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0101a20 0x7fa2a019e4f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 --2- 172.21.15.39:0/3460116171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2a0101060 0x7fa2a0197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.601+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 >> 172.21.15.39:0/3460116171 conn(0x7fa2a00fb4a0 msgr2=0x7fa2a0068950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.602+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 shutdown_connections 2024-05-24T11:37:54.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:54.602+0000 7fa2a754e700 1 -- 172.21.15.39:0/3460116171 wait complete. 2024-05-24T11:37:55.027 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-24T11:37:55.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:37:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.166+0000 7f736bcb0700 1 -- 172.21.15.39:0/798232152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 msgr2=0x7f7364101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.166+0000 7f736bcb0700 1 --2- 172.21.15.39:0/798232152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 0x7f7364101290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7360009990 tx=0x7f7360009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.166+0000 7f736bcb0700 1 -- 172.21.15.39:0/798232152 shutdown_connections 2024-05-24T11:37:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.166+0000 7f736bcb0700 1 --2- 172.21.15.39:0/798232152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7364102980 0x7f73641092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.166+0000 7f736bcb0700 1 --2- 172.21.15.39:0/798232152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 0x7f7364101290 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.166+0000 7f736bcb0700 1 --2- 172.21.15.39:0/798232152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364101fc0 0x7f7364102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.166+0000 7f736bcb0700 1 -- 172.21.15.39:0/798232152 >> 172.21.15.39:0/798232152 conn(0x7f73640fc400 msgr2=0x7f73640fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.166+0000 7f736bcb0700 1 -- 172.21.15.39:0/798232152 shutdown_connections 2024-05-24T11:37:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.166+0000 7f736bcb0700 1 -- 172.21.15.39:0/798232152 wait complete. 2024-05-24T11:37:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736bcb0700 1 Processor -- start 2024-05-24T11:37:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736bcb0700 1 -- start start 2024-05-24T11:37:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736bcb0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 0x7f7364196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736bcb0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7364101fc0 0x7f73641972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736bcb0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364102980 0x7f736419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736bcb0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736419e860 con 0x7f7364100e70 2024-05-24T11:37:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736bcb0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736419e9d0 con 0x7f7364102980 2024-05-24T11:37:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736bcb0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736419eb90 con 0x7f7364101fc0 2024-05-24T11:37:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f7369a4c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 0x7f7364196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f7369a4c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 0x7f7364196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42398/0 (socket says 172.21.15.39:42398) 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f7369a4c700 1 -- 172.21.15.39:0/1857063986 learned_addr learned my addr 172.21.15.39:0/1857063986 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736a24d700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364102980 0x7f736419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736924b700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7364101fc0 0x7f73641972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f7369a4c700 1 -- 172.21.15.39:0/1857063986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7364101fc0 msgr2=0x7f73641972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f7369a4c700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7364101fc0 0x7f73641972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f7369a4c700 1 -- 172.21.15.39:0/1857063986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364102980 msgr2=0x7f736419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f7369a4c700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364102980 0x7f736419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f7369a4c700 1 -- 172.21.15.39:0/1857063986 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600095f0 con 0x7f7364100e70 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f736a24d700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364102980 0x7f736419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.167+0000 7f7369a4c700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 0x7f7364196d70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f736000b5c0 tx=0x7f73600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.168+0000 7f7356ffd700 1 -- 172.21.15.39:0/1857063986 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736001d070 con 0x7f7364100e70 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.168+0000 7f7356ffd700 1 -- 172.21.15.39:0/1857063986 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7360003f40 con 0x7f7364100e70 2024-05-24T11:37:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.168+0000 7f7356ffd700 1 -- 172.21.15.39:0/1857063986 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360022c60 con 0x7f7364100e70 2024-05-24T11:37:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.168+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736419ee10 con 0x7f7364100e70 2024-05-24T11:37:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.168+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736419f180 con 0x7f7364100e70 2024-05-24T11:37:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.169+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7364108320 con 0x7f7364100e70 2024-05-24T11:37:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.170+0000 7f7356ffd700 1 -- 172.21.15.39:0/1857063986 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f73600040b0 con 0x7f7364100e70 2024-05-24T11:37:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.170+0000 7f7356ffd700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73500a11b0 0x7f73500a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.170+0000 7f736924b700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73500a11b0 0x7f73500a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.170+0000 7f7356ffd700 1 -- 172.21.15.39:0/1857063986 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f73600ce670 con 0x7f7364100e70 2024-05-24T11:37:55.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.171+0000 7f736924b700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73500a11b0 0x7f73500a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7358005950 tx=0x7f73580058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:55.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.174+0000 7f7356ffd700 1 -- 172.21.15.39:0/1857063986 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7360097e30 con 0x7f7364100e70 2024-05-24T11:37:55.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:54 smithi053 ceph-mon[47448]: pgmap v255: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:55.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:54 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3460116171' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-24T11:37:55.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:54 smithi137 ceph-mon[47907]: pgmap v255: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:55.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:54 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3460116171' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-24T11:37:55.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:54 smithi039 ceph-mon[42919]: pgmap v255: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:55.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:54 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3460116171' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-24T11:37:55.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.385+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7364068060 con 0x7f7364100e70 2024-05-24T11:37:55.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.386+0000 7f7356ffd700 1 -- 172.21.15.39:0/1857063986 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22344 (secure 0 0 0) 0x7f736009b450 con 0x7f7364100e70 2024-05-24T11:37:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:55.392 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:37:43.820939+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:37:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73500a11b0 msgr2=0x7f73500a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73500a11b0 0x7f73500a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7358005950 tx=0x7f73580058e0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 msgr2=0x7f7364196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 0x7f7364196d70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f736000b5c0 tx=0x7f73600057f0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 shutdown_connections 2024-05-24T11:37:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73500a11b0 0x7f73500a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364102980 0x7f736419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364100e70 0x7f7364196d70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 --2- 172.21.15.39:0/1857063986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7364101fc0 0x7f73641972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 >> 172.21.15.39:0/1857063986 conn(0x7f73640fc400 msgr2=0x7f7364105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 shutdown_connections 2024-05-24T11:37:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.388+0000 7f736bcb0700 1 -- 172.21.15.39:0/1857063986 wait complete. 2024-05-24T11:37:55.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-24T11:36:46.509049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-24T11:36:47.866055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:37:55.408 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-24T11:37:55.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.531+0000 7f63435da700 1 -- 172.21.15.39:0/4206686964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c104370 msgr2=0x7f633c106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.531+0000 7f63435da700 1 --2- 172.21.15.39:0/4206686964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c104370 0x7f633c106760 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f632c009990 tx=0x7f632c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.531+0000 7f63435da700 1 -- 172.21.15.39:0/4206686964 shutdown_connections 2024-05-24T11:37:55.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.531+0000 7f63435da700 1 --2- 172.21.15.39:0/4206686964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f633c106ca0 0x7f633c1090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.531+0000 7f63435da700 1 --2- 172.21.15.39:0/4206686964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f633c101a40 0x7f633c103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.531+0000 7f63435da700 1 --2- 172.21.15.39:0/4206686964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c104370 0x7f633c106760 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.531+0000 7f63435da700 1 -- 172.21.15.39:0/4206686964 >> 172.21.15.39:0/4206686964 conn(0x7f633c0fb3f0 msgr2=0x7f633c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.531+0000 7f63435da700 1 -- 172.21.15.39:0/4206686964 shutdown_connections 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.531+0000 7f63435da700 1 -- 172.21.15.39:0/4206686964 wait complete. 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f63435da700 1 Processor -- start 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f63435da700 1 -- start start 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f63435da700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f633c101a40 0x7f633c101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f63435da700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f633c104370 0x7f633c0ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f63435da700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c106ca0 0x7f633c0fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f63435da700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1006d0 con 0x7f633c106ca0 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f63435da700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c100840 con 0x7f633c104370 2024-05-24T11:37:55.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f63435da700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1a0e30 con 0x7f633c101a40 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6341b77700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c106ca0 0x7f633c0fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6341b77700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c106ca0 0x7f633c0fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42412/0 (socket says 172.21.15.39:42412) 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6341b77700 1 -- 172.21.15.39:0/1111599581 learned_addr learned my addr 172.21.15.39:0/1111599581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6341376700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f633c101a40 0x7f633c101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6340b75700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f633c104370 0x7f633c0ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6341b77700 1 -- 172.21.15.39:0/1111599581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f633c101a40 msgr2=0x7f633c101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6341b77700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f633c101a40 0x7f633c101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6341b77700 1 -- 172.21.15.39:0/1111599581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f633c104370 msgr2=0x7f633c0ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6341b77700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f633c104370 0x7f633c0ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.532+0000 7f6341b77700 1 -- 172.21.15.39:0/1111599581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0095f0 con 0x7f633c106ca0 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.533+0000 7f6341b77700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c106ca0 0x7f633c0fff30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6334007d70 tx=0x7f633400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.533+0000 7f63327fc700 1 -- 172.21.15.39:0/1111599581 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633400ad40 con 0x7f633c106ca0 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.533+0000 7f63327fc700 1 -- 172.21.15.39:0/1111599581 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6334007e60 con 0x7f633c106ca0 2024-05-24T11:37:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.533+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c1a10c0 con 0x7f633c106ca0 2024-05-24T11:37:55.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.533+0000 7f63327fc700 1 -- 172.21.15.39:0/1111599581 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340115e0 con 0x7f633c106ca0 2024-05-24T11:37:55.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.533+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c1a15d0 con 0x7f633c106ca0 2024-05-24T11:37:55.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.534+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c0fcf70 con 0x7f633c106ca0 2024-05-24T11:37:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.535+0000 7f63327fc700 1 -- 172.21.15.39:0/1111599581 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6334012940 con 0x7f633c106ca0 2024-05-24T11:37:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.535+0000 7f63327fc700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f63280a1230 0x7f63280a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.535+0000 7f63327fc700 1 -- 172.21.15.39:0/1111599581 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f63340cce40 con 0x7f633c106ca0 2024-05-24T11:37:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.535+0000 7f6341376700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f63280a1230 0x7f63280a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.536+0000 7f6341376700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f63280a1230 0x7f63280a36f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6338005950 tx=0x7f633800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:55.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.539+0000 7f63327fc700 1 -- 172.21.15.39:0/1111599581 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6334096680 con 0x7f633c106ca0 2024-05-24T11:37:55.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.746+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f633c10c600 con 0x7f633c106ca0 2024-05-24T11:37:55.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.746+0000 7f63327fc700 1 -- 172.21.15.39:0/1111599581 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63340160a0 con 0x7f633c106ca0 2024-05-24T11:37:55.752 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:37:55.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.748+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f63280a1230 msgr2=0x7f63280a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.748+0000 7f63435da700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f63280a1230 0x7f63280a36f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6338005950 tx=0x7f633800a300 comp rx=0 tx=0).stop 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.748+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c106ca0 msgr2=0x7f633c0fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.748+0000 7f63435da700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c106ca0 0x7f633c0fff30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6334007d70 tx=0x7f633400a340 comp rx=0 tx=0).stop 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.749+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 shutdown_connections 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.749+0000 7f63435da700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f63280a1230 0x7f63280a36f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.749+0000 7f63435da700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f633c101a40 0x7f633c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.749+0000 7f63435da700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c106ca0 0x7f633c0fff30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.749+0000 7f63435da700 1 --2- 172.21.15.39:0/1111599581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f633c104370 0x7f633c0ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.749+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 >> 172.21.15.39:0/1111599581 conn(0x7f633c0fb3f0 msgr2=0x7f633c102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.749+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 shutdown_connections 2024-05-24T11:37:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.749+0000 7f63435da700 1 -- 172.21.15.39:0/1111599581 wait complete. 2024-05-24T11:37:55.767 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-24T11:37:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.891+0000 7fddd7bca700 1 -- 172.21.15.39:0/2914353414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 msgr2=0x7fddd01013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.891+0000 7fddd7bca700 1 --2- 172.21.15.39:0/2914353414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 0x7fddd01013a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0099c0 tx=0x7fddcc009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.891+0000 7fddd7bca700 1 -- 172.21.15.39:0/2914353414 shutdown_connections 2024-05-24T11:37:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.891+0000 7fddd7bca700 1 --2- 172.21.15.39:0/2914353414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddd0102b10 0x7fddd01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.891+0000 7fddd7bca700 1 --2- 172.21.15.39:0/2914353414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 0x7fddd01013a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.891+0000 7fddd7bca700 1 --2- 172.21.15.39:0/2914353414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0102170 0x7fddd01025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.891+0000 7fddd7bca700 1 -- 172.21.15.39:0/2914353414 >> 172.21.15.39:0/2914353414 conn(0x7fddd00fc4f0 msgr2=0x7fddd00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.891+0000 7fddd7bca700 1 -- 172.21.15.39:0/2914353414 shutdown_connections 2024-05-24T11:37:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.891+0000 7fddd7bca700 1 -- 172.21.15.39:0/2914353414 wait complete. 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd7bca700 1 Processor -- start 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd7bca700 1 -- start start 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd7bca700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 0x7fddd0196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd7bca700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddd0102170 0x7fddd0197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd7bca700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0102b10 0x7fddd019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd7bca700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd019e9e0 con 0x7fddd0100f80 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd7bca700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd019eb20 con 0x7fddd0102b10 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd7bca700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd019ecb0 con 0x7fddd0102170 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5966700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 0x7fddd0196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5966700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 0x7fddd0196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42432/0 (socket says 172.21.15.39:42432) 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5966700 1 -- 172.21.15.39:0/233751649 learned_addr learned my addr 172.21.15.39:0/233751649 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd6167700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0102b10 0x7fddd019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5165700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddd0102170 0x7fddd0197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5966700 1 -- 172.21.15.39:0/233751649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddd0102170 msgr2=0x7fddd0197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5966700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddd0102170 0x7fddd0197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5966700 1 -- 172.21.15.39:0/233751649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0102b10 msgr2=0x7fddd019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5966700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0102b10 0x7fddd019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5966700 1 -- 172.21.15.39:0/233751649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc0095f0 con 0x7fddd0100f80 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5165700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddd0102170 0x7fddd0197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd6167700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0102b10 0x7fddd019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.892+0000 7fddd5966700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 0x7fddd0196f00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009f50 tx=0x7fddcc005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.893+0000 7fddc6ffd700 1 -- 172.21.15.39:0/233751649 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc01d070 con 0x7fddd0100f80 2024-05-24T11:37:55.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.893+0000 7fddc6ffd700 1 -- 172.21.15.39:0/233751649 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fddcc00bda0 con 0x7fddd0100f80 2024-05-24T11:37:55.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.893+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd019ef30 con 0x7fddd0100f80 2024-05-24T11:37:55.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.893+0000 7fddc6ffd700 1 -- 172.21.15.39:0/233751649 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc022e30 con 0x7fddd0100f80 2024-05-24T11:37:55.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.893+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd019f2f0 con 0x7fddd0100f80 2024-05-24T11:37:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.894+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd01084f0 con 0x7fddd0100f80 2024-05-24T11:37:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.895+0000 7fddc6ffd700 1 -- 172.21.15.39:0/233751649 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fddcc022600 con 0x7fddd0100f80 2024-05-24T11:37:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.896+0000 7fddc6ffd700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fddbc0a1230 0x7fddbc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.896+0000 7fddd5165700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fddbc0a1230 0x7fddbc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.896+0000 7fddc6ffd700 1 -- 172.21.15.39:0/233751649 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fddcc0ce360 con 0x7fddd0100f80 2024-05-24T11:37:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.896+0000 7fddd5165700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fddbc0a1230 0x7fddbc0a36f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fddc0005950 tx=0x7fddc00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:55.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:55.899+0000 7fddc6ffd700 1 -- 172.21.15.39:0/233751649 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fddcc097aa0 con 0x7fddd0100f80 2024-05-24T11:37:56.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:55 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1857063986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:56.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:55 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1111599581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:56.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.110+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fddd010c7a0 con 0x7fddd0100f80 2024-05-24T11:37:56.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.110+0000 7fddc6ffd700 1 -- 172.21.15.39:0/233751649 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7fddcc09b0c0 con 0x7fddd0100f80 2024-05-24T11:37:56.116 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-05-24T11:37:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fddbc0a1230 msgr2=0x7fddbc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fddbc0a1230 0x7fddbc0a36f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fddc0005950 tx=0x7fddc00058e0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 msgr2=0x7fddd0196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 0x7fddd0196f00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009f50 tx=0x7fddcc005b00 comp rx=0 tx=0).stop 2024-05-24T11:37:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 shutdown_connections 2024-05-24T11:37:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fddbc0a1230 0x7fddbc0a36f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddd0102b10 0x7fddd019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddd0100f80 0x7fddd0196f00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 --2- 172.21.15.39:0/233751649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddd0102170 0x7fddd0197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 >> 172.21.15.39:0/233751649 conn(0x7fddd00fc4f0 msgr2=0x7fddd0105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:56.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 shutdown_connections 2024-05-24T11:37:56.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.113+0000 7fddd7bca700 1 -- 172.21.15.39:0/233751649 wait complete. 2024-05-24T11:37:56.131 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-24T11:37:56.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:55 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1857063986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:56.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:55 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1111599581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:56.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:55 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1857063986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:56.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:55 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1111599581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:56.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 -- 172.21.15.39:0/2579887763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c8100fb0 msgr2=0x7ff5c8101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/2579887763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c8100fb0 0x7ff5c8101430 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009990 tx=0x7ff5b0009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 -- 172.21.15.39:0/2579887763 shutdown_connections 2024-05-24T11:37:56.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/2579887763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c8101970 0x7ff5c8108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/2579887763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c80ffe60 0x7ff5c8100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/2579887763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c8100fb0 0x7ff5c8101430 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 -- 172.21.15.39:0/2579887763 >> 172.21.15.39:0/2579887763 conn(0x7ff5c80fb3f0 msgr2=0x7ff5c80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:56.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 -- 172.21.15.39:0/2579887763 shutdown_connections 2024-05-24T11:37:56.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 -- 172.21.15.39:0/2579887763 wait complete. 2024-05-24T11:37:56.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.255+0000 7ff5cddc6700 1 Processor -- start 2024-05-24T11:37:56.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5cddc6700 1 -- start start 2024-05-24T11:37:56.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5cddc6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c80ffe60 0x7ff5c8068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5cddc6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c8100fb0 0x7ff5c819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5cddc6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c8101970 0x7ff5c819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5cddc6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c819e8a0 con 0x7ff5c80ffe60 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5cddc6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c819ea10 con 0x7ff5c8100fb0 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5cddc6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c819ebd0 con 0x7ff5c8101970 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c80ffe60 0x7ff5c8068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c80ffe60 0x7ff5c8068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42454/0 (socket says 172.21.15.39:42454) 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c77fe700 1 -- 172.21.15.39:0/3887297616 learned_addr learned my addr 172.21.15.39:0/3887297616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c6ffd700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c8100fb0 0x7ff5c819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c7fff700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c8101970 0x7ff5c819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c77fe700 1 -- 172.21.15.39:0/3887297616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c8101970 msgr2=0x7ff5c819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c77fe700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c8101970 0x7ff5c819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c77fe700 1 -- 172.21.15.39:0/3887297616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c8100fb0 msgr2=0x7ff5c819bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c77fe700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c8100fb0 0x7ff5c819bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c77fe700 1 -- 172.21.15.39:0/3887297616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00095f0 con 0x7ff5c80ffe60 2024-05-24T11:37:56.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c77fe700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c80ffe60 0x7ff5c8068b10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800d950 tx=0x7ff5b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:56.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.256+0000 7ff5c4ff9700 1 -- 172.21.15.39:0/3887297616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8009940 con 0x7ff5c80ffe60 2024-05-24T11:37:56.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.257+0000 7ff5c4ff9700 1 -- 172.21.15.39:0/3887297616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5b8010460 con 0x7ff5c80ffe60 2024-05-24T11:37:56.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.257+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c819eeb0 con 0x7ff5c80ffe60 2024-05-24T11:37:56.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.257+0000 7ff5c4ff9700 1 -- 172.21.15.39:0/3887297616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800f620 con 0x7ff5c80ffe60 2024-05-24T11:37:56.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.257+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c819f2d0 con 0x7ff5c80ffe60 2024-05-24T11:37:56.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.257+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c8107320 con 0x7ff5c80ffe60 2024-05-24T11:37:56.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.259+0000 7ff5c4ff9700 1 -- 172.21.15.39:0/3887297616 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff5b80105d0 con 0x7ff5c80ffe60 2024-05-24T11:37:56.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.259+0000 7ff5c4ff9700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff5b40a11b0 0x7ff5b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:56.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.259+0000 7ff5c6ffd700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff5b40a11b0 0x7ff5b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:56.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.259+0000 7ff5c4ff9700 1 -- 172.21.15.39:0/3887297616 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff5b80cc7d0 con 0x7ff5c80ffe60 2024-05-24T11:37:56.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.260+0000 7ff5c6ffd700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff5b40a11b0 0x7ff5b40a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80690d0 tx=0x7ff5b000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:56.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.263+0000 7ff5c4ff9700 1 -- 172.21.15.39:0/3887297616 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff5b8095f00 con 0x7ff5c80ffe60 2024-05-24T11:37:56.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.475+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5c810b710 con 0x7ff5c80ffe60 2024-05-24T11:37:56.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.475+0000 7ff5c4ff9700 1 -- 172.21.15.39:0/3887297616 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5b80160a0 con 0x7ff5c80ffe60 2024-05-24T11:37:56.481 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-05-24T11:37:56.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff5b40a11b0 msgr2=0x7ff5b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff5b40a11b0 0x7ff5b40a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80690d0 tx=0x7ff5b000b540 comp rx=0 tx=0).stop 2024-05-24T11:37:56.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c80ffe60 msgr2=0x7ff5c8068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c80ffe60 0x7ff5c8068b10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800d950 tx=0x7ff5b800dc60 comp rx=0 tx=0).stop 2024-05-24T11:37:56.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 shutdown_connections 2024-05-24T11:37:56.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff5b40a11b0 0x7ff5b40a3670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c8101970 0x7ff5c819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c80ffe60 0x7ff5c8068b10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 --2- 172.21.15.39:0/3887297616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5c8100fb0 0x7ff5c819bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 >> 172.21.15.39:0/3887297616 conn(0x7ff5c80fb3f0 msgr2=0x7ff5c8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:56.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 shutdown_connections 2024-05-24T11:37:56.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.477+0000 7ff5cddc6700 1 -- 172.21.15.39:0/3887297616 wait complete. 2024-05-24T11:37:56.496 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:37:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.617+0000 7f74188b9700 1 -- 172.21.15.39:0/3669534747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414106ba0 msgr2=0x7f7414108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.617+0000 7f74188b9700 1 --2- 172.21.15.39:0/3669534747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414106ba0 0x7f7414108fb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f740c00b0a0 tx=0x7f740c00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.617+0000 7f74188b9700 1 -- 172.21.15.39:0/3669534747 shutdown_connections 2024-05-24T11:37:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.617+0000 7f74188b9700 1 --2- 172.21.15.39:0/3669534747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414106ba0 0x7f7414108fb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.617+0000 7f74188b9700 1 --2- 172.21.15.39:0/3669534747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7414101940 0x7f7414103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.617+0000 7f74188b9700 1 --2- 172.21.15.39:0/3669534747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7414104270 0x7f7414106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.617+0000 7f74188b9700 1 -- 172.21.15.39:0/3669534747 >> 172.21.15.39:0/3669534747 conn(0x7f74140fb2f0 msgr2=0x7f74140fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:56.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.617+0000 7f74188b9700 1 -- 172.21.15.39:0/3669534747 shutdown_connections 2024-05-24T11:37:56.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.617+0000 7f74188b9700 1 -- 172.21.15.39:0/3669534747 wait complete. 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f74188b9700 1 Processor -- start 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f74188b9700 1 -- start start 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f74188b9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7414101940 0x7f7414101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f74188b9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414104270 0x7f74140ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f74188b9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7414106ba0 0x7f74140ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f74188b9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7414100530 con 0x7f7414104270 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f74188b9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74141006a0 con 0x7f7414106ba0 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f74188b9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74141a0e70 con 0x7f7414101940 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f7411d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414104270 0x7f74140ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f7411d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414104270 0x7f74140ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42472/0 (socket says 172.21.15.39:42472) 2024-05-24T11:37:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f7411d9b700 1 -- 172.21.15.39:0/3223686917 learned_addr learned my addr 172.21.15.39:0/3223686917 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f7412d9d700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7414106ba0 0x7f74140ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f741259c700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7414101940 0x7f7414101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f7411d9b700 1 -- 172.21.15.39:0/3223686917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7414101940 msgr2=0x7f7414101210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f7411d9b700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7414101940 0x7f7414101210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f7411d9b700 1 -- 172.21.15.39:0/3223686917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7414106ba0 msgr2=0x7f74140ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f7411d9b700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7414106ba0 0x7f74140ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.618+0000 7f7411d9b700 1 -- 172.21.15.39:0/3223686917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c009d00 con 0x7f7414104270 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.619+0000 7f7411d9b700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414104270 0x7f74140ff8f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f740800c920 tx=0x7f740800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.619+0000 7f74037fe700 1 -- 172.21.15.39:0/3223686917 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408004a30 con 0x7f7414104270 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.619+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74141a10d0 con 0x7f7414104270 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.619+0000 7f74037fe700 1 -- 172.21.15.39:0/3223686917 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74080054b0 con 0x7f7414104270 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.619+0000 7f74037fe700 1 -- 172.21.15.39:0/3223686917 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408007ab0 con 0x7f7414104270 2024-05-24T11:37:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.619+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74141a1630 con 0x7f7414104270 2024-05-24T11:37:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.620+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74140fce70 con 0x7f7414104270 2024-05-24T11:37:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.621+0000 7f74037fe700 1 -- 172.21.15.39:0/3223686917 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7408007c50 con 0x7f7414104270 2024-05-24T11:37:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.621+0000 7f74037fe700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73fc0a11b0 0x7f73fc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.622+0000 7f741259c700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73fc0a11b0 0x7f73fc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.622+0000 7f74037fe700 1 -- 172.21.15.39:0/3223686917 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f74080cc910 con 0x7f7414104270 2024-05-24T11:37:56.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.622+0000 7f741259c700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73fc0a11b0 0x7f73fc0a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7404009c60 tx=0x7f7404009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:56.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.625+0000 7f74037fe700 1 -- 172.21.15.39:0/3223686917 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7408096040 con 0x7f7414104270 2024-05-24T11:37:56.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.881+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f741410c4a0 con 0x7f7414104270 2024-05-24T11:37:56.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.882+0000 7f74037fe700 1 -- 172.21.15.39:0/3223686917 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10906 (secure 0 0 0) 0x7f740801d020 con 0x7f7414104270 2024-05-24T11:37:56.888 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:56.888 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-24T11:36:48.500682+0000","modified":"2024-05-24T11:36:55.584150+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24511,"mds_2":24506,"mds_3":24490,"mds_4":14631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/3006971240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3006971240},{"type":"v1","addr":"172.21.15.39:6839","nonce":3006971240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/3024712815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3024712815},{"type":"v1","addr":"172.21.15.39:6841","nonce":3024712815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/218688120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":218688120},{"type":"v1","addr":"172.21.15.137:6839","nonce":218688120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3196235839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3196235839},{"type":"v1","addr":"172.21.15.53:6839","nonce":3196235839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-24T11:37:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73fc0a11b0 msgr2=0x7f73fc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73fc0a11b0 0x7f73fc0a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7404009c60 tx=0x7f7404009260 comp rx=0 tx=0).stop 2024-05-24T11:37:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414104270 msgr2=0x7f74140ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414104270 0x7f74140ff8f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f740800c920 tx=0x7f740800cc30 comp rx=0 tx=0).stop 2024-05-24T11:37:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 shutdown_connections 2024-05-24T11:37:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f73fc0a11b0 0x7f73fc0a3670 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7414101940 0x7f7414101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7414106ba0 0x7f74140ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 --2- 172.21.15.39:0/3223686917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7414104270 0x7f74140ff8f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 >> 172.21.15.39:0/3223686917 conn(0x7f74140fb2f0 msgr2=0x7f7414102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 shutdown_connections 2024-05-24T11:37:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:56.884+0000 7f74188b9700 1 -- 172.21.15.39:0/3223686917 wait complete. 2024-05-24T11:37:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-24T11:37:56.903 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.024+0000 7fbe54a88700 1 -- 172.21.15.39:0/2157967556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 msgr2=0x7fbe501013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.024+0000 7fbe54a88700 1 --2- 172.21.15.39:0/2157967556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 0x7fbe501013b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbe380099c0 tx=0x7fbe38009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.024+0000 7fbe54a88700 1 -- 172.21.15.39:0/2157967556 shutdown_connections 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.024+0000 7fbe54a88700 1 --2- 172.21.15.39:0/2157967556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe501018f0 0x7fbe501081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.024+0000 7fbe54a88700 1 --2- 172.21.15.39:0/2157967556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe500ffd60 0x7fbe50100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.024+0000 7fbe54a88700 1 --2- 172.21.15.39:0/2157967556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 0x7fbe501013b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.024+0000 7fbe54a88700 1 -- 172.21.15.39:0/2157967556 >> 172.21.15.39:0/2157967556 conn(0x7fbe500fb2f0 msgr2=0x7fbe500fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.024+0000 7fbe54a88700 1 -- 172.21.15.39:0/2157967556 shutdown_connections 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.024+0000 7fbe54a88700 1 -- 172.21.15.39:0/2157967556 wait complete. 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe54a88700 1 Processor -- start 2024-05-24T11:37:57.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe54a88700 1 -- start start 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe54a88700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe500ffd60 0x7fbe50103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe54a88700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 0x7fbe50102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe54a88700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe501018f0 0x7fbe50102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe54a88700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50103010 con 0x7fbe50100f50 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe54a88700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50103180 con 0x7fbe501018f0 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe54a88700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe501a0e10 con 0x7fbe500ffd60 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe4dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 0x7fbe50102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe4dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 0x7fbe50102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42496/0 (socket says 172.21.15.39:42496) 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.025+0000 7fbe4dd9b700 1 -- 172.21.15.39:0/1989853560 learned_addr learned my addr 172.21.15.39:0/1989853560 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe4ed9d700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe501018f0 0x7fbe50102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe4e59c700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe500ffd60 0x7fbe50103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe4dd9b700 1 -- 172.21.15.39:0/1989853560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe500ffd60 msgr2=0x7fbe50103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe4dd9b700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe500ffd60 0x7fbe50103ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe4dd9b700 1 -- 172.21.15.39:0/1989853560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe501018f0 msgr2=0x7fbe50102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe4dd9b700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe501018f0 0x7fbe50102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe4dd9b700 1 -- 172.21.15.39:0/1989853560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380095f0 con 0x7fbe50100f50 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe4ed9d700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe501018f0 0x7fbe50102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe4dd9b700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 0x7fbe50102330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbe38000c00 tx=0x7fbe38005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe477fe700 1 -- 172.21.15.39:0/1989853560 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801d070 con 0x7fbe50100f50 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe477fe700 1 -- 172.21.15.39:0/1989853560 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe3800bd30 con 0x7fbe50100f50 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe501a1040 con 0x7fbe50100f50 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe477fe700 1 -- 172.21.15.39:0/1989853560 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38022d60 con 0x7fbe50100f50 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.026+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe501a1490 con 0x7fbe50100f50 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.027+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe50067c10 con 0x7fbe50100f50 2024-05-24T11:37:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.028+0000 7fbe477fe700 1 -- 172.21.15.39:0/1989853560 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbe38004000 con 0x7fbe50100f50 2024-05-24T11:37:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.029+0000 7fbe477fe700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbe3c0a11b0 0x7fbe3c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.029+0000 7fbe477fe700 1 -- 172.21.15.39:0/1989853560 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbe380ce200 con 0x7fbe50100f50 2024-05-24T11:37:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.029+0000 7fbe4e59c700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbe3c0a11b0 0x7fbe3c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.029+0000 7fbe4e59c700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbe3c0a11b0 0x7fbe3c0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006fd0 tx=0x7fbe40008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.032+0000 7fbe477fe700 1 -- 172.21.15.39:0/1989853560 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbe380979c0 con 0x7fbe50100f50 2024-05-24T11:37:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:56 smithi053 ceph-mon[47448]: pgmap v256: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:56 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/233751649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:56 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3887297616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:57.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:56 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3223686917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:57.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:56 smithi137 ceph-mon[47907]: pgmap v256: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:57.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:56 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/233751649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:57.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:56 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3887297616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:57.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:56 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3223686917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.241+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe5010b4c0 con 0x7fbe50100f50 2024-05-24T11:37:57.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.243+0000 7fbe477fe700 1 -- 172.21.15.39:0/1989853560 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22344 (secure 0 0 0) 0x7fbe3809afe0 con 0x7fbe50100f50 2024-05-24T11:37:57.249 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:57.249 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:37:43.820939+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:37:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbe3c0a11b0 msgr2=0x7fbe3c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbe3c0a11b0 0x7fbe3c0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006fd0 tx=0x7fbe40008040 comp rx=0 tx=0).stop 2024-05-24T11:37:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 msgr2=0x7fbe50102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 0x7fbe50102330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbe38000c00 tx=0x7fbe38005b00 comp rx=0 tx=0).stop 2024-05-24T11:37:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 shutdown_connections 2024-05-24T11:37:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbe3c0a11b0 0x7fbe3c0a3670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe500ffd60 0x7fbe50103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe501018f0 0x7fbe50102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 --2- 172.21.15.39:0/1989853560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe50100f50 0x7fbe50102330 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 >> 172.21.15.39:0/1989853560 conn(0x7fbe500fb2f0 msgr2=0x7fbe50068060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:57.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 shutdown_connections 2024-05-24T11:37:57.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.245+0000 7fbe54a88700 1 -- 172.21.15.39:0/1989853560 wait complete. 2024-05-24T11:37:57.262 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:56 smithi039 ceph-mon[42919]: pgmap v256: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-24T11:37:57.263 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:56 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/233751649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:57.263 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:56 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3887297616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:57.263 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:56 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3223686917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:57.264 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-24T11:36:46.509049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-24T11:36:47.866055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:37:57.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-24T11:37:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.388+0000 7fef8e1a6700 1 -- 172.21.15.39:0/2349217437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88100fb0 msgr2=0x7fef88101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.388+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/2349217437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88100fb0 0x7fef88101430 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fef70009990 tx=0x7fef70009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.388+0000 7fef8e1a6700 1 -- 172.21.15.39:0/2349217437 shutdown_connections 2024-05-24T11:37:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.388+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/2349217437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef88101970 0x7fef88108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.388+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/2349217437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef880ffe60 0x7fef88100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.388+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/2349217437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88100fb0 0x7fef88101430 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.388+0000 7fef8e1a6700 1 -- 172.21.15.39:0/2349217437 >> 172.21.15.39:0/2349217437 conn(0x7fef880fb3f0 msgr2=0x7fef880fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.388+0000 7fef8e1a6700 1 -- 172.21.15.39:0/2349217437 shutdown_connections 2024-05-24T11:37:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.388+0000 7fef8e1a6700 1 -- 172.21.15.39:0/2349217437 wait complete. 2024-05-24T11:37:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef8e1a6700 1 Processor -- start 2024-05-24T11:37:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef8e1a6700 1 -- start start 2024-05-24T11:37:57.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef8e1a6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef880ffe60 0x7fef88068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef8e1a6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef88100fb0 0x7fef8819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef8e1a6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88101970 0x7fef8819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef8e1a6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8819e8f0 con 0x7fef88101970 2024-05-24T11:37:57.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef8e1a6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8819ea60 con 0x7fef88100fb0 2024-05-24T11:37:57.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef8e1a6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8819ec20 con 0x7fef880ffe60 2024-05-24T11:37:57.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef87fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88101970 0x7fef8819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef87fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88101970 0x7fef8819e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45754/0 (socket says 172.21.15.39:45754) 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef87fff700 1 -- 172.21.15.39:0/1888420958 learned_addr learned my addr 172.21.15.39:0/1888420958 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef86ffd700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef88100fb0 0x7fef8819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef877fe700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef880ffe60 0x7fef88068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.389+0000 7fef87fff700 1 -- 172.21.15.39:0/1888420958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef880ffe60 msgr2=0x7fef88068b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef87fff700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef880ffe60 0x7fef88068b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef87fff700 1 -- 172.21.15.39:0/1888420958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef88100fb0 msgr2=0x7fef8819bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef87fff700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef88100fb0 0x7fef8819bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef87fff700 1 -- 172.21.15.39:0/1888420958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700095f0 con 0x7fef88101970 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef87fff700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88101970 0x7fef8819e150 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fef80007ee0 tx=0x7fef8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef84ff9700 1 -- 172.21.15.39:0/1888420958 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80017070 con 0x7fef88101970 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef84ff9700 1 -- 172.21.15.39:0/1888420958 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef8000f040 con 0x7fef88101970 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef84ff9700 1 -- 172.21.15.39:0/1888420958 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80005710 con 0x7fef88101970 2024-05-24T11:37:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8819ef00 con 0x7fef88101970 2024-05-24T11:37:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.390+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8819f330 con 0x7fef88101970 2024-05-24T11:37:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.391+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef88107320 con 0x7fef88101970 2024-05-24T11:37:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.392+0000 7fef84ff9700 1 -- 172.21.15.39:0/1888420958 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fef80005870 con 0x7fef88101970 2024-05-24T11:37:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.392+0000 7fef84ff9700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fef740a11b0 0x7fef740a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.393+0000 7fef877fe700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fef740a11b0 0x7fef740a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.393+0000 7fef84ff9700 1 -- 172.21.15.39:0/1888420958 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fef800cc150 con 0x7fef88101970 2024-05-24T11:37:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.393+0000 7fef877fe700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fef740a11b0 0x7fef740a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef78005950 tx=0x7fef780058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:57.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.397+0000 7fef84ff9700 1 -- 172.21.15.39:0/1888420958 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fef80095880 con 0x7fef88101970 2024-05-24T11:37:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.606+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef8810b560 con 0x7fef88101970 2024-05-24T11:37:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.606+0000 7fef84ff9700 1 -- 172.21.15.39:0/1888420958 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef80098ea0 con 0x7fef88101970 2024-05-24T11:37:57.612 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fef740a11b0 msgr2=0x7fef740a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fef740a11b0 0x7fef740a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef78005950 tx=0x7fef780058e0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88101970 msgr2=0x7fef8819e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88101970 0x7fef8819e150 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fef80007ee0 tx=0x7fef8000d3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 shutdown_connections 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fef740a11b0 0x7fef740a3670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef880ffe60 0x7fef88068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef88101970 0x7fef8819e150 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 --2- 172.21.15.39:0/1888420958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef88100fb0 0x7fef8819bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 >> 172.21.15.39:0/1888420958 conn(0x7fef880fb3f0 msgr2=0x7fef88104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:57.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 shutdown_connections 2024-05-24T11:37:57.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.609+0000 7fef8e1a6700 1 -- 172.21.15.39:0/1888420958 wait complete. 2024-05-24T11:37:57.628 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-24T11:37:57.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.751+0000 7fbaf01ae700 1 -- 172.21.15.39:0/2223896311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102170 msgr2=0x7fbae81025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.751+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/2223896311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102170 0x7fbae81025d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbad80099c0 tx=0x7fbad8009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 -- 172.21.15.39:0/2223896311 shutdown_connections 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/2223896311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae8102b10 0x7fbae81093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/2223896311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae8100f80 0x7fbae81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/2223896311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102170 0x7fbae81025d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 -- 172.21.15.39:0/2223896311 >> 172.21.15.39:0/2223896311 conn(0x7fbae80fc4f0 msgr2=0x7fbae80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 -- 172.21.15.39:0/2223896311 shutdown_connections 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 -- 172.21.15.39:0/2223896311 wait complete. 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 Processor -- start 2024-05-24T11:37:57.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 -- start start 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae8100f80 0x7fbae8196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae8102170 0x7fbae8197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102b10 0x7fbae819e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae819e980 con 0x7fbae8102b10 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae819eac0 con 0x7fbae8102170 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.752+0000 7fbaf01ae700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae819ec50 con 0x7fbae8100f80 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaee74b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102b10 0x7fbae819e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaee74b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102b10 0x7fbae819e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45772/0 (socket says 172.21.15.39:45772) 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaee74b700 1 -- 172.21.15.39:0/3945034505 learned_addr learned my addr 172.21.15.39:0/3945034505 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaedf4a700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae8100f80 0x7fbae8196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaed749700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae8102170 0x7fbae8197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaee74b700 1 -- 172.21.15.39:0/3945034505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae8100f80 msgr2=0x7fbae8196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaee74b700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae8100f80 0x7fbae8196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaee74b700 1 -- 172.21.15.39:0/3945034505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae8102170 msgr2=0x7fbae8197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaee74b700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae8102170 0x7fbae8197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaee74b700 1 -- 172.21.15.39:0/3945034505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80095f0 con 0x7fbae8102b10 2024-05-24T11:37:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaee74b700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102b10 0x7fbae819e3b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbae000ada0 tx=0x7fbae000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbadeffd700 1 -- 172.21.15.39:0/3945034505 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae000f070 con 0x7fbae8102b10 2024-05-24T11:37:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae819ef30 con 0x7fbae8102b10 2024-05-24T11:37:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbadeffd700 1 -- 172.21.15.39:0/3945034505 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbae0007b30 con 0x7fbae8102b10 2024-05-24T11:37:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae819f400 con 0x7fbae8102b10 2024-05-24T11:37:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.753+0000 7fbadeffd700 1 -- 172.21.15.39:0/3945034505 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0005220 con 0x7fbae8102b10 2024-05-24T11:37:57.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.754+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae8108450 con 0x7fbae8102b10 2024-05-24T11:37:57.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.756+0000 7fbadeffd700 1 -- 172.21.15.39:0/3945034505 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbae000c8d0 con 0x7fbae8102b10 2024-05-24T11:37:57.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.756+0000 7fbadeffd700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbad40a11b0 0x7fbad40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:57.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.756+0000 7fbaedf4a700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbad40a11b0 0x7fbad40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:57.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.756+0000 7fbadeffd700 1 -- 172.21.15.39:0/3945034505 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbae00cbe30 con 0x7fbae8102b10 2024-05-24T11:37:57.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.757+0000 7fbaedf4a700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbad40a11b0 0x7fbad40a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbae8101fd0 tx=0x7fbae400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:57.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.761+0000 7fbadeffd700 1 -- 172.21.15.39:0/3945034505 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbae00955f0 con 0x7fbae8102b10 2024-05-24T11:37:57.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.969+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbae810c7a0 con 0x7fbae8102b10 2024-05-24T11:37:57.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.972+0000 7fbadeffd700 1 -- 172.21.15.39:0/3945034505 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbae0098c10 con 0x7fbae8102b10 2024-05-24T11:37:57.977 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-05-24T11:37:57.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbad40a11b0 msgr2=0x7fbad40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbad40a11b0 0x7fbad40a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbae8101fd0 tx=0x7fbae400b410 comp rx=0 tx=0).stop 2024-05-24T11:37:57.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102b10 msgr2=0x7fbae819e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:57.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102b10 0x7fbae819e3b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbae000ada0 tx=0x7fbae000c330 comp rx=0 tx=0).stop 2024-05-24T11:37:57.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 shutdown_connections 2024-05-24T11:37:57.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbad40a11b0 0x7fbad40a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae8100f80 0x7fbae8196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae8102b10 0x7fbae819e3b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 --2- 172.21.15.39:0/3945034505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbae8102170 0x7fbae8197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:57.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 >> 172.21.15.39:0/3945034505 conn(0x7fbae80fc4f0 msgr2=0x7fbae8105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:57.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 shutdown_connections 2024-05-24T11:37:57.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:57.974+0000 7fbaf01ae700 1 -- 172.21.15.39:0/3945034505 wait complete. 2024-05-24T11:37:57.993 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-24T11:37:58.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.116+0000 7fa2ce461700 1 -- 172.21.15.39:0/2042992446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8101970 msgr2=0x7fa2c8108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.116+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2042992446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8101970 0x7fa2c8108290 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000b0a0 tx=0x7fa2c000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.116+0000 7fa2ce461700 1 -- 172.21.15.39:0/2042992446 shutdown_connections 2024-05-24T11:37:58.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.116+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2042992446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c80ffe60 0x7fa2c8100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.116+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2042992446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8101970 0x7fa2c8108290 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.116+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2042992446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2c8100fb0 0x7fa2c8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.116+0000 7fa2ce461700 1 -- 172.21.15.39:0/2042992446 >> 172.21.15.39:0/2042992446 conn(0x7fa2c80fb3f0 msgr2=0x7fa2c80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:58.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.116+0000 7fa2ce461700 1 -- 172.21.15.39:0/2042992446 shutdown_connections 2024-05-24T11:37:58.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.116+0000 7fa2ce461700 1 -- 172.21.15.39:0/2042992446 wait complete. 2024-05-24T11:37:58.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.117+0000 7fa2ce461700 1 Processor -- start 2024-05-24T11:37:58.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.117+0000 7fa2ce461700 1 -- start start 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2ce461700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c80ffe60 0x7fa2c8196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2ce461700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2c8100fb0 0x7fa2c81973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2ce461700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c8101970 0x7fa2c819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2ce461700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c819e9e0 con 0x7fa2c80ffe60 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2ce461700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c819eb20 con 0x7fa2c8100fb0 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2ce461700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c819ecb0 con 0x7fa2c8101970 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c80ffe60 0x7fa2c8196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c80ffe60 0x7fa2c8196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45792/0 (socket says 172.21.15.39:45792) 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c7fff700 1 -- 172.21.15.39:0/2341166237 learned_addr learned my addr 172.21.15.39:0/2341166237 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2cc9fe700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c8101970 0x7fa2c819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c77fe700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2c8100fb0 0x7fa2c81973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c7fff700 1 -- 172.21.15.39:0/2341166237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c8101970 msgr2=0x7fa2c819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c7fff700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c8101970 0x7fa2c819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c7fff700 1 -- 172.21.15.39:0/2341166237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2c8100fb0 msgr2=0x7fa2c81973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c7fff700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2c8100fb0 0x7fa2c81973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c7fff700 1 -- 172.21.15.39:0/2341166237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c0009d00 con 0x7fa2c80ffe60 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c77fe700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2c8100fb0 0x7fa2c81973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.118+0000 7fa2c7fff700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c80ffe60 0x7fa2c8196eb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c920 tx=0x7fa2b800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.119+0000 7fa2c57fa700 1 -- 172.21.15.39:0/2341166237 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8007b00 con 0x7fa2c80ffe60 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.119+0000 7fa2c57fa700 1 -- 172.21.15.39:0/2341166237 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2b800ce70 con 0x7fa2c80ffe60 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.119+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c819ef90 con 0x7fa2c80ffe60 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.119+0000 7fa2c57fa700 1 -- 172.21.15.39:0/2341166237 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b80053e0 con 0x7fa2c80ffe60 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.119+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c819f3b0 con 0x7fa2c80ffe60 2024-05-24T11:37:58.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.120+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c8067cf0 con 0x7fa2c80ffe60 2024-05-24T11:37:58.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.121+0000 7fa2c57fa700 1 -- 172.21.15.39:0/2341166237 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa2b8005580 con 0x7fa2c80ffe60 2024-05-24T11:37:58.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.121+0000 7fa2c57fa700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2b00a11b0 0x7fa2b00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.121+0000 7fa2c77fe700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2b00a11b0 0x7fa2b00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.121+0000 7fa2c57fa700 1 -- 172.21.15.39:0/2341166237 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa2b80cc560 con 0x7fa2c80ffe60 2024-05-24T11:37:58.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.122+0000 7fa2c77fe700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2b00a11b0 0x7fa2b00a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc005fd0 tx=0x7fa2bc005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:58.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.126+0000 7fa2c57fa700 1 -- 172.21.15.39:0/2341166237 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa2b8095c90 con 0x7fa2c80ffe60 2024-05-24T11:37:58.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:57 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1989853560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:58.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:57 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1888420958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:58.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:57 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1989853560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:58.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:57 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1888420958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:58.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:57 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1989853560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:37:58.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:57 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1888420958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:37:58.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.330+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2c819f770 con 0x7fa2c80ffe60 2024-05-24T11:37:58.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.330+0000 7fa2c57fa700 1 -- 172.21.15.39:0/2341166237 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2b80992b0 con 0x7fa2c80ffe60 2024-05-24T11:37:58.336 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-05-24T11:37:58.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.332+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2b00a11b0 msgr2=0x7fa2b00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.332+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2b00a11b0 0x7fa2b00a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc005fd0 tx=0x7fa2bc005e20 comp rx=0 tx=0).stop 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.332+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c80ffe60 msgr2=0x7fa2c8196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.332+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c80ffe60 0x7fa2c8196eb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c920 tx=0x7fa2b800cc30 comp rx=0 tx=0).stop 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.332+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 shutdown_connections 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.332+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fa2b00a11b0 0x7fa2b00a3670 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.333+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c8101970 0x7fa2c819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.333+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c80ffe60 0x7fa2c8196eb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.333+0000 7fa2ce461700 1 --2- 172.21.15.39:0/2341166237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2c8100fb0 0x7fa2c81973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.333+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 >> 172.21.15.39:0/2341166237 conn(0x7fa2c80fb3f0 msgr2=0x7fa2c80fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.333+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 shutdown_connections 2024-05-24T11:37:58.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.333+0000 7fa2ce461700 1 -- 172.21.15.39:0/2341166237 wait complete. 2024-05-24T11:37:58.351 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-24T11:37:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.471+0000 7f8112488700 1 -- 172.21.15.39:0/779262565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c100f80 msgr2=0x7f810c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.471+0000 7f8112488700 1 --2- 172.21.15.39:0/779262565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c100f80 0x7f810c1013e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f81000099c0 tx=0x7f8100009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.471+0000 7f8112488700 1 -- 172.21.15.39:0/779262565 shutdown_connections 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.471+0000 7f8112488700 1 --2- 172.21.15.39:0/779262565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f810c101920 0x7f810c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.471+0000 7f8112488700 1 --2- 172.21.15.39:0/779262565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f810c0ffd90 0x7f810c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.471+0000 7f8112488700 1 --2- 172.21.15.39:0/779262565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c100f80 0x7f810c1013e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.471+0000 7f8112488700 1 -- 172.21.15.39:0/779262565 >> 172.21.15.39:0/779262565 conn(0x7f810c0fb300 msgr2=0x7f810c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 -- 172.21.15.39:0/779262565 shutdown_connections 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 -- 172.21.15.39:0/779262565 wait complete. 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 Processor -- start 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 -- start start 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c0ffd90 0x7f810c103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f810c100f80 0x7f810c102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f810c101920 0x7f810c1028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c102fa0 con 0x7f810c0ffd90 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c103110 con 0x7f810c101920 2024-05-24T11:37:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.472+0000 7f8112488700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c1a0dd0 con 0x7f810c100f80 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c0ffd90 0x7f810c103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c0ffd90 0x7f810c103c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45808/0 (socket says 172.21.15.39:45808) 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810bfff700 1 -- 172.21.15.39:0/1457032954 learned_addr learned my addr 172.21.15.39:0/1457032954 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f8110a25700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f810c101920 0x7f810c1028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810b7fe700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f810c100f80 0x7f810c102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810bfff700 1 -- 172.21.15.39:0/1457032954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f810c100f80 msgr2=0x7f810c102360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810bfff700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f810c100f80 0x7f810c102360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810bfff700 1 -- 172.21.15.39:0/1457032954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f810c101920 msgr2=0x7f810c1028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810bfff700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f810c101920 0x7f810c1028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810bfff700 1 -- 172.21.15.39:0/1457032954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81000095f0 con 0x7f810c0ffd90 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810b7fe700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f810c100f80 0x7f810c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f810bfff700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c0ffd90 0x7f810c103c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00d980 tx=0x7f80fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f81097fa700 1 -- 172.21.15.39:0/1457032954 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc004020 con 0x7f810c0ffd90 2024-05-24T11:37:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f81097fa700 1 -- 172.21.15.39:0/1457032954 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80fc009730 con 0x7f810c0ffd90 2024-05-24T11:37:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f81097fa700 1 -- 172.21.15.39:0/1457032954 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc003a00 con 0x7f810c0ffd90 2024-05-24T11:37:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810c1a0fd0 con 0x7f810c0ffd90 2024-05-24T11:37:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.473+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810c1a13f0 con 0x7f810c0ffd90 2024-05-24T11:37:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.474+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810c0673f0 con 0x7f810c0ffd90 2024-05-24T11:37:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.476+0000 7f81097fa700 1 -- 172.21.15.39:0/1457032954 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f80fc0098a0 con 0x7f810c0ffd90 2024-05-24T11:37:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.476+0000 7f81097fa700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f80f40a11b0 0x7f80f40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.476+0000 7f81097fa700 1 -- 172.21.15.39:0/1457032954 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f80fc0cc810 con 0x7f810c0ffd90 2024-05-24T11:37:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.476+0000 7f810b7fe700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f80f40a11b0 0x7f80f40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.477+0000 7f810b7fe700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f80f40a11b0 0x7f80f40a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f810000c010 tx=0x7f8100005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:58.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.480+0000 7f81097fa700 1 -- 172.21.15.39:0/1457032954 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f80fc095f40 con 0x7f810c0ffd90 2024-05-24T11:37:58.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.735+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f810c10b6e0 con 0x7f810c0ffd90 2024-05-24T11:37:58.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.736+0000 7f81097fa700 1 -- 172.21.15.39:0/1457032954 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10906 (secure 0 0 0) 0x7f80fc099560 con 0x7f810c0ffd90 2024-05-24T11:37:58.742 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:58.742 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2983171503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2983171503},{"type":"v1","addr":"172.21.15.53:6833","nonce":2983171503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1458882734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1458882734},{"type":"v1","addr":"172.21.15.39:6835","nonce":1458882734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-24T11:36:48.500682+0000","modified":"2024-05-24T11:36:55.584150+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24511,"mds_2":24506,"mds_3":24490,"mds_4":14631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/3631576511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3631576511},{"type":"v1","addr":"172.21.15.39:6837","nonce":3631576511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6839/3006971240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":3006971240},{"type":"v1","addr":"172.21.15.39:6839","nonce":3006971240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/3024712815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3024712815},{"type":"v1","addr":"172.21.15.39:6841","nonce":3024712815}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/451866213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":451866213},{"type":"v1","addr":"172.21.15.137:6833","nonce":451866213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/3629179598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3629179598},{"type":"v1","addr":"172.21.15.53:6835","nonce":3629179598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1490077205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1490077205},{"type":"v1","addr":"172.21.15.137:6835","nonce":1490077205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/967972436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":967972436},{"type":"v1","addr":"172.21.15.137:6837","nonce":967972436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.53:6837/3764768463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3764768463},{"type":"v1","addr":"172.21.15.53:6837","nonce":3764768463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24511":{"gid":24511,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/218688120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":218688120},{"type":"v1","addr":"172.21.15.137:6839","nonce":218688120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6839/3196235839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":3196235839},{"type":"v1","addr":"172.21.15.53:6839","nonce":3196235839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-24T11:37:58.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f80f40a11b0 msgr2=0x7f80f40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f80f40a11b0 0x7f80f40a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f810000c010 tx=0x7f8100005fd0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c0ffd90 msgr2=0x7f810c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c0ffd90 0x7f810c103c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00d980 tx=0x7f80fc00dc90 comp rx=0 tx=0).stop 2024-05-24T11:37:58.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 shutdown_connections 2024-05-24T11:37:58.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f80f40a11b0 0x7f80f40a3670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f810c101920 0x7f810c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f810c0ffd90 0x7f810c103c80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 --2- 172.21.15.39:0/1457032954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f810c100f80 0x7f810c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 >> 172.21.15.39:0/1457032954 conn(0x7f810c0fb300 msgr2=0x7f810c069240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:58.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 shutdown_connections 2024-05-24T11:37:58.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.738+0000 7f8112488700 1 -- 172.21.15.39:0/1457032954 wait complete. 2024-05-24T11:37:58.745 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-24T11:37:58.757 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:37:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 -- 172.21.15.39:0/89329408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd81029a0 msgr2=0x7f6cd81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 --2- 172.21.15.39:0/89329408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd81029a0 0x7f6cd81092c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000b0a0 tx=0x7f6cd000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 -- 172.21.15.39:0/89329408 shutdown_connections 2024-05-24T11:37:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 --2- 172.21.15.39:0/89329408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cd8100e90 0x7f6cd81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 --2- 172.21.15.39:0/89329408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd81029a0 0x7f6cd81092c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 --2- 172.21.15.39:0/89329408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8101fe0 0x7f6cd8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 -- 172.21.15.39:0/89329408 >> 172.21.15.39:0/89329408 conn(0x7f6cd80fc400 msgr2=0x7f6cd80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 -- 172.21.15.39:0/89329408 shutdown_connections 2024-05-24T11:37:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 -- 172.21.15.39:0/89329408 wait complete. 2024-05-24T11:37:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.878+0000 7f6cde596700 1 Processor -- start 2024-05-24T11:37:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cde596700 1 -- start start 2024-05-24T11:37:58.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cde596700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd8100e90 0x7f6cd8196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cde596700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8101fe0 0x7f6cd8197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cde596700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cd81029a0 0x7f6cd819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cde596700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd819e8e0 con 0x7f6cd8100e90 2024-05-24T11:37:58.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cde596700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd819ea50 con 0x7f6cd8101fe0 2024-05-24T11:37:58.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cde596700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd819ec10 con 0x7f6cd81029a0 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cd7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd8100e90 0x7f6cd8196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cd7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd8100e90 0x7f6cd8196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45828/0 (socket says 172.21.15.39:45828) 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cd7fff700 1 -- 172.21.15.39:0/3440528518 learned_addr learned my addr 172.21.15.39:0/3440528518 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cd77fe700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8101fe0 0x7f6cd8197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cdcb33700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cd81029a0 0x7f6cd819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cd7fff700 1 -- 172.21.15.39:0/3440528518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cd81029a0 msgr2=0x7f6cd819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cd7fff700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cd81029a0 0x7f6cd819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cd7fff700 1 -- 172.21.15.39:0/3440528518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8101fe0 msgr2=0x7f6cd8197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cd7fff700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8101fe0 0x7f6cd8197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cd7fff700 1 -- 172.21.15.39:0/3440528518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd0009d00 con 0x7f6cd8100e90 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.879+0000 7f6cdcb33700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cd81029a0 0x7f6cd819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.880+0000 7f6cd7fff700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd8100e90 0x7f6cd8196df0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800d950 tx=0x7f6cc800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.880+0000 7f6cd57fa700 1 -- 172.21.15.39:0/3440528518 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8009940 con 0x7f6cd8100e90 2024-05-24T11:37:58.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.880+0000 7f6cd57fa700 1 -- 172.21.15.39:0/3440528518 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6cc8010460 con 0x7f6cd8100e90 2024-05-24T11:37:58.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.880+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd819eef0 con 0x7f6cd8100e90 2024-05-24T11:37:58.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.880+0000 7f6cd57fa700 1 -- 172.21.15.39:0/3440528518 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800f620 con 0x7f6cd8100e90 2024-05-24T11:37:58.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.880+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd819f310 con 0x7f6cd8100e90 2024-05-24T11:37:58.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.881+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd8108280 con 0x7f6cd8100e90 2024-05-24T11:37:58.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.882+0000 7f6cd57fa700 1 -- 172.21.15.39:0/3440528518 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6cc80105d0 con 0x7f6cd8100e90 2024-05-24T11:37:58.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.883+0000 7f6cd57fa700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6cc00a11b0 0x7f6cc00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:58.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.883+0000 7f6cd77fe700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6cc00a11b0 0x7f6cc00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:58.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.883+0000 7f6cd57fa700 1 -- 172.21.15.39:0/3440528518 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6cc80cc7d0 con 0x7f6cd8100e90 2024-05-24T11:37:58.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.883+0000 7f6cd77fe700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6cc00a11b0 0x7f6cc00a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005950 tx=0x7f6ccc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:58.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:58.886+0000 7f6cd57fa700 1 -- 172.21.15.39:0/3440528518 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6cc8095f00 con 0x7f6cd8100e90 2024-05-24T11:37:59.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:58 smithi039 ceph-mon[42919]: pgmap v257: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-24T11:37:59.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:58 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3945034505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:59.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:58 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2341166237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:59.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:58 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1457032954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.099+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cd810c570 con 0x7f6cd8100e90 2024-05-24T11:37:59.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.100+0000 7f6cd57fa700 1 -- 172.21.15.39:0/3440528518 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+22344 (secure 0 0 0) 0x7f6cc8016090 con 0x7f6cd8100e90 2024-05-24T11:37:59.106 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:37:59.106 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:37:43.820939+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:37:59.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6cc00a11b0 msgr2=0x7f6cc00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6cc00a11b0 0x7f6cc00a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005950 tx=0x7f6ccc00b410 comp rx=0 tx=0).stop 2024-05-24T11:37:59.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd8100e90 msgr2=0x7f6cd8196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd8100e90 0x7f6cd8196df0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800d950 tx=0x7f6cc800dc60 comp rx=0 tx=0).stop 2024-05-24T11:37:59.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 shutdown_connections 2024-05-24T11:37:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f6cc00a11b0 0x7f6cc00a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cd81029a0 0x7f6cd819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cd8100e90 0x7f6cd8196df0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 --2- 172.21.15.39:0/3440528518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cd8101fe0 0x7f6cd8197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 >> 172.21.15.39:0/3440528518 conn(0x7f6cd80fc400 msgr2=0x7f6cd81053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 shutdown_connections 2024-05-24T11:37:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.102+0000 7f6cde596700 1 -- 172.21.15.39:0/3440528518 wait complete. 2024-05-24T11:37:59.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-24T11:36:48.500682+0000', 'modified': '2024-05-24T11:36:55.584150+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24542, 'mds_1': 24511, 'mds_2': 24506, 'mds_3': 24490, 'mds_4': 14631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6837/3631576511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 3631576511}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 3631576511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6839/3006971240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 3006971240}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 3006971240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6841/3024712815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 3024712815}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 3024712815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/451866213', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 451866213}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 451866213}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24476': {'gid': 24476, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/3629179598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 3629179598}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 3629179598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/1490077205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 1490077205}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 1490077205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.137:6837/967972436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 967972436}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 967972436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.53:6837/3764768463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 3764768463}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 3764768463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.137:6839/218688120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 218688120}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 218688120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6839/3196235839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 3196235839}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 3196235839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-24T11:37:59.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-24T11:37:59.122 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-24T11:37:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:58 smithi053 ceph-mon[47448]: pgmap v257: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-24T11:37:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:58 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3945034505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:58 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2341166237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:59.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:58 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1457032954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:59.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:58 smithi137 ceph-mon[47907]: pgmap v257: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-24T11:37:59.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:58 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3945034505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:37:59.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:58 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2341166237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:37:59.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:58 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1457032954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-24T11:37:59.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.243+0000 7fbefaaad700 1 -- 172.21.15.39:0/2127523564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef4100f20 msgr2=0x7fbef4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.243+0000 7fbefaaad700 1 --2- 172.21.15.39:0/2127523564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef4100f20 0x7fbef4101360 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009990 tx=0x7fbee8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 -- 172.21.15.39:0/2127523564 shutdown_connections 2024-05-24T11:37:59.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 --2- 172.21.15.39:0/2127523564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef4102ab0 0x7fbef41093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 --2- 172.21.15.39:0/2127523564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef4100f20 0x7fbef4101360 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 --2- 172.21.15.39:0/2127523564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbef41020f0 0x7fbef4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 -- 172.21.15.39:0/2127523564 >> 172.21.15.39:0/2127523564 conn(0x7fbef40fc4e0 msgr2=0x7fbef40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 -- 172.21.15.39:0/2127523564 shutdown_connections 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 -- 172.21.15.39:0/2127523564 wait complete. 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 Processor -- start 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 -- start start 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbef4100f20 0x7fbef4196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef41020f0 0x7fbef41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef4102ab0 0x7fbef419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef419e940 con 0x7fbef41020f0 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef419ea80 con 0x7fbef4100f20 2024-05-24T11:37:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.244+0000 7fbefaaad700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef419ec10 con 0x7fbef4102ab0 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef41020f0 0x7fbef41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef41020f0 0x7fbef41973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45848/0 (socket says 172.21.15.39:45848) 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef3fff700 1 -- 172.21.15.39:0/1593619379 learned_addr learned my addr 172.21.15.39:0/1593619379 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef904a700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef4102ab0 0x7fbef419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef8849700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbef4100f20 0x7fbef4196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef3fff700 1 -- 172.21.15.39:0/1593619379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef4102ab0 msgr2=0x7fbef419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef3fff700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef4102ab0 0x7fbef419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef3fff700 1 -- 172.21.15.39:0/1593619379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbef4100f20 msgr2=0x7fbef4196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef3fff700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbef4100f20 0x7fbef4196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef3fff700 1 -- 172.21.15.39:0/1593619379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee80095f0 con 0x7fbef41020f0 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef8849700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbef4100f20 0x7fbef4196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef3fff700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef41020f0 0x7fbef41973a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbee000e9a0 tx=0x7fbee000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef1ffb700 1 -- 172.21.15.39:0/1593619379 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0009790 con 0x7fbef41020f0 2024-05-24T11:37:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef1ffb700 1 -- 172.21.15.39:0/1593619379 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbee000cbd0 con 0x7fbef41020f0 2024-05-24T11:37:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbefaaad700 1 -- 172.21.15.39:0/1593619379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef419eef0 con 0x7fbef41020f0 2024-05-24T11:37:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.245+0000 7fbef1ffb700 1 -- 172.21.15.39:0/1593619379 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0018680 con 0x7fbef41020f0 2024-05-24T11:37:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.246+0000 7fbefaaad700 1 -- 172.21.15.39:0/1593619379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef419f370 con 0x7fbef41020f0 2024-05-24T11:37:59.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.248+0000 7fbef1ffb700 1 -- 172.21.15.39:0/1593619379 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbee000cd40 con 0x7fbef41020f0 2024-05-24T11:37:59.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.248+0000 7fbef1ffb700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbee40a11b0 0x7fbee40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.248+0000 7fbef8849700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbee40a11b0 0x7fbee40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.248+0000 7fbef1ffb700 1 -- 172.21.15.39:0/1593619379 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbee0014070 con 0x7fbef41020f0 2024-05-24T11:37:59.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbef8849700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbee40a11b0 0x7fbee40a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005910 tx=0x7fbee8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbf06c68b80 1 Processor -- start 2024-05-24T11:37:59.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbf06c68b80 1 -- start start 2024-05-24T11:37:59.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83a4f0 0x55b87f83a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f83b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83b810 0x55b87f83bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f791100 con 0x55b87f83a4f0 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f78eac0 con 0x55b87f83ae50 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f78a3b0 con 0x55b87f83b810 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbef8849700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83a4f0 0x55b87f83a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbef8849700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83a4f0 0x55b87f83a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45862/0 (socket says 172.21.15.39:45862) 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.249+0000 7fbef8849700 1 -- 172.21.15.39:0/1642381233 learned_addr learned my addr 172.21.15.39:0/1642381233 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbef3fff700 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f83b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbef8849700 1 -- 172.21.15.39:0/1642381233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83b810 msgr2=0x55b87f83bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbef904a700 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83b810 0x55b87f83bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbef8849700 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83b810 0x55b87f83bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbef8849700 1 -- 172.21.15.39:0/1642381233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 msgr2=0x55b87f83b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbef8849700 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f83b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbef8849700 1 -- 172.21.15.39:0/1642381233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87f60b360 con 0x55b87f83a4f0 2024-05-24T11:37:59.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbef3fff700 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f83b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:59.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbef8849700 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83a4f0 0x55b87f83a910 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbef4100c40 tx=0x7fbee801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1d252b7e745e64 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbedeffd700 1 -- 172.21.15.39:0/1642381233 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee802b070 con 0x55b87f83a4f0 2024-05-24T11:37:59.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbedeffd700 1 -- 172.21.15.39:0/1642381233 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbee801e040 con 0x55b87f83a4f0 2024-05-24T11:37:59.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.250+0000 7fbedeffd700 1 -- 172.21.15.39:0/1642381233 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee800bd70 con 0x55b87f83a4f0 2024-05-24T11:37:59.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 -- 172.21.15.39:0/1642381233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83a4f0 msgr2=0x55b87f83a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83a4f0 0x55b87f83a910 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbef4100c40 tx=0x7fbee801f000 comp rx=0 tx=0).stop 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 -- 172.21.15.39:0/1642381233 shutdown_connections 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83b810 0x55b87f83bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83a4f0 0x55b87f83a910 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/1642381233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f83b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 -- 172.21.15.39:0/1642381233 >> 172.21.15.39:0/1642381233 conn(0x55b87f7981c0 msgr2=0x55b87f834d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 -- 172.21.15.39:0/1642381233 shutdown_connections 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 -- 172.21.15.39:0/1642381233 wait complete. 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 -- start start 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.251+0000 7fbf06c68b80 1 Processor -- start 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83a4f0 0x55b87f843d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83ae50 0x55b87f844290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f726510 con 0x55b87f83b810 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f7261b0 con 0x55b87f83a4f0 2024-05-24T11:37:59.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f60d650 con 0x55b87f83ae50 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbf06c68b80 10 client.0 _ll_register_callbacks cb 0x55b87f72fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef904a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef904a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45868/0 (socket says 172.21.15.39:45868) 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef904a700 1 -- 172.21.15.39:0/369340810 learned_addr learned my addr 172.21.15.39:0/369340810 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef8849700 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83a4f0 0x55b87f843d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef3fff700 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83ae50 0x55b87f844290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef904a700 1 -- 172.21.15.39:0/369340810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83ae50 msgr2=0x55b87f844290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef904a700 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83ae50 0x55b87f844290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef904a700 1 -- 172.21.15.39:0/369340810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83a4f0 msgr2=0x55b87f843d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef904a700 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83a4f0 0x55b87f843d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef904a700 1 -- 172.21.15.39:0/369340810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef4109d00 con 0x55b87f83b810 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbef904a700 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84b2d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbeec010a40 tx=0x7fbeec010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9fe61f56fbfbe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbed3fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbed3fff700 1 -- 172.21.15.39:0/369340810 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00e850 con 0x55b87f83b810 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbed3fff700 1 -- 172.21.15.39:0/369340810 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbeec00e9b0 con 0x55b87f83b810 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.252+0000 7fbf06c68b80 1 -- 172.21.15.39:0/369340810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b87f6e3220 con 0x55b87f83b810 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbed3fff700 1 -- 172.21.15.39:0/369340810 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeec00eb70 con 0x55b87f83b810 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbf06c68b80 10 client.15255 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbf06c68b80 1 -- 172.21.15.39:0/369340810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b87f5b7b10 con 0x55b87f83b810 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbed3fff700 1 -- 172.21.15.39:0/369340810 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b87f6e3220 con 0x55b87f83b810 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbed3fff700 1 -- 172.21.15.39:0/369340810 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbeec021430 con 0x55b87f83b810 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbf06c68b80 10 client.15255 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbf06c68b80 10 client.15255 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbf06c68b80 20 client.15255 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x55b87f84e520 0x55b87f854340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbf06c68b80 4 client.15255 mds_command: new command op to 14631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbf06c68b80 1 -- 172.21.15.39:0/369340810 --> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b87f8097c0 con 0x55b87f84e520 2024-05-24T11:37:59.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.253+0000 7fbef8849700 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x55b87f84e520 0x55b87f854340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbed3fff700 10 client.15255 ms_handle_connect on v2:172.21.15.39:6838/3006971240 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbef8849700 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x55b87f84e520 0x55b87f854340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f4be6bd3303c32d2 server_cookie=84f0bbb69da6fa29 in_seq=0 out_seq=0 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbed3fff700 1 -- 172.21.15.39:0/369340810 <== mds.4 v2:172.21.15.39:6838/3006971240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b87f8097c0 con 0x55b87f84e520 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbed3fff700 10 client.15255 handle_command_reply: tid=0 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 client.15255 shutdown 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 -- 172.21.15.39:0/369340810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 msgr2=0x55b87f84b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84b2d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbeec010a40 tx=0x7fbeec010d50 comp rx=0 tx=0).stop 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 -- 172.21.15.39:0/369340810 shutdown_connections 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x55b87f84e520 0x55b87f854340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84b2d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83a4f0 0x55b87f843d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbed3fff700 0 client.15255 ms_handle_reset on v2:172.21.15.39:6838/3006971240 2024-05-24T11:37:59.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/369340810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83ae50 0x55b87f844290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 -- 172.21.15.39:0/369340810 >> 172.21.15.39:0/369340810 conn(0x55b87f7981c0 msgr2=0x55b87f834550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 -- 172.21.15.39:0/369340810 shutdown_connections 2024-05-24T11:37:59.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.254+0000 7fbf06c68b80 1 -- 172.21.15.39:0/369340810 wait complete. 2024-05-24T11:37:59.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.255+0000 7fbf06c68b80 20 client.15255 trim_cache size 0 max 16384 2024-05-24T11:37:59.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbf06c68b80 1 Processor -- start 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbf06c68b80 1 -- start start 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83a4f0 0x55b87f83a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f84b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f843260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f835b60 con 0x55b87f83b810 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f783150 con 0x55b87f83ae50 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f787010 con 0x55b87f83a4f0 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbef904a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f843260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbef904a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f843260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45872/0 (socket says 172.21.15.39:45872) 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbef904a700 1 -- 172.21.15.39:0/2638428278 learned_addr learned my addr 172.21.15.39:0/2638428278 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.273+0000 7fbef3fff700 1 --2- 172.21.15.39:0/2638428278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f84b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbef904a700 1 -- 172.21.15.39:0/2638428278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83a4f0 msgr2=0x55b87f83a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbef904a700 1 --2- 172.21.15.39:0/2638428278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83a4f0 0x55b87f83a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbef904a700 1 -- 172.21.15.39:0/2638428278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 msgr2=0x55b87f84b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbef904a700 1 --2- 172.21.15.39:0/2638428278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f84b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbef904a700 1 -- 172.21.15.39:0/2638428278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87f750510 con 0x55b87f83b810 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbef3fff700 1 --2- 172.21.15.39:0/2638428278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f84b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbef904a700 1 --2- 172.21.15.39:0/2638428278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f843260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbeec002230 tx=0x7fbeec00e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1c8f175b7c2dec server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbedf7fe700 1 -- 172.21.15.39:0/2638428278 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec003c00 con 0x55b87f83b810 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbedf7fe700 1 -- 172.21.15.39:0/2638428278 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbeec021e90 con 0x55b87f83b810 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbedf7fe700 1 -- 172.21.15.39:0/2638428278 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00ecc0 con 0x55b87f83b810 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.274+0000 7fbf06c68b80 1 -- 172.21.15.39:0/2638428278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 msgr2=0x55b87f843260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/2638428278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f843260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbeec002230 tx=0x7fbeec00e4c0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 -- 172.21.15.39:0/2638428278 shutdown_connections 2024-05-24T11:37:59.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/2638428278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83a4f0 0x55b87f83a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/2638428278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f843260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/2638428278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f84b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 -- 172.21.15.39:0/2638428278 >> 172.21.15.39:0/2638428278 conn(0x55b87f7981c0 msgr2=0x55b87f83bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 -- 172.21.15.39:0/2638428278 shutdown_connections 2024-05-24T11:37:59.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 -- 172.21.15.39:0/2638428278 wait complete. 2024-05-24T11:37:59.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 -- start start 2024-05-24T11:37:59.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 Processor -- start 2024-05-24T11:37:59.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83a4f0 0x55b87f845410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f845950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f84ab10 con 0x55b87f83b810 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f84ac80 con 0x55b87f83ae50 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f84ae40 con 0x55b87f83a4f0 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbf06c68b80 10 client.0 _ll_register_callbacks cb 0x7fbeec010580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.275+0000 7fbef904a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef904a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45878/0 (socket says 172.21.15.39:45878) 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef904a700 1 -- 172.21.15.39:0/3859971824 learned_addr learned my addr 172.21.15.39:0/3859971824 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef3fff700 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f845950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef904a700 1 -- 172.21.15.39:0/3859971824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83a4f0 msgr2=0x55b87f845410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef904a700 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83a4f0 0x55b87f845410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef904a700 1 -- 172.21.15.39:0/3859971824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 msgr2=0x55b87f845950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef904a700 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f845950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef904a700 1 -- 172.21.15.39:0/3859971824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b87f5b7b10 con 0x55b87f83b810 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef3fff700 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f845950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbef904a700 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84a5d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbeec008fd0 tx=0x7fbeec004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9cfe364f78125e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbed3fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbed3fff700 1 -- 172.21.15.39:0/3859971824 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00ee50 con 0x55b87f83b810 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbed3fff700 1 -- 172.21.15.39:0/3859971824 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbeec00e650 con 0x55b87f83b810 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbf06c68b80 1 -- 172.21.15.39:0/3859971824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbee801e040 con 0x55b87f83b810 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbed3fff700 1 -- 172.21.15.39:0/3859971824 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeec033450 con 0x55b87f83b810 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbf06c68b80 10 client.15261 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.276+0000 7fbf06c68b80 1 -- 172.21.15.39:0/3859971824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b87f60fc80 con 0x55b87f83b810 2024-05-24T11:37:59.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbed3fff700 1 -- 172.21.15.39:0/3859971824 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbee801e040 con 0x55b87f83b810 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbed3fff700 1 -- 172.21.15.39:0/3859971824 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbeec02a450 con 0x55b87f83b810 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbf06c68b80 10 client.15261 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbf06c68b80 10 client.15261 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbf06c68b80 20 client.15261 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x55b87f8aa070 0x55b87f8a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbf06c68b80 4 client.15261 mds_command: new command op to 14631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbf06c68b80 1 -- 172.21.15.39:0/3859971824 --> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b87f782e70 con 0x55b87f8aa070 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbef8849700 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x55b87f8aa070 0x55b87f8a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbed3fff700 10 client.15261 ms_handle_connect on v2:172.21.15.39:6838/3006971240 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbef8849700 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x55b87f8aa070 0x55b87f8a69a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96ec3a8d8241bbc1 server_cookie=449969255527a0a8 in_seq=0 out_seq=0 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbed3fff700 1 -- 172.21.15.39:0/3859971824 <== mds.4 v2:172.21.15.39:6838/3006971240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b87f782e70 con 0x55b87f8aa070 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbed3fff700 10 client.15261 handle_command_reply: tid=0 2024-05-24T11:37:59.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.277+0000 7fbf06c68b80 1 client.15261 shutdown 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 -- 172.21.15.39:0/3859971824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 msgr2=0x55b87f84a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84a5d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbeec008fd0 tx=0x7fbeec004cf0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 -- 172.21.15.39:0/3859971824 shutdown_connections 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.39:6838/3006971240,v1:172.21.15.39:6839/3006971240] conn(0x55b87f8aa070 0x55b87f8a69a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b87f83a4f0 0x55b87f845410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b87f83b810 0x55b87f84a5d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbed3fff700 0 client.15261 ms_handle_reset on v2:172.21.15.39:6838/3006971240 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 --2- 172.21.15.39:0/3859971824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b87f83ae50 0x55b87f845950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 -- 172.21.15.39:0/3859971824 >> 172.21.15.39:0/3859971824 conn(0x55b87f7981c0 msgr2=0x55b87f843d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 -- 172.21.15.39:0/3859971824 shutdown_connections 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 1 -- 172.21.15.39:0/3859971824 wait complete. 2024-05-24T11:37:59.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbf06c68b80 20 client.15261 trim_cache size 0 max 16384 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 14631, 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 64.699063082999999, 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-05-24T11:37:59.291 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:59.292 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 80.127993716000006 2024-05-24T11:37:59.292 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:59.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 -- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbee40a11b0 msgr2=0x7fbee40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbee40a11b0 0x7fbee40a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005910 tx=0x7fbee8005fb0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 -- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef41020f0 msgr2=0x7fbef41973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef41020f0 0x7fbef41973a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbee000e9a0 tx=0x7fbee000ecb0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 -- 172.21.15.39:0/1593619379 shutdown_connections 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fbee40a11b0 0x7fbee40a3670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef4102ab0 0x7fbef419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbef4100f20 0x7fbef4196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 --2- 172.21.15.39:0/1593619379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbef41020f0 0x7fbef41973a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.278+0000 7fbefaaad700 1 -- 172.21.15.39:0/1593619379 >> 172.21.15.39:0/1593619379 conn(0x7fbef40fc4e0 msgr2=0x7fbef41055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.279+0000 7fbefaaad700 1 -- 172.21.15.39:0/1593619379 shutdown_connections 2024-05-24T11:37:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.279+0000 7fbefaaad700 1 -- 172.21.15.39:0/1593619379 wait complete. 2024-05-24T11:37:59.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-24T11:37:59.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.418+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3046501777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 msgr2=0x7fe3d410b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.418+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3046501777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 0x7fe3d410b2b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00b0a0 tx=0x7fe3cc00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.418+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3046501777 shutdown_connections 2024-05-24T11:37:59.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.418+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3046501777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d4074f80 0x7fe3d40733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.418+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3046501777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 0x7fe3d410b2b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.418+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3046501777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d40739a0 0x7fe3d4073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.418+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3046501777 >> 172.21.15.39:0/3046501777 conn(0x7fe3d40fc4e0 msgr2=0x7fe3d40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.418+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3046501777 shutdown_connections 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.418+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3046501777 wait complete. 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3dae2b700 1 Processor -- start 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3dae2b700 1 -- start start 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3dae2b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d40739a0 0x7fe3d41990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3dae2b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d4074f80 0x7fe3d41995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3dae2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 0x7fe3d41a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3dae2b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d41a0bd0 con 0x7fe3d4108e40 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3dae2b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d41a0d10 con 0x7fe3d40739a0 2024-05-24T11:37:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3dae2b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d41a0ea0 con 0x7fe3d4074f80 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d93c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 0x7fe3d41a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d93c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 0x7fe3d41a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45890/0 (socket says 172.21.15.39:45890) 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d93c8700 1 -- 172.21.15.39:0/3769703500 learned_addr learned my addr 172.21.15.39:0/3769703500 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d40739a0 0x7fe3d41990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d3fff700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d4074f80 0x7fe3d41995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d93c8700 1 -- 172.21.15.39:0/3769703500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d4074f80 msgr2=0x7fe3d41995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d93c8700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d4074f80 0x7fe3d41995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d93c8700 1 -- 172.21.15.39:0/3769703500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d40739a0 msgr2=0x7fe3d41990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d93c8700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d40739a0 0x7fe3d41990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.419+0000 7fe3d93c8700 1 -- 172.21.15.39:0/3769703500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc009d00 con 0x7fe3d4108e40 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.420+0000 7fe3d93c8700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 0x7fe3d41a0600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc007a30 tx=0x7fe3cc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.420+0000 7fe3d1ffb700 1 -- 172.21.15.39:0/3769703500 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00e040 con 0x7fe3d4108e40 2024-05-24T11:37:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.420+0000 7fe3d1ffb700 1 -- 172.21.15.39:0/3769703500 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3cc0056e0 con 0x7fe3d4108e40 2024-05-24T11:37:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.420+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3769703500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d41a1120 con 0x7fe3d4108e40 2024-05-24T11:37:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.420+0000 7fe3d1ffb700 1 -- 172.21.15.39:0/3769703500 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc01dad0 con 0x7fe3d4108e40 2024-05-24T11:37:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.420+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3769703500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d41a15a0 con 0x7fe3d4108e40 2024-05-24T11:37:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.422+0000 7fe3d1ffb700 1 -- 172.21.15.39:0/3769703500 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe3cc005850 con 0x7fe3d4108e40 2024-05-24T11:37:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.423+0000 7fe3d1ffb700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c40a1230 0x7fe3c40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.423+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c40a1230 0x7fe3c40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.423+0000 7fe3d1ffb700 1 -- 172.21.15.39:0/3769703500 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe3cc0ce120 con 0x7fe3d4108e40 2024-05-24T11:37:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.423+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c40a1230 0x7fe3c40a36f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800ac50 tx=0x7fe3c800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.423+0000 7fe3e6fe6b80 1 Processor -- start 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3e6fe6b80 1 -- start start 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 0x556e243ef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243f0810 0x556e243f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e24346100 con 0x556e243efe50 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e24343ac0 con 0x556e243ef4f0 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e2433f3b0 con 0x556e243f0810 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45894/0 (socket says 172.21.15.39:45894) 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d3fff700 1 -- 172.21.15.39:0/3539963865 learned_addr learned my addr 172.21.15.39:0/3539963865 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3539963865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 0x556e243ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d93c8700 1 --2- 172.21.15.39:0/3539963865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243f0810 0x556e243f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d3fff700 1 -- 172.21.15.39:0/3539963865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243f0810 msgr2=0x556e243f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d3fff700 1 --2- 172.21.15.39:0/3539963865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243f0810 0x556e243f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d3fff700 1 -- 172.21.15.39:0/3539963865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 msgr2=0x556e243ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d3fff700 1 --2- 172.21.15.39:0/3539963865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 0x556e243ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d3fff700 1 -- 172.21.15.39:0/3539963865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e241c0360 con 0x556e243efe50 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3d3fff700 1 --2- 172.21.15.39:0/3539963865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f02d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000bd70 tx=0x7fe3c000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee05353bf7c2dda1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.424+0000 7fe3beffd700 1 -- 172.21.15.39:0/3539963865 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000fda0 con 0x556e243efe50 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3beffd700 1 -- 172.21.15.39:0/3539963865 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c001a8f0 con 0x556e243efe50 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3beffd700 1 -- 172.21.15.39:0/3539963865 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0011730 con 0x556e243efe50 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3539963865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 msgr2=0x556e243f02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3539963865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f02d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000bd70 tx=0x7fe3c000f640 comp rx=0 tx=0).stop 2024-05-24T11:37:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3539963865 shutdown_connections 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3539963865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243f0810 0x556e243f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3539963865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 0x556e243ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3539963865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f02d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3539963865 >> 172.21.15.39:0/3539963865 conn(0x556e2434d1c0 msgr2=0x556e243e9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3539963865 shutdown_connections 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.425+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3539963865 wait complete. 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3e6fe6b80 1 -- start start 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3e6fe6b80 1 Processor -- start 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 0x556e243f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243f0810 0x556e24400330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e242db510 con 0x556e243efe50 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e242db1b0 con 0x556e243ef4f0 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e241c2650 con 0x556e243f0810 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3e6fe6b80 10 client.0 _ll_register_callbacks cb 0x556e242e4ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3d3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3d3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45910/0 (socket says 172.21.15.39:45910) 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3d3fff700 1 -- 172.21.15.39:0/3506448364 learned_addr learned my addr 172.21.15.39:0/3506448364 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 0x556e243f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3d3fff700 1 -- 172.21.15.39:0/3506448364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243f0810 msgr2=0x556e24400330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3d3fff700 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243f0810 0x556e24400330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3d3fff700 1 -- 172.21.15.39:0/3506448364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 msgr2=0x556e243f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3d3fff700 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 0x556e243f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.426+0000 7fe3d3fff700 1 -- 172.21.15.39:0/3506448364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c000b9a0 con 0x556e243efe50 2024-05-24T11:37:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3d3fff700 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f92f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000b8c0 tx=0x7fe3c00057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db55e0dcd8e7885b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3506448364 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c001ac60 con 0x556e243efe50 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3506448364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e24298220 con 0x556e243efe50 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3506448364 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe3c0004020 con 0x556e243efe50 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3506448364 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3c001adc0 con 0x556e243efe50 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3e6fe6b80 10 client.15273 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3506448364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556e2416cb10 con 0x556e243efe50 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3506448364 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c0034080 con 0x556e243efe50 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3506448364 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3c0022070 con 0x556e243efe50 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3e6fe6b80 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3e6fe6b80 10 client.15273 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3e6fe6b80 20 client.15273 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x556e24403880 0x556e24409170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3e6fe6b80 4 client.15273 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.427+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3506448364 --> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e243be7c0 con 0x556e24403880 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.428+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x556e24403880 0x556e24409170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.428+0000 7fe3b3fff700 10 client.15273 ms_handle_connect on v2:172.21.15.137:6836/967972436 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.428+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x556e24403880 0x556e24409170 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=286f496fa2bff581 server_cookie=7aa6d22bd1c70ac6 in_seq=0 out_seq=0 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3506448364 <== mds.3 v2:172.21.15.137:6836/967972436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e243be7c0 con 0x556e24403880 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3b3fff700 10 client.15273 handle_command_reply: tid=0 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 client.15273 shutdown 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3506448364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 msgr2=0x556e243f92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f92f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000b8c0 tx=0x7fe3c00057d0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3506448364 shutdown_connections 2024-05-24T11:37:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x556e24403880 0x556e24409170 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243f0810 0x556e24400330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3b3fff700 0 client.15273 ms_handle_reset on v2:172.21.15.137:6836/967972436 2024-05-24T11:37:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243ef4f0 0x556e243f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3506448364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243efe50 0x556e243f92f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3506448364 >> 172.21.15.39:0/3506448364 conn(0x556e2434d1c0 msgr2=0x556e243e9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3506448364 shutdown_connections 2024-05-24T11:37:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3506448364 wait complete. 2024-05-24T11:37:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.429+0000 7fe3e6fe6b80 20 client.15273 trim_cache size 0 max 16384 2024-05-24T11:37:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.447+0000 7fe3e6fe6b80 1 Processor -- start 2024-05-24T11:37:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.447+0000 7fe3e6fe6b80 1 -- start start 2024-05-24T11:37:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243ef4f0 0x556e243f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243efe50 0x556e243ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243f0810 0x556e244009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e242de5a0 con 0x556e243ef4f0 2024-05-24T11:37:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e242fd2b0 con 0x556e243f0810 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e243e47b0 con 0x556e243efe50 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3d8bc7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243ef4f0 0x556e243f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3d8bc7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243ef4f0 0x556e243f9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45922/0 (socket says 172.21.15.39:45922) 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3d8bc7700 1 -- 172.21.15.39:0/54098976 learned_addr learned my addr 172.21.15.39:0/54098976 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3d8bc7700 1 -- 172.21.15.39:0/54098976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243efe50 msgr2=0x556e243ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/54098976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243efe50 0x556e243ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3d8bc7700 1 -- 172.21.15.39:0/54098976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243f0810 msgr2=0x556e244009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/54098976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243f0810 0x556e244009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3d8bc7700 1 -- 172.21.15.39:0/54098976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e24305510 con 0x556e243ef4f0 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/54098976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243ef4f0 0x556e243f9a30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800fa30 tx=0x7fe3c8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01571b13f642827 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3bf7fe700 1 -- 172.21.15.39:0/54098976 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8023040 con 0x556e243ef4f0 2024-05-24T11:37:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3bf7fe700 1 -- 172.21.15.39:0/54098976 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c80045a0 con 0x556e243ef4f0 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.448+0000 7fe3bf7fe700 1 -- 172.21.15.39:0/54098976 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80108c0 con 0x556e243ef4f0 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/54098976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243ef4f0 msgr2=0x556e243f9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/54098976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243ef4f0 0x556e243f9a30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800fa30 tx=0x7fe3c8012000 comp rx=0 tx=0).stop 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/54098976 shutdown_connections 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/54098976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243f0810 0x556e244009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/54098976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243ef4f0 0x556e243f9a30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/54098976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243efe50 0x556e243ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/54098976 >> 172.21.15.39:0/54098976 conn(0x556e2434d1c0 msgr2=0x556e243f0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/54098976 shutdown_connections 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/54098976 wait complete. 2024-05-24T11:37:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 -- start start 2024-05-24T11:37:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.449+0000 7fe3e6fe6b80 1 Processor -- start 2024-05-24T11:37:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243ef4f0 0x556e243e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243efe50 0x556e243e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3e6fe6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243f0810 0x556e243ff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e243ffea0 con 0x556e243f0810 2024-05-24T11:37:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e24400010 con 0x556e243efe50 2024-05-24T11:37:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3e6fe6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e244001d0 con 0x556e243ef4f0 2024-05-24T11:37:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3e6fe6b80 10 client.0 _ll_register_callbacks cb 0x556e242e4ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d93c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243f0810 0x556e243ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d93c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243f0810 0x556e243ff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45932/0 (socket says 172.21.15.39:45932) 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d93c8700 1 -- 172.21.15.39:0/3558959028 learned_addr learned my addr 172.21.15.39:0/3558959028 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d3fff700 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243efe50 0x556e243e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243ef4f0 0x556e243e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d93c8700 1 -- 172.21.15.39:0/3558959028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243ef4f0 msgr2=0x556e243e8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d93c8700 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243ef4f0 0x556e243e8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d93c8700 1 -- 172.21.15.39:0/3558959028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243efe50 msgr2=0x556e243e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d93c8700 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243efe50 0x556e243e9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d93c8700 1 -- 172.21.15.39:0/3558959028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c800fd70 con 0x556e243f0810 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d3fff700 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243efe50 0x556e243e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3d93c8700 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243f0810 0x556e243ff960 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc005b00 tx=0x7fe3cc09b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7976090a82f387d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3558959028 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc01d480 con 0x556e243f0810 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3558959028 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe3cc09bca0 con 0x556e243f0810 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3558959028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3c001a8f0 con 0x556e243f0810 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.450+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3558959028 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3cc09b5a0 con 0x556e243f0810 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3e6fe6b80 10 client.15279 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3558959028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556e241c4c80 con 0x556e243f0810 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3558959028 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c001a8f0 con 0x556e243f0810 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3558959028 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3cc09ab20 con 0x556e243f0810 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3e6fe6b80 10 client.15279 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3e6fe6b80 10 client.15279 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-24T11:37:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3e6fe6b80 20 client.15279 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x556e2445f260 0x556e2445bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3e6fe6b80 4 client.15279 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3558959028 --> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556e24337e70 con 0x556e2445f260 2024-05-24T11:37:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.451+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x556e2445f260 0x556e2445bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3d8bc7700 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x556e2445f260 0x556e2445bc50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd5b24385fe43141 server_cookie=d88a0527b88c642b in_seq=0 out_seq=0 2024-05-24T11:37:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3b3fff700 10 client.15279 ms_handle_connect on v2:172.21.15.137:6836/967972436 2024-05-24T11:37:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3b3fff700 1 -- 172.21.15.39:0/3558959028 <== mds.3 v2:172.21.15.137:6836/967972436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556e24337e70 con 0x556e2445f260 2024-05-24T11:37:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3b3fff700 10 client.15279 handle_command_reply: tid=0 2024-05-24T11:37:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 client.15279 shutdown 2024-05-24T11:37:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3558959028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243f0810 msgr2=0x556e243ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243f0810 0x556e243ff960 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc005b00 tx=0x7fe3cc09b420 comp rx=0 tx=0).stop 2024-05-24T11:37:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3558959028 shutdown_connections 2024-05-24T11:37:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.137:6836/967972436,v1:172.21.15.137:6837/967972436] conn(0x556e2445f260 0x556e2445bc50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e243ef4f0 0x556e243e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e243f0810 0x556e243ff960 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3b3fff700 0 client.15279 ms_handle_reset on v2:172.21.15.137:6836/967972436 2024-05-24T11:37:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 --2- 172.21.15.39:0/3558959028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e243efe50 0x556e243e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3558959028 >> 172.21.15.39:0/3558959028 conn(0x556e2434d1c0 msgr2=0x556e243f0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3558959028 shutdown_connections 2024-05-24T11:37:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 1 -- 172.21.15.39:0/3558959028 wait complete. 2024-05-24T11:37:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.452+0000 7fe3e6fe6b80 20 client.15279 trim_cache size 0 max 16384 2024-05-24T11:37:59.469 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:59.469 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:59.469 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-05-24T11:37:59.469 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24490, 2024-05-24T11:37:59.469 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:59.470 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:59.470 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:59.470 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 65.845648705000002, 2024-05-24T11:37:59.470 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:59.470 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-05-24T11:37:59.470 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:59.470 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 85.901013821999996 2024-05-24T11:37:59.470 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c40a1230 msgr2=0x7fe3c40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c40a1230 0x7fe3c40a36f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800ac50 tx=0x7fe3c800a380 comp rx=0 tx=0).stop 2024-05-24T11:37:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 msgr2=0x7fe3d41a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 0x7fe3d41a0600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc007a30 tx=0x7fe3cc004c80 comp rx=0 tx=0).stop 2024-05-24T11:37:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3769703500 shutdown_connections 2024-05-24T11:37:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fe3c40a1230 0x7fe3c40a36f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d4108e40 0x7fe3d41a0600 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d40739a0 0x7fe3d41990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 --2- 172.21.15.39:0/3769703500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d4074f80 0x7fe3d41995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3769703500 >> 172.21.15.39:0/3769703500 conn(0x7fe3d40fc4e0 msgr2=0x7fe3d4104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3769703500 shutdown_connections 2024-05-24T11:37:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.453+0000 7fe3dae2b700 1 -- 172.21.15.39:0/3769703500 wait complete. 2024-05-24T11:37:59.472 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-24T11:37:59.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.594+0000 7ff267c2e700 1 -- 172.21.15.39:0/1251429805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2601014e0 msgr2=0x7ff260101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.594+0000 7ff267c2e700 1 --2- 172.21.15.39:0/1251429805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2601014e0 0x7ff260101900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009990 tx=0x7ff25c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 -- 172.21.15.39:0/1251429805 shutdown_connections 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 --2- 172.21.15.39:0/1251429805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff260100a60 0x7ff260108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 --2- 172.21.15.39:0/1251429805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2601014e0 0x7ff260101900 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 --2- 172.21.15.39:0/1251429805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2601000a0 0x7ff260100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 -- 172.21.15.39:0/1251429805 >> 172.21.15.39:0/1251429805 conn(0x7ff2600fab50 msgr2=0x7ff2600fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 -- 172.21.15.39:0/1251429805 shutdown_connections 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 -- 172.21.15.39:0/1251429805 wait complete. 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 Processor -- start 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 -- start start 2024-05-24T11:37:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2601000a0 0x7ff260198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff260100a60 0x7ff260199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2601014e0 0x7ff2601a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601a0a80 con 0x7ff260100a60 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601a0bc0 con 0x7ff2601014e0 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.595+0000 7ff267c2e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601a0d50 con 0x7ff2601000a0 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2651c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff260100a60 0x7ff260199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2651c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff260100a60 0x7ff260199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45956/0 (socket says 172.21.15.39:45956) 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2651c9700 1 -- 172.21.15.39:0/2696997414 learned_addr learned my addr 172.21.15.39:0/2696997414 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2661cb700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2601014e0 0x7ff2601a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2659ca700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2601000a0 0x7ff260198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2651c9700 1 -- 172.21.15.39:0/2696997414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2601000a0 msgr2=0x7ff260198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2651c9700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2601000a0 0x7ff260198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2651c9700 1 -- 172.21.15.39:0/2696997414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2601014e0 msgr2=0x7ff2601a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2651c9700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2601014e0 0x7ff2601a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2651c9700 1 -- 172.21.15.39:0/2696997414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c0095f0 con 0x7ff260100a60 2024-05-24T11:37:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff2651c9700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff260100a60 0x7ff260199490 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c9b0 tx=0x7ff25000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff256ffd700 1 -- 172.21.15.39:0/2696997414 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250007b00 con 0x7ff260100a60 2024-05-24T11:37:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff256ffd700 1 -- 172.21.15.39:0/2696997414 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff25000ce50 con 0x7ff260100a60 2024-05-24T11:37:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff256ffd700 1 -- 172.21.15.39:0/2696997414 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250018420 con 0x7ff260100a60 2024-05-24T11:37:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff267c2e700 1 -- 172.21.15.39:0/2696997414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2601a0fd0 con 0x7ff260100a60 2024-05-24T11:37:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.596+0000 7ff267c2e700 1 -- 172.21.15.39:0/2696997414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2601a1460 con 0x7ff260100a60 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.598+0000 7ff256ffd700 1 -- 172.21.15.39:0/2696997414 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff25000f450 con 0x7ff260100a60 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.599+0000 7ff256ffd700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff24c0a11b0 0x7ff24c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.599+0000 7ff2659ca700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff24c0a11b0 0x7ff24c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.599+0000 7ff256ffd700 1 -- 172.21.15.39:0/2696997414 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff2500cc8a0 con 0x7ff260100a60 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.599+0000 7ff2659ca700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff24c0a11b0 0x7ff24c0a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff25c000c00 tx=0x7ff25c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff273de9b80 1 Processor -- start 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff273de9b80 1 -- start start 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520dee4f0 0x563520dee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520def2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520def810 0x563520defc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff273de9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520d45100 con 0x563520dee4f0 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff273de9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520d42ac0 con 0x563520def810 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff273de9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520d3e3b0 con 0x563520deee50 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff2659ca700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520dee4f0 0x563520dee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff2659ca700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520dee4f0 0x563520dee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45960/0 (socket says 172.21.15.39:45960) 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.600+0000 7ff2659ca700 1 -- 172.21.15.39:0/145276215 learned_addr learned my addr 172.21.15.39:0/145276215 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff2651c9700 1 --2- 172.21.15.39:0/145276215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520def2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff2659ca700 1 -- 172.21.15.39:0/145276215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 msgr2=0x563520def2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff2659ca700 1 --2- 172.21.15.39:0/145276215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520def2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff2661cb700 1 --2- 172.21.15.39:0/145276215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520def810 0x563520defc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff2659ca700 1 -- 172.21.15.39:0/145276215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520def810 msgr2=0x563520defc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff2659ca700 1 --2- 172.21.15.39:0/145276215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520def810 0x563520defc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff2659ca700 1 -- 172.21.15.39:0/145276215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563520bbf360 con 0x563520dee4f0 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff2659ca700 1 --2- 172.21.15.39:0/145276215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520dee4f0 0x563520dee910 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff25c015000 tx=0x7ff25c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2f19d869d897da server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff24bfff700 1 -- 172.21.15.39:0/145276215 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c02b070 con 0x563520dee4f0 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff24bfff700 1 -- 172.21.15.39:0/145276215 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff25c01e040 con 0x563520dee4f0 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.601+0000 7ff24bfff700 1 -- 172.21.15.39:0/145276215 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c003820 con 0x563520dee4f0 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 -- 172.21.15.39:0/145276215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520dee4f0 msgr2=0x563520dee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 --2- 172.21.15.39:0/145276215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520dee4f0 0x563520dee910 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff25c015000 tx=0x7ff25c020000 comp rx=0 tx=0).stop 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 -- 172.21.15.39:0/145276215 shutdown_connections 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 --2- 172.21.15.39:0/145276215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520def810 0x563520defc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 --2- 172.21.15.39:0/145276215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520dee4f0 0x563520dee910 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 --2- 172.21.15.39:0/145276215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520def2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 -- 172.21.15.39:0/145276215 >> 172.21.15.39:0/145276215 conn(0x563520d4c1c0 msgr2=0x563520de8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 -- 172.21.15.39:0/145276215 shutdown_connections 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 -- 172.21.15.39:0/145276215 wait complete. 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 -- start start 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.602+0000 7ff273de9b80 1 Processor -- start 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 0x563520df7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520df8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff273de9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520cda510 con 0x563520def810 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff273de9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520cda1b0 con 0x563520dee4f0 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff273de9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520bc1650 con 0x563520deee50 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff273de9b80 10 client.0 _ll_register_callbacks cb 0x563520ce3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2661cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2661cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45974/0 (socket says 172.21.15.39:45974) 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2661cb700 1 -- 172.21.15.39:0/1425661597 learned_addr learned my addr 172.21.15.39:0/1425661597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2651c9700 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520df8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2659ca700 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 0x563520df7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2661cb700 1 -- 172.21.15.39:0/1425661597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 msgr2=0x563520df8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2661cb700 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520df8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2661cb700 1 -- 172.21.15.39:0/1425661597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 msgr2=0x563520df7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2661cb700 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 0x563520df7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2661cb700 1 -- 172.21.15.39:0/1425661597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff260108cb0 con 0x563520def810 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2661cb700 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff258003f70 tx=0x7ff258004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b573d7e0bed29c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2497fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2497fa700 1 -- 172.21.15.39:0/1425661597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258004bb0 con 0x563520def810 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff2497fa700 1 -- 172.21.15.39:0/1425661597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff258004d10 con 0x563520def810 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.603+0000 7ff273de9b80 1 -- 172.21.15.39:0/1425661597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563520c97220 con 0x563520def810 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff2497fa700 1 -- 172.21.15.39:0/1425661597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff258007710 con 0x563520def810 2024-05-24T11:37:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff273de9b80 10 client.15291 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff273de9b80 1 -- 172.21.15.39:0/1425661597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563520b6bb10 con 0x563520def810 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff2497fa700 1 -- 172.21.15.39:0/1425661597 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff258024020 con 0x563520def810 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff2497fa700 1 -- 172.21.15.39:0/1425661597 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff258014070 con 0x563520def810 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff273de9b80 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff273de9b80 10 client.15291 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff273de9b80 20 client.15291 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x563520e02440 0x563520e08260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff273de9b80 4 client.15291 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff273de9b80 1 -- 172.21.15.39:0/1425661597 --> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563520dbd7c0 con 0x563520e02440 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.604+0000 7ff2659ca700 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x563520e02440 0x563520e08260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.605+0000 7ff2497fa700 10 client.15291 ms_handle_connect on v2:172.21.15.53:6836/3764768463 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.605+0000 7ff2659ca700 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x563520e02440 0x563520e08260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7cf4d263669ca5d server_cookie=4a11b1dc772356d in_seq=0 out_seq=0 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.605+0000 7ff2497fa700 1 -- 172.21.15.39:0/1425661597 <== mds.2 v2:172.21.15.53:6836/3764768463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563520dbd7c0 con 0x563520e02440 2024-05-24T11:37:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.605+0000 7ff2497fa700 10 client.15291 handle_command_reply: tid=0 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.605+0000 7ff273de9b80 1 client.15291 shutdown 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 -- 172.21.15.39:0/1425661597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 msgr2=0x563520dff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff258003f70 tx=0x7ff258004320 comp rx=0 tx=0).stop 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 -- 172.21.15.39:0/1425661597 shutdown_connections 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x563520e02440 0x563520e08260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff240 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 0x563520df7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff2497fa700 0 client.15291 ms_handle_reset on v2:172.21.15.53:6836/3764768463 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1425661597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520df8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 -- 172.21.15.39:0/1425661597 >> 172.21.15.39:0/1425661597 conn(0x563520d4c1c0 msgr2=0x563520de8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 -- 172.21.15.39:0/1425661597 shutdown_connections 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 1 -- 172.21.15.39:0/1425661597 wait complete. 2024-05-24T11:37:59.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.606+0000 7ff273de9b80 20 client.15291 trim_cache size 0 max 16384 2024-05-24T11:37:59.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff273de9b80 1 Processor -- start 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff273de9b80 1 -- start start 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520dee4f0 0x563520df8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520deee50 0x563520dee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff273de9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520d3b010 con 0x563520def810 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff273de9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520cdd5a0 con 0x563520deee50 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff273de9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520e00fd0 con 0x563520dee4f0 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff2661cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff2661cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45980/0 (socket says 172.21.15.39:45980) 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.624+0000 7ff2661cb700 1 -- 172.21.15.39:0/3787723301 learned_addr learned my addr 172.21.15.39:0/3787723301 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff2651c9700 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520deee50 0x563520dee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff2659ca700 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520dee4f0 0x563520df8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff2661cb700 1 -- 172.21.15.39:0/3787723301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520dee4f0 msgr2=0x563520df8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff2661cb700 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520dee4f0 0x563520df8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff2661cb700 1 -- 172.21.15.39:0/3787723301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520deee50 msgr2=0x563520dee8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff2661cb700 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520deee50 0x563520dee8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff2661cb700 1 -- 172.21.15.39:0/3787723301 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563520d04510 con 0x563520def810 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff2651c9700 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520deee50 0x563520dee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff2661cb700 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff8f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff258009fd0 tx=0x7ff258003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d372da1064cce3a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff254da5700 1 -- 172.21.15.39:0/3787723301 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2580153f0 con 0x563520def810 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff254da5700 1 -- 172.21.15.39:0/3787723301 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff258024030 con 0x563520def810 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.625+0000 7ff254da5700 1 -- 172.21.15.39:0/3787723301 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258015850 con 0x563520def810 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 -- 172.21.15.39:0/3787723301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 msgr2=0x563520dff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff8f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff258009fd0 tx=0x7ff258003b50 comp rx=0 tx=0).stop 2024-05-24T11:37:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 -- 172.21.15.39:0/3787723301 shutdown_connections 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520dee4f0 0x563520df8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dff8f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 --2- 172.21.15.39:0/3787723301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520deee50 0x563520dee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 -- 172.21.15.39:0/3787723301 >> 172.21.15.39:0/3787723301 conn(0x563520d4c1c0 msgr2=0x563520defbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 -- 172.21.15.39:0/3787723301 shutdown_connections 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 -- 172.21.15.39:0/3787723301 wait complete. 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 -- start start 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 Processor -- start 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 0x563520de7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520de8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dfe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520dfe9a0 con 0x563520def810 2024-05-24T11:37:59.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520dfeb10 con 0x563520dee4f0 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.626+0000 7ff273de9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563520dfecd0 con 0x563520deee50 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff273de9b80 10 client.0 _ll_register_callbacks cb 0x563520ce3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2661cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dfe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2661cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dfe460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45982/0 (socket says 172.21.15.39:45982) 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2661cb700 1 -- 172.21.15.39:0/1918421014 learned_addr learned my addr 172.21.15.39:0/1918421014 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2659ca700 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 0x563520de7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2661cb700 1 -- 172.21.15.39:0/1918421014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 msgr2=0x563520de8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2661cb700 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520de8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2661cb700 1 -- 172.21.15.39:0/1918421014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 msgr2=0x563520de7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2661cb700 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 0x563520de7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2661cb700 1 -- 172.21.15.39:0/1918421014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563520b6bb10 con 0x563520def810 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2659ca700 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 0x563520de7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2661cb700 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dfe460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff25801e950 tx=0x7ff25801ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43e224845837679 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2497fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2497fa700 1 -- 172.21.15.39:0/1918421014 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258015530 con 0x563520def810 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2497fa700 1 -- 172.21.15.39:0/1918421014 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff258025080 con 0x563520def810 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff273de9b80 1 -- 172.21.15.39:0/1918421014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff25c01e040 con 0x563520def810 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff2497fa700 1 -- 172.21.15.39:0/1918421014 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2580157d0 con 0x563520def810 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff273de9b80 10 client.15297 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.627+0000 7ff273de9b80 1 -- 172.21.15.39:0/1918421014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563520bc3c80 con 0x563520def810 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff2497fa700 1 -- 172.21.15.39:0/1918421014 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff258041020 con 0x563520def810 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff2497fa700 1 -- 172.21.15.39:0/1918421014 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff258030040 con 0x563520def810 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff273de9b80 10 client.15297 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff273de9b80 10 client.15297 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff273de9b80 20 client.15297 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x563520e5de70 0x563520e5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff273de9b80 4 client.15297 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:59.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff273de9b80 1 -- 172.21.15.39:0/1918421014 --> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563520d36e70 con 0x563520e5de70 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff2659ca700 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x563520e5de70 0x563520e5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff2659ca700 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x563520e5de70 0x563520e5a760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ad5b37c5c9f5723 server_cookie=d0b99d52d12c5995 in_seq=0 out_seq=0 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.628+0000 7ff2497fa700 10 client.15297 ms_handle_connect on v2:172.21.15.53:6836/3764768463 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff2497fa700 1 -- 172.21.15.39:0/1918421014 <== mds.2 v2:172.21.15.53:6836/3764768463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563520d36e70 con 0x563520e5de70 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff2497fa700 10 client.15297 handle_command_reply: tid=0 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 client.15297 shutdown 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 -- 172.21.15.39:0/1918421014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 msgr2=0x563520dfe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dfe460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff25801e950 tx=0x7ff25801ea80 comp rx=0 tx=0).stop 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 -- 172.21.15.39:0/1918421014 shutdown_connections 2024-05-24T11:37:59.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.53:6836/3764768463,v1:172.21.15.53:6837/3764768463] conn(0x563520e5de70 0x563520e5a760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563520def810 0x563520dfe460 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563520dee4f0 0x563520de7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 --2- 172.21.15.39:0/1918421014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563520deee50 0x563520de8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff2497fa700 0 client.15297 ms_handle_reset on v2:172.21.15.53:6836/3764768463 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 -- 172.21.15.39:0/1918421014 >> 172.21.15.39:0/1918421014 conn(0x563520d4c1c0 msgr2=0x563520df5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 -- 172.21.15.39:0/1918421014 shutdown_connections 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 1 -- 172.21.15.39:0/1918421014 wait complete. 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff273de9b80 20 client.15297 trim_cache size 0 max 16384 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:59.643 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24506, 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 67.028196296000004, 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 84.111549178999994 2024-05-24T11:37:59.644 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:59.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff267c2e700 1 -- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff24c0a11b0 msgr2=0x7ff24c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff267c2e700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff24c0a11b0 0x7ff24c0a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff25c000c00 tx=0x7ff25c005fb0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff267c2e700 1 -- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff260100a60 msgr2=0x7ff260199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.629+0000 7ff267c2e700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff260100a60 0x7ff260199490 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c9b0 tx=0x7ff25000ccc0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.630+0000 7ff267c2e700 1 -- 172.21.15.39:0/2696997414 shutdown_connections 2024-05-24T11:37:59.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.630+0000 7ff267c2e700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7ff24c0a11b0 0x7ff24c0a3670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.630+0000 7ff267c2e700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2601000a0 0x7ff260198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.630+0000 7ff267c2e700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2601014e0 0x7ff2601a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.630+0000 7ff267c2e700 1 --2- 172.21.15.39:0/2696997414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff260100a60 0x7ff260199490 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.630+0000 7ff267c2e700 1 -- 172.21.15.39:0/2696997414 >> 172.21.15.39:0/2696997414 conn(0x7ff2600fab50 msgr2=0x7ff2600fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.630+0000 7ff267c2e700 1 -- 172.21.15.39:0/2696997414 shutdown_connections 2024-05-24T11:37:59.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.630+0000 7ff267c2e700 1 -- 172.21.15.39:0/2696997414 wait complete. 2024-05-24T11:37:59.648 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-24T11:37:59.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3266645310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 msgr2=0x7fb6e01013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3266645310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 0x7fb6e01013a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00099c0 tx=0x7fb6d0009cd0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3266645310 shutdown_connections 2024-05-24T11:37:59.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3266645310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6e0102b10 0x7fb6e01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3266645310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 0x7fb6e01013a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3266645310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e0102170 0x7fb6e01025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3266645310 >> 172.21.15.39:0/3266645310 conn(0x7fb6e00fc4f0 msgr2=0x7fb6e00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3266645310 shutdown_connections 2024-05-24T11:37:59.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3266645310 wait complete. 2024-05-24T11:37:59.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.770+0000 7fb6e5a19700 1 Processor -- start 2024-05-24T11:37:59.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6e5a19700 1 -- start start 2024-05-24T11:37:59.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6e5a19700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 0x7fb6e0196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6e5a19700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6e0102170 0x7fb6e01972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6e5a19700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e0102b10 0x7fb6e019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6e5a19700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019e8d0 con 0x7fb6e0100f80 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6e5a19700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019ea10 con 0x7fb6e0102b10 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6e5a19700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019eba0 con 0x7fb6e0102170 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6deffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 0x7fb6e0196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6deffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 0x7fb6e0196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46000/0 (socket says 172.21.15.39:46000) 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6deffd700 1 -- 172.21.15.39:0/3538592754 learned_addr learned my addr 172.21.15.39:0/3538592754 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6de7fc700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6e0102170 0x7fb6e01972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e0102b10 0x7fb6e019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6deffd700 1 -- 172.21.15.39:0/3538592754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6e0102170 msgr2=0x7fb6e01972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6deffd700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6e0102170 0x7fb6e01972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6deffd700 1 -- 172.21.15.39:0/3538592754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e0102b10 msgr2=0x7fb6e019e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6deffd700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e0102b10 0x7fb6e019e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6deffd700 1 -- 172.21.15.39:0/3538592754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d00095f0 con 0x7fb6e0100f80 2024-05-24T11:37:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6deffd700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 0x7fb6e0196da0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009f50 tx=0x7fb6d0003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6e4a17700 1 -- 172.21.15.39:0/3538592754 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d001d070 con 0x7fb6e0100f80 2024-05-24T11:37:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.771+0000 7fb6e4a17700 1 -- 172.21.15.39:0/3538592754 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6d000bda0 con 0x7fb6e0100f80 2024-05-24T11:37:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.772+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3538592754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e019ee20 con 0x7fb6e0100f80 2024-05-24T11:37:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.772+0000 7fb6e4a17700 1 -- 172.21.15.39:0/3538592754 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0022cd0 con 0x7fb6e0100f80 2024-05-24T11:37:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.772+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3538592754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e019f190 con 0x7fb6e0100f80 2024-05-24T11:37:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.774+0000 7fb6e4a17700 1 -- 172.21.15.39:0/3538592754 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb6d0022470 con 0x7fb6e0100f80 2024-05-24T11:37:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.774+0000 7fb6e4a17700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb6cc0a11b0 0x7fb6cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.775+0000 7fb6de7fc700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb6cc0a11b0 0x7fb6cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.775+0000 7fb6e4a17700 1 -- 172.21.15.39:0/3538592754 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb6d00cddf0 con 0x7fb6e0100f80 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.775+0000 7fb6de7fc700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb6cc0a11b0 0x7fb6cc0a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8005950 tx=0x7fb6c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.775+0000 7fb6f1bd4b80 1 Processor -- start 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6f1bd4b80 1 -- start start 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 0x558974b79910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b79e50 0x558974b7a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 0x558974b7ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974ad0100 con 0x558974b79e50 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974acdac0 con 0x558974b7a810 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974ac93b0 con 0x558974b794f0 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6de7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b79e50 0x558974b7a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6de7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b79e50 0x558974b7a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46004/0 (socket says 172.21.15.39:46004) 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6de7fc700 1 -- 172.21.15.39:0/67289706 learned_addr learned my addr 172.21.15.39:0/67289706 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6deffd700 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 0x558974b79910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6de7fc700 1 -- 172.21.15.39:0/67289706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 msgr2=0x558974b79910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6de7fc700 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 0x558974b79910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6de7fc700 1 -- 172.21.15.39:0/67289706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 msgr2=0x558974b7ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 0x558974b7ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6de7fc700 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 0x558974b7ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6de7fc700 1 -- 172.21.15.39:0/67289706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55897494a360 con 0x558974b79e50 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 0x558974b7ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.776+0000 7fb6de7fc700 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b79e50 0x558974b7a2d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8010fa0 tx=0x7fb6c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a1b4f6e25b78d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6d5ffb700 1 -- 172.21.15.39:0/67289706 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c801e620 con 0x558974b79e50 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6d5ffb700 1 -- 172.21.15.39:0/67289706 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6c8015070 con 0x558974b79e50 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6d5ffb700 1 -- 172.21.15.39:0/67289706 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c801c8d0 con 0x558974b79e50 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/67289706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b79e50 msgr2=0x558974b7a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b79e50 0x558974b7a2d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8010fa0 tx=0x7fb6c8013000 comp rx=0 tx=0).stop 2024-05-24T11:37:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/67289706 shutdown_connections 2024-05-24T11:37:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 0x558974b79910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 0x558974b7ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/67289706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b79e50 0x558974b7a2d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/67289706 >> 172.21.15.39:0/67289706 conn(0x558974ad71c0 msgr2=0x558974b73d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/67289706 shutdown_connections 2024-05-24T11:37:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.777+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/67289706 wait complete. 2024-05-24T11:37:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.778+0000 7fb6f1bd4b80 1 -- start start 2024-05-24T11:37:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.778+0000 7fb6f1bd4b80 1 Processor -- start 2024-05-24T11:37:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.778+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 0x558974b82db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.778+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b79e50 0x558974b832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.778+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b8a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.778+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974a65510 con 0x558974b7a810 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.778+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974a651b0 con 0x558974b79e50 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.778+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55897494c650 con 0x558974b794f0 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.778+0000 7fb6f1bd4b80 10 client.0 _ll_register_callbacks cb 0x558974a6eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6df7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b8a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6df7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b8a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46012/0 (socket says 172.21.15.39:46012) 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6df7fe700 1 -- 172.21.15.39:0/915871374 learned_addr learned my addr 172.21.15.39:0/915871374 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6de7fc700 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b79e50 0x558974b832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6deffd700 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 0x558974b82db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6df7fe700 1 -- 172.21.15.39:0/915871374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 msgr2=0x558974b82db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 0x558974b82db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6df7fe700 1 -- 172.21.15.39:0/915871374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b79e50 msgr2=0x558974b832f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b79e50 0x558974b832f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6df7fe700 1 -- 172.21.15.39:0/915871374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c8010c50 con 0x558974b7a810 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6deffd700 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 0x558974b82db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b8a330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8010a40 tx=0x7fb6d8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db16205026f1f9fd server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6beffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6beffd700 1 -- 172.21.15.39:0/915871374 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800ea60 con 0x558974b7a810 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6beffd700 1 -- 172.21.15.39:0/915871374 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb6d8010e70 con 0x558974b7a810 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/915871374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558974a22220 con 0x558974b7a810 2024-05-24T11:37:59.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6beffd700 1 -- 172.21.15.39:0/915871374 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6d800ebc0 con 0x558974b7a810 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6f1bd4b80 10 client.15309 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/915871374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5589748f6b10 con 0x558974b7a810 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.779+0000 7fb6beffd700 1 -- 172.21.15.39:0/915871374 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558974a22220 con 0x558974b7a810 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6beffd700 1 -- 172.21.15.39:0/915871374 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6d8021430 con 0x558974b7a810 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6f1bd4b80 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6f1bd4b80 10 client.15309 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6f1bd4b80 20 client.15309 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x558974b8d4a0 0x558974b932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6f1bd4b80 4 client.15309 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/915871374 --> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558974b487c0 con 0x558974b8d4a0 2024-05-24T11:37:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6deffd700 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x558974b8d4a0 0x558974b932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6beffd700 10 client.15309 ms_handle_connect on v2:172.21.15.137:6838/218688120 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.780+0000 7fb6deffd700 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x558974b8d4a0 0x558974b932a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63c9a484877b0a67 server_cookie=98d7a4ff07c69708 in_seq=0 out_seq=0 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6beffd700 1 -- 172.21.15.39:0/915871374 <== mds.1 v2:172.21.15.137:6838/218688120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558974b487c0 con 0x558974b8d4a0 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6beffd700 10 client.15309 handle_command_reply: tid=0 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 client.15309 shutdown 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/915871374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 msgr2=0x558974b8a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b8a330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8010a40 tx=0x7fb6d8010d50 comp rx=0 tx=0).stop 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/915871374 shutdown_connections 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x558974b8d4a0 0x558974b932a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b794f0 0x558974b82db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b8a330 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6beffd700 0 client.15309 ms_handle_reset on v2:172.21.15.137:6838/218688120 2024-05-24T11:37:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/915871374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b79e50 0x558974b832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/915871374 >> 172.21.15.39:0/915871374 conn(0x558974ad71c0 msgr2=0x558974b73550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.781+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/915871374 shutdown_connections 2024-05-24T11:37:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.782+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/915871374 wait complete. 2024-05-24T11:37:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.782+0000 7fb6f1bd4b80 20 client.15309 trim_cache size 0 max 16384 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6f1bd4b80 1 Processor -- start 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6f1bd4b80 1 -- start start 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b794f0 0x558974b833f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 0x558974b798d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 0x558974b8a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974a22220 con 0x558974b794f0 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974a872b0 con 0x558974b7a810 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974ac2150 con 0x558974b79e50 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6deffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b794f0 0x558974b833f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6deffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b794f0 0x558974b833f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46022/0 (socket says 172.21.15.39:46022) 2024-05-24T11:37:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6deffd700 1 -- 172.21.15.39:0/181300366 learned_addr learned my addr 172.21.15.39:0/181300366 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6de7fc700 1 --2- 172.21.15.39:0/181300366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 0x558974b798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/181300366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 0x558974b8a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6deffd700 1 -- 172.21.15.39:0/181300366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 msgr2=0x558974b798d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6deffd700 1 --2- 172.21.15.39:0/181300366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 0x558974b798d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6deffd700 1 -- 172.21.15.39:0/181300366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 msgr2=0x558974b8a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6deffd700 1 --2- 172.21.15.39:0/181300366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 0x558974b8a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6deffd700 1 -- 172.21.15.39:0/181300366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558974a8f510 con 0x558974b794f0 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6deffd700 1 --2- 172.21.15.39:0/181300366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b794f0 0x558974b833f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009fd0 tx=0x7fb6d0003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a6f373ded92bdd server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.800+0000 7fb6d67fc700 1 -- 172.21.15.39:0/181300366 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d009efa0 con 0x558974b794f0 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6d67fc700 1 -- 172.21.15.39:0/181300366 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6d00a3560 con 0x558974b794f0 2024-05-24T11:37:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6d67fc700 1 -- 172.21.15.39:0/181300366 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d009b3d0 con 0x558974b794f0 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/181300366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b794f0 msgr2=0x558974b833f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/181300366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b794f0 0x558974b833f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009fd0 tx=0x7fb6d0003740 comp rx=0 tx=0).stop 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/181300366 shutdown_connections 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/181300366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b7a810 0x558974b8a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/181300366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b794f0 0x558974b833f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/181300366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 0x558974b798d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/181300366 >> 172.21.15.39:0/181300366 conn(0x558974ad71c0 msgr2=0x558974b7abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/181300366 shutdown_connections 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/181300366 wait complete. 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.801+0000 7fb6f1bd4b80 1 -- start start 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6f1bd4b80 1 Processor -- start 2024-05-24T11:37:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b794f0 0x558974b84a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 0x558974b84fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6f1bd4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b89e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974b8a380 con 0x558974b7a810 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974b8a4f0 con 0x558974b794f0 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6f1bd4b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558974b8a6b0 con 0x558974b79e50 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6f1bd4b80 10 client.0 _ll_register_callbacks cb 0x558974a6eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6df7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b89e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6df7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b89e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46036/0 (socket says 172.21.15.39:46036) 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6df7fe700 1 -- 172.21.15.39:0/1380055154 learned_addr learned my addr 172.21.15.39:0/1380055154 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6deffd700 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b794f0 0x558974b84a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6de7fc700 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 0x558974b84fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6df7fe700 1 -- 172.21.15.39:0/1380055154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 msgr2=0x558974b84fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 0x558974b84fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6df7fe700 1 -- 172.21.15.39:0/1380055154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b794f0 msgr2=0x558974b84a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b794f0 0x558974b84a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.802+0000 7fb6df7fe700 1 -- 172.21.15.39:0/1380055154 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e019f190 con 0x558974b7a810 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6df7fe700 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b89e40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80217e0 tx=0x7fb6d800e580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614d7abcc6359c0a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6beffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6beffd700 1 -- 172.21.15.39:0/1380055154 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8003e90 con 0x558974b7a810 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6beffd700 1 -- 172.21.15.39:0/1380055154 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb6d8021a00 con 0x558974b7a810 2024-05-24T11:37:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/1380055154 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6c8015070 con 0x558974b7a810 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6beffd700 1 -- 172.21.15.39:0/1380055154 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6d8021c00 con 0x558974b7a810 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6f1bd4b80 10 client.15315 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/1380055154 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55897494ec80 con 0x558974b7a810 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6beffd700 1 -- 172.21.15.39:0/1380055154 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6c8015070 con 0x558974b7a810 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6beffd700 1 -- 172.21.15.39:0/1380055154 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6d8016a10 con 0x558974b7a810 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6f1bd4b80 10 client.15315 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6f1bd4b80 10 client.15315 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6f1bd4b80 20 client.15315 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x558974be9a10 0x558974be63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6f1bd4b80 4 client.15315 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/1380055154 --> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558974ac1e70 con 0x558974be9a10 2024-05-24T11:37:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.803+0000 7fb6deffd700 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x558974be9a10 0x558974be63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6deffd700 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x558974be9a10 0x558974be63a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97bc18049caba5fc server_cookie=c9ef1b8dba3abf06 in_seq=0 out_seq=0 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6beffd700 10 client.15315 ms_handle_connect on v2:172.21.15.137:6838/218688120 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6beffd700 1 -- 172.21.15.39:0/1380055154 <== mds.1 v2:172.21.15.137:6838/218688120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558974ac1e70 con 0x558974be9a10 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6beffd700 10 client.15315 handle_command_reply: tid=0 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 client.15315 shutdown 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/1380055154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 msgr2=0x558974b89e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b89e40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80217e0 tx=0x7fb6d800e580 comp rx=0 tx=0).stop 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/1380055154 shutdown_connections 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.137:6838/218688120,v1:172.21.15.137:6839/218688120] conn(0x558974be9a10 0x558974be63a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558974b7a810 0x558974b89e40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558974b794f0 0x558974b84a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6beffd700 0 client.15315 ms_handle_reset on v2:172.21.15.137:6838/218688120 2024-05-24T11:37:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 --2- 172.21.15.39:0/1380055154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558974b79e50 0x558974b84fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/1380055154 >> 172.21.15.39:0/1380055154 conn(0x558974ad71c0 msgr2=0x558974b80c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/1380055154 shutdown_connections 2024-05-24T11:37:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.804+0000 7fb6f1bd4b80 1 -- 172.21.15.39:0/1380055154 wait complete. 2024-05-24T11:37:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6f1bd4b80 20 client.15315 trim_cache size 0 max 16384 2024-05-24T11:37:59.818 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:59.818 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24511, 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 68.215317674999994, 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 82.570188286000004 2024-05-24T11:37:59.819 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:59.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb6cc0a11b0 msgr2=0x7fb6cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb6cc0a11b0 0x7fb6cc0a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8005950 tx=0x7fb6c800b410 comp rx=0 tx=0).stop 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 msgr2=0x7fb6e0196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 0x7fb6e0196da0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009f50 tx=0x7fb6d0003a50 comp rx=0 tx=0).stop 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3538592754 shutdown_connections 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7fb6cc0a11b0 0x7fb6cc0a3670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6e0102b10 0x7fb6e019e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e0100f80 0x7fb6e0196da0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 --2- 172.21.15.39:0/3538592754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6e0102170 0x7fb6e01972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3538592754 >> 172.21.15.39:0/3538592754 conn(0x7fb6e00fc4f0 msgr2=0x7fb6e0105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3538592754 shutdown_connections 2024-05-24T11:37:59.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.805+0000 7fb6e5a19700 1 -- 172.21.15.39:0/3538592754 wait complete. 2024-05-24T11:37:59.823 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 -- 172.21.15.39:0/2625687382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24101a10 msgr2=0x7efe24103e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 --2- 172.21.15.39:0/2625687382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24101a10 0x7efe24103e00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009990 tx=0x7efe0c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 -- 172.21.15.39:0/2625687382 shutdown_connections 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 --2- 172.21.15.39:0/2625687382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe24106c70 0x7efe241090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 --2- 172.21.15.39:0/2625687382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24101a10 0x7efe24103e00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 --2- 172.21.15.39:0/2625687382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24104340 0x7efe24106730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 -- 172.21.15.39:0/2625687382 >> 172.21.15.39:0/2625687382 conn(0x7efe240fb400 msgr2=0x7efe240fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 -- 172.21.15.39:0/2625687382 shutdown_connections 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 -- 172.21.15.39:0/2625687382 wait complete. 2024-05-24T11:37:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 Processor -- start 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.946+0000 7efe293eb700 1 -- start start 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe293eb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24101a10 0x7efe24196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe293eb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24104340 0x7efe24197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe293eb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe24106c70 0x7efe2419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe293eb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419e970 con 0x7efe24104340 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe293eb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419eab0 con 0x7efe24101a10 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe293eb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419ec40 con 0x7efe24106c70 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe227fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24104340 0x7efe24197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe227fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24104340 0x7efe24197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46052/0 (socket says 172.21.15.39:46052) 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe227fc700 1 -- 172.21.15.39:0/1558882769 learned_addr learned my addr 172.21.15.39:0/1558882769 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe22ffd700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24101a10 0x7efe24196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe237fe700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe24106c70 0x7efe2419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe227fc700 1 -- 172.21.15.39:0/1558882769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe24106c70 msgr2=0x7efe2419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe227fc700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe24106c70 0x7efe2419e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe227fc700 1 -- 172.21.15.39:0/1558882769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24101a10 msgr2=0x7efe24196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe227fc700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24101a10 0x7efe24196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe227fc700 1 -- 172.21.15.39:0/1558882769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0095f0 con 0x7efe24104340 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe227fc700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24104340 0x7efe24197380 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d950 tx=0x7efe1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe1bfff700 1 -- 172.21.15.39:0/1558882769 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1400f890 con 0x7efe24104340 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe1bfff700 1 -- 172.21.15.39:0/1558882769 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe1400fe80 con 0x7efe24104340 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe293eb700 1 -- 172.21.15.39:0/1558882769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2419ef20 con 0x7efe24104340 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe1bfff700 1 -- 172.21.15.39:0/1558882769 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1400e3b0 con 0x7efe24104340 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.947+0000 7efe293eb700 1 -- 172.21.15.39:0/1558882769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2419f3a0 con 0x7efe24104340 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.949+0000 7efe1bfff700 1 -- 172.21.15.39:0/1558882769 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7efe14010460 con 0x7efe24104340 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.950+0000 7efe1bfff700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efe100a11b0 0x7efe100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.950+0000 7efe22ffd700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efe100a11b0 0x7efe100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.950+0000 7efe1bfff700 1 -- 172.21.15.39:0/1558882769 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7efe140cc700 con 0x7efe24104340 2024-05-24T11:37:59.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.951+0000 7efe22ffd700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efe100a11b0 0x7efe100a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe0c003780 tx=0x7efe0c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.951+0000 7efe355a6b80 1 Processor -- start 2024-05-24T11:37:59.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.951+0000 7efe355a6b80 1 -- start start 2024-05-24T11:37:59.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.951+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 0x564a387ff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.951+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ffe50 0x564a388002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.951+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 0x564a38800c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.951+0000 7efe355a6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a38756100 con 0x564a387ffe50 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.951+0000 7efe355a6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a38753ac0 con 0x564a387ff4f0 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.951+0000 7efe355a6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3874f3b0 con 0x564a38800810 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe227fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ffe50 0x564a388002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe227fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ffe50 0x564a388002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46058/0 (socket says 172.21.15.39:46058) 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe227fc700 1 -- 172.21.15.39:0/315077047 learned_addr learned my addr 172.21.15.39:0/315077047 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe22ffd700 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 0x564a387ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe237fe700 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 0x564a38800c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe227fc700 1 -- 172.21.15.39:0/315077047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 msgr2=0x564a38800c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe227fc700 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 0x564a38800c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe227fc700 1 -- 172.21.15.39:0/315077047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 msgr2=0x564a387ff910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe227fc700 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 0x564a387ff910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe227fc700 1 -- 172.21.15.39:0/315077047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a385d0360 con 0x564a387ffe50 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe237fe700 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 0x564a38800c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe22ffd700 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 0x564a387ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe227fc700 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ffe50 0x564a388002d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efe140a1a40 tx=0x7efe14099760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed719473c20bcb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe197fa700 1 -- 172.21.15.39:0/315077047 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14099af0 con 0x564a387ffe50 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe197fa700 1 -- 172.21.15.39:0/315077047 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe1409d920 con 0x564a387ffe50 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.952+0000 7efe197fa700 1 -- 172.21.15.39:0/315077047 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140a1d40 con 0x564a387ffe50 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 -- 172.21.15.39:0/315077047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ffe50 msgr2=0x564a388002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ffe50 0x564a388002d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efe140a1a40 tx=0x7efe14099760 comp rx=0 tx=0).stop 2024-05-24T11:37:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 -- 172.21.15.39:0/315077047 shutdown_connections 2024-05-24T11:37:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 0x564a38800c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 0x564a387ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 --2- 172.21.15.39:0/315077047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ffe50 0x564a388002d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 -- 172.21.15.39:0/315077047 >> 172.21.15.39:0/315077047 conn(0x564a3875d1c0 msgr2=0x564a387f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 -- 172.21.15.39:0/315077047 shutdown_connections 2024-05-24T11:37:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 -- 172.21.15.39:0/315077047 wait complete. 2024-05-24T11:37:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 -- start start 2024-05-24T11:37:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.953+0000 7efe355a6b80 1 Processor -- start 2024-05-24T11:37:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a387ff4f0 0x564a38808e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 0x564a38809380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a388103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe355a6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a386eb510 con 0x564a38800810 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe355a6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a386eb1b0 con 0x564a387ffe50 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe355a6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a385d2650 con 0x564a387ff4f0 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe355a6b80 10 client.0 _ll_register_callbacks cb 0x564a386f4ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a388103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a388103c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46064/0 (socket says 172.21.15.39:46064) 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe237fe700 1 -- 172.21.15.39:0/2659310646 learned_addr learned my addr 172.21.15.39:0/2659310646 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe22ffd700 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a387ff4f0 0x564a38808e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe227fc700 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 0x564a38809380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe237fe700 1 -- 172.21.15.39:0/2659310646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a387ff4f0 msgr2=0x564a38808e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe237fe700 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a387ff4f0 0x564a38808e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe237fe700 1 -- 172.21.15.39:0/2659310646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 msgr2=0x564a38809380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe237fe700 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 0x564a38809380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe237fe700 1 -- 172.21.15.39:0/2659310646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2419f3a0 con 0x564a38800810 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe227fc700 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 0x564a38809380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe237fe700 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a388103c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efe1c004040 tx=0x7efe1c004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a3e7794ed57f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe0a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2659310646 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c004bb0 con 0x564a38800810 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2659310646 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7efe1c004d10 con 0x564a38800810 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.954+0000 7efe355a6b80 1 -- 172.21.15.39:0/2659310646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a386a8220 con 0x564a38800810 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2659310646 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1c007710 con 0x564a38800810 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe355a6b80 10 client.15327 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe355a6b80 1 -- 172.21.15.39:0/2659310646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564a3857cb10 con 0x564a38800810 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2659310646 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe1c024020 con 0x564a38800810 2024-05-24T11:37:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2659310646 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efe1c014070 con 0x564a38800810 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe355a6b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe355a6b80 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe355a6b80 20 client.15327 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564a38813580 0x564a388193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe355a6b80 4 client.15327 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe355a6b80 1 -- 172.21.15.39:0/2659310646 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a387ce7c0 con 0x564a38813580 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.955+0000 7efe22ffd700 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564a38813580 0x564a388193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.956+0000 7efe0a7fc700 10 client.15327 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.956+0000 7efe22ffd700 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564a38813580 0x564a388193d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c39aaac4301b01c9 server_cookie=ba7ad6f1c382c46d in_seq=0 out_seq=0 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.956+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2659310646 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a387ce7c0 con 0x564a38813580 2024-05-24T11:37:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.956+0000 7efe0a7fc700 10 client.15327 handle_command_reply: tid=0 2024-05-24T11:37:59.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.956+0000 7efe355a6b80 1 client.15327 shutdown 2024-05-24T11:37:59.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 -- 172.21.15.39:0/2659310646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 msgr2=0x564a388103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a388103c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efe1c004040 tx=0x7efe1c004350 comp rx=0 tx=0).stop 2024-05-24T11:37:59.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 -- 172.21.15.39:0/2659310646 shutdown_connections 2024-05-24T11:37:59.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a387ff4f0 0x564a38808e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564a38813580 0x564a388193d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a388103c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2659310646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 0x564a38809380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe0a7fc700 0 client.15327 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:37:59.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 -- 172.21.15.39:0/2659310646 >> 172.21.15.39:0/2659310646 conn(0x564a3875d1c0 msgr2=0x564a387f9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 -- 172.21.15.39:0/2659310646 shutdown_connections 2024-05-24T11:37:59.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 1 -- 172.21.15.39:0/2659310646 wait complete. 2024-05-24T11:37:59.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.957+0000 7efe355a6b80 20 client.15327 trim_cache size 0 max 16384 2024-05-24T11:37:59.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.975+0000 7efe355a6b80 1 Processor -- start 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.975+0000 7efe355a6b80 1 -- start start 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ff4f0 0x564a388111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 0x564a387ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 0x564a38810a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe355a6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a387ce7f0 con 0x564a387ff4f0 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe355a6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a38748180 con 0x564a387ffe50 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe355a6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a386ee5a0 con 0x564a38800810 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe22ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ff4f0 0x564a388111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe22ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ff4f0 0x564a388111a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46078/0 (socket says 172.21.15.39:46078) 2024-05-24T11:37:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe22ffd700 1 -- 172.21.15.39:0/4099626685 learned_addr learned my addr 172.21.15.39:0/4099626685 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe22ffd700 1 -- 172.21.15.39:0/4099626685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 msgr2=0x564a38810a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe237fe700 1 --2- 172.21.15.39:0/4099626685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 0x564a38810a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe227fc700 1 --2- 172.21.15.39:0/4099626685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 0x564a387ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe22ffd700 1 --2- 172.21.15.39:0/4099626685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 0x564a38810a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe22ffd700 1 -- 172.21.15.39:0/4099626685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 msgr2=0x564a387ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe22ffd700 1 --2- 172.21.15.39:0/4099626685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 0x564a387ff8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe22ffd700 1 -- 172.21.15.39:0/4099626685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a38715510 con 0x564a387ff4f0 2024-05-24T11:37:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe22ffd700 1 --2- 172.21.15.39:0/4099626685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ff4f0 0x564a388111a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000c00 tx=0x7efe0c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcee5f41d7d1401 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe19ffb700 1 -- 172.21.15.39:0/4099626685 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0056e0 con 0x564a387ff4f0 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe19ffb700 1 -- 172.21.15.39:0/4099626685 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe0c02c040 con 0x564a387ff4f0 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.976+0000 7efe19ffb700 1 -- 172.21.15.39:0/4099626685 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0048c0 con 0x564a387ff4f0 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 -- 172.21.15.39:0/4099626685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ff4f0 msgr2=0x564a388111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 --2- 172.21.15.39:0/4099626685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ff4f0 0x564a388111a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000c00 tx=0x7efe0c023000 comp rx=0 tx=0).stop 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 -- 172.21.15.39:0/4099626685 shutdown_connections 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 --2- 172.21.15.39:0/4099626685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a38800810 0x564a38810a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 --2- 172.21.15.39:0/4099626685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a387ff4f0 0x564a388111a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 --2- 172.21.15.39:0/4099626685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ffe50 0x564a387ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 -- 172.21.15.39:0/4099626685 >> 172.21.15.39:0/4099626685 conn(0x564a3875d1c0 msgr2=0x564a387f61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 -- 172.21.15.39:0/4099626685 shutdown_connections 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 -- 172.21.15.39:0/4099626685 wait complete. 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.977+0000 7efe355a6b80 1 -- start start 2024-05-24T11:37:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe355a6b80 1 Processor -- start 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 0x564a3880a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a387ffe50 0x564a3880aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe355a6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a3880f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe355a6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3880fd40 con 0x564a38800810 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe355a6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3880feb0 con 0x564a387ff4f0 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe355a6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a38810070 con 0x564a387ffe50 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe355a6b80 10 client.0 _ll_register_callbacks cb 0x7efe1c0098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a3880f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a3880f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46090/0 (socket says 172.21.15.39:46090) 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe237fe700 1 -- 172.21.15.39:0/2526213238 learned_addr learned my addr 172.21.15.39:0/2526213238 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe22ffd700 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 0x564a3880a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe237fe700 1 -- 172.21.15.39:0/2526213238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a387ffe50 msgr2=0x564a3880aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe237fe700 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a387ffe50 0x564a3880aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe237fe700 1 -- 172.21.15.39:0/2526213238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 msgr2=0x564a3880a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:37:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe237fe700 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 0x564a3880a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe237fe700 1 -- 172.21.15.39:0/2526213238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe24109cd0 con 0x564a38800810 2024-05-24T11:37:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe237fe700 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a3880f790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009fd0 tx=0x7efe1c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573a0af96a6638b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:37:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.978+0000 7efe0a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:37:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.979+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2526213238 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c007710 con 0x564a38800810 2024-05-24T11:37:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.979+0000 7efe355a6b80 1 -- 172.21.15.39:0/2526213238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe1409d920 con 0x564a38800810 2024-05-24T11:37:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.979+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2526213238 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7efe1c025080 con 0x564a38800810 2024-05-24T11:37:59.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.986+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2526213238 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1c007870 con 0x564a38800810 2024-05-24T11:37:59.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.986+0000 7efe355a6b80 10 client.15333 fetch_fsmap learned FSMap version 27 2024-05-24T11:37:59.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.986+0000 7efe355a6b80 1 -- 172.21.15.39:0/2526213238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564a385d4c80 con 0x564a38800810 2024-05-24T11:37:59.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.987+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2526213238 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe1409d920 con 0x564a38800810 2024-05-24T11:37:59.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.987+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2526213238 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efe1c014070 con 0x564a38800810 2024-05-24T11:37:59.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.987+0000 7efe355a6b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:37:59.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.987+0000 7efe355a6b80 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-24T11:37:59.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.987+0000 7efe355a6b80 20 client.15333 populate_metadata read hostname 'smithi039' 2024-05-24T11:37:59.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.987+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564a3886f650 0x564a3886bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:37:59.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.987+0000 7efe355a6b80 4 client.15333 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-24T11:37:59.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.987+0000 7efe355a6b80 1 -- 172.21.15.39:0/2526213238 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564a38747e70 con 0x564a3886f650 2024-05-24T11:37:59.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.987+0000 7efe22ffd700 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564a3886f650 0x564a3886bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:37:59.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe0a7fc700 10 client.15333 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:37:59.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe22ffd700 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564a3886f650 0x564a3886bf70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e802430a391532b server_cookie=114f372f8a2395a3 in_seq=0 out_seq=0 2024-05-24T11:37:59.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe0a7fc700 1 -- 172.21.15.39:0/2526213238 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564a38747e70 con 0x564a3886f650 2024-05-24T11:37:59.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe0a7fc700 10 client.15333 handle_command_reply: tid=0 2024-05-24T11:37:59.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 client.15333 shutdown 2024-05-24T11:37:59.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 -- 172.21.15.39:0/2526213238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 msgr2=0x564a3880f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a3880f790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009fd0 tx=0x7efe1c009540 comp rx=0 tx=0).stop 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 -- 172.21.15.39:0/2526213238 shutdown_connections 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564a3886f650 0x564a3886bf70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564a38800810 0x564a3880f790 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a387ff4f0 0x564a3880a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 --2- 172.21.15.39:0/2526213238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a387ffe50 0x564a3880aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe0a7fc700 0 client.15333 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 -- 172.21.15.39:0/2526213238 >> 172.21.15.39:0/2526213238 conn(0x564a3875d1c0 msgr2=0x564a38800bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 -- 172.21.15.39:0/2526213238 shutdown_connections 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 1 -- 172.21.15.39:0/2526213238 wait complete. 2024-05-24T11:37:59.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.988+0000 7efe355a6b80 20 client.15333 trim_cache size 0 max 16384 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "f68008c4-19c0-11ef-bc9a-c7b262605968", 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24542, 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 71.485681479999997, 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 79, 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 77, 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 77.118539108999997 2024-05-24T11:37:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:37:59.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 -- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efe100a11b0 msgr2=0x7efe100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efe100a11b0 0x7efe100a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe0c003780 tx=0x7efe0c005fb0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 -- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24104340 msgr2=0x7efe24197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:37:59.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24104340 0x7efe24197380 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d950 tx=0x7efe1400dc60 comp rx=0 tx=0).stop 2024-05-24T11:37:59.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 -- 172.21.15.39:0/1558882769 shutdown_connections 2024-05-24T11:37:59.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7efe100a11b0 0x7efe100a3670 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe24106c70 0x7efe2419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24101a10 0x7efe24196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 --2- 172.21.15.39:0/1558882769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe24104340 0x7efe24197380 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:37:59.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 -- 172.21.15.39:0/1558882769 >> 172.21.15.39:0/1558882769 conn(0x7efe240fb400 msgr2=0x7efe24103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:37:59.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 -- 172.21.15.39:0/1558882769 shutdown_connections 2024-05-24T11:37:59.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:37:59.989+0000 7efe293eb700 1 -- 172.21.15.39:0/1558882769 wait complete. 2024-05-24T11:38:00.007 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-24T11:38:00.033 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-24T11:38:00.033 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-24T11:38:00.034 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-24T11:38:00.034 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-24T11:38:00.034 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-24T11:38:00.034 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-24T11:38:00.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:00.034 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-05-24T11:38:00.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:00.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-05-24T11:38:00.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:37:59 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3440528518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:38:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2024-05-24T11:38:00.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T11:38:00.109 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-24T11:38:00.110 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-24T11:38:00.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:37:59 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3440528518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:38:00.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:37:59 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3440528518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:38:00.378 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:38:01.238 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:00 smithi039 ceph-mon[42919]: pgmap v258: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:38:01.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.418+0000 7f0fa9868700 1 -- 172.21.15.39:0/2021337780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4100ff0 msgr2=0x7f0fa4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:01.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.418+0000 7f0fa9868700 1 --2- 172.21.15.39:0/2021337780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4100ff0 0x7f0fa4101450 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009990 tx=0x7f0f8c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.418+0000 7f0fa9868700 1 -- 172.21.15.39:0/2021337780 shutdown_connections 2024-05-24T11:38:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.418+0000 7f0fa9868700 1 --2- 172.21.15.39:0/2021337780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fa4107030 0x7f0fa4107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.418+0000 7f0fa9868700 1 --2- 172.21.15.39:0/2021337780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fa4101990 0x7f0fa410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.418+0000 7f0fa9868700 1 --2- 172.21.15.39:0/2021337780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4100ff0 0x7f0fa4101450 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.418+0000 7f0fa9868700 1 -- 172.21.15.39:0/2021337780 >> 172.21.15.39:0/2021337780 conn(0x7f0fa4078190 msgr2=0x7f0fa40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.418+0000 7f0fa9868700 1 -- 172.21.15.39:0/2021337780 shutdown_connections 2024-05-24T11:38:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.418+0000 7f0fa9868700 1 -- 172.21.15.39:0/2021337780 wait complete. 2024-05-24T11:38:01.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.419+0000 7f0fa9868700 1 Processor -- start 2024-05-24T11:38:01.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.419+0000 7f0fa9868700 1 -- start start 2024-05-24T11:38:01.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.419+0000 7f0fa9868700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fa4100ff0 0x7f0fa41973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.419+0000 7f0fa9868700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4101990 0x7f0fa4197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa9868700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fa4107030 0x7f0fa419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa9868700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa419c2d0 con 0x7f0fa4101990 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa9868700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa419c440 con 0x7f0fa4107030 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa9868700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa419c600 con 0x7f0fa4100ff0 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4101990 0x7f0fa4197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4101990 0x7f0fa4197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46094/0 (socket says 172.21.15.39:46094) 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa27fc700 1 -- 172.21.15.39:0/4173459841 learned_addr learned my addr 172.21.15.39:0/4173459841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa2ffd700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fa4100ff0 0x7f0fa41973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa37fe700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fa4107030 0x7f0fa419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa27fc700 1 -- 172.21.15.39:0/4173459841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fa4100ff0 msgr2=0x7f0fa41973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:01.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa27fc700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fa4100ff0 0x7f0fa41973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa27fc700 1 -- 172.21.15.39:0/4173459841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fa4107030 msgr2=0x7f0fa419bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa27fc700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fa4107030 0x7f0fa419bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa27fc700 1 -- 172.21.15.39:0/4173459841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0095f0 con 0x7f0fa4101990 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa37fe700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fa4107030 0x7f0fa419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa27fc700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4101990 0x7f0fa4197910 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c006010 tx=0x7f0f8c00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa8866700 1 -- 172.21.15.39:0/4173459841 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c01d070 con 0x7f0fa4101990 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa8866700 1 -- 172.21.15.39:0/4173459841 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f8c00fde0 con 0x7f0fa4101990 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa419c880 con 0x7f0fa4101990 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa8866700 1 -- 172.21.15.39:0/4173459841 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0173f0 con 0x7f0fa4101990 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.420+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa41a4140 con 0x7f0fa4101990 2024-05-24T11:38:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.421+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa4109550 con 0x7f0fa4101990 2024-05-24T11:38:01.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.424+0000 7f0fa8866700 1 -- 172.21.15.39:0/4173459841 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0f8c022670 con 0x7f0fa4101990 2024-05-24T11:38:01.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.424+0000 7f0fa8866700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0f900a1230 0x7f0f900a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:01.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.424+0000 7f0fa2ffd700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0f900a1230 0x7f0f900a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:01.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.424+0000 7f0fa8866700 1 -- 172.21.15.39:0/4173459841 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0f8c0ce240 con 0x7f0fa4101990 2024-05-24T11:38:01.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.424+0000 7f0fa2ffd700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0f900a1230 0x7f0f900a36f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009f20 tx=0x7f0f940094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:01.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.427+0000 7f0fa8866700 1 -- 172.21.15.39:0/4173459841 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0f8c097a00 con 0x7f0fa4101990 2024-05-24T11:38:01.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:00 smithi053 ceph-mon[47448]: pgmap v258: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:38:01.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:00 smithi137 ceph-mon[47907]: pgmap v258: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-24T11:38:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.674+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0fa41986e0 con 0x7f0fa4101990 2024-05-24T11:38:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.674+0000 7f0fa8866700 1 -- 172.21.15.39:0/4173459841 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f8c09b020 con 0x7f0fa4101990 2024-05-24T11:38:01.680 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-24T11:38:01.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0f900a1230 msgr2=0x7f0f900a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:01.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0f900a1230 0x7f0f900a36f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009f20 tx=0x7f0f940094f0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4101990 msgr2=0x7f0fa4197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:01.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4101990 0x7f0fa4197910 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c006010 tx=0x7f0f8c00f460 comp rx=0 tx=0).stop 2024-05-24T11:38:01.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 shutdown_connections 2024-05-24T11:38:01.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f0f900a1230 0x7f0f900a36f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fa4100ff0 0x7f0fa41973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fa4107030 0x7f0fa419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 --2- 172.21.15.39:0/4173459841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa4101990 0x7f0fa4197910 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:01.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 >> 172.21.15.39:0/4173459841 conn(0x7f0fa4078190 msgr2=0x7f0fa40fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:01.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 shutdown_connections 2024-05-24T11:38:01.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:01.676+0000 7f0fa9868700 1 -- 172.21.15.39:0/4173459841 wait complete. 2024-05-24T11:38:02.068 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-24T11:38:02.068 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-24T11:38:02.069 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-05-24T11:38:02.069 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-24T11:38:02.069 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-24T11:38:02.069 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-24T11:38:02.069 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-24T11:38:02.069 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-24T11:38:02.069 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-24T11:38:02.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:02.069 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-05-24T11:38:02.088 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-24T11:38:02.088 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-24T11:38:02.088 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-05-24T11:38:02.089 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-05-24T11:38:02.090 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-24T11:38:02.090 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41796sec preferred_lft 41796sec 2024-05-24T11:38:02.090 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-05-24T11:38:02.090 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-05-24T11:38:02.090 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-24T11:38:02.090 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-05-24T11:38:02.090 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-05-24T11:38:02.090 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-24T11:38:02.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:02.091 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-05-24T11:38:02.091 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-24T11:38:02.091 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2024-05-24T11:38:02.091 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2024-05-24T11:38:02.091 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2024-05-24T11:38:02.091 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-24T11:38:02.091 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-24T11:38:02.244 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:02 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4173459841' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-24T11:38:02.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:02.271 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-24T11:38:02.301 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-05-24T11:38:02.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:02.302 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-05-24T11:38:02.378 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-05-24T11:38:02.378 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-24T11:38:02.378 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-24T11:38:02.378 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-24T11:38:02.378 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-24T11:38:02.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:02.379 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-05-24T11:38:02.379 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-24T11:38:02.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-24T11:38:02.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-24T11:38:02.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-24T11:38:02.379 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-24T11:38:02.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:02 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4173459841' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-24T11:38:02.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:02 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4173459841' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-24T11:38:02.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:02.658 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-05-24T11:38:02.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:02.677 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-05-24T11:38:02.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:02.735 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-05-24T11:38:02.735 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-24T11:38:02.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-24T11:38:02.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-24T11:38:02.735 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-24T11:38:02.833 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-24T11:38:02.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:02.833 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-05-24T11:38:02.834 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-24T11:38:02.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-24T11:38:02.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-24T11:38:02.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-24T11:38:02.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-24T11:38:02.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-24T11:38:02.834 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-24T11:38:03.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:03.033 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-05-24T11:38:03.033 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-24T11:38:03.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-05-24T11:38:03.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2024-05-24T11:38:03.033 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-24T11:38:03.073 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:03 smithi039 ceph-mon[42919]: pgmap v259: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-24T11:38:03.133 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-24T11:38:03.133 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-24T11:38:03.133 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /tmp/cephfs 2024-05-24T11:38:03.191 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-24T11:38:03.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-24T11:38:03.192 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /tmp/cephfs 2024-05-24T11:38:03.248 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-05-24T11:38:03.316 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-24T11:38:03.317 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-24T11:38:03.384 INFO:teuthology.orchestra.run.smithi039.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-24T11:38:03.385 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-24T11:38:03.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:03.385 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-05-24T11:38:03.443 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-24T11:38:03.443 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/_nogroup/sv_0/650282f3-ff17-4113-9a25-acb8b2736148 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-05-24T11:38:03.486 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-05-24T11:38:03.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:03 smithi053 ceph-mon[47448]: pgmap v259: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-24T11:38:03.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:03 smithi137 ceph-mon[47907]: pgmap v259: 97 pgs: 97 active+clean; 639 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-24T11:38:03.515 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-24T11:38:03.515 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-24T11:38:03.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:2024-05-24T11:38:03.575+0000 7fe1fe9c95c0 -1 init, newargv = 0x55dca8c1ec50 newargc=17 2024-05-24T11:38:03.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:ceph-fuse[107557]: starting ceph client 2024-05-24T11:38:03.584 INFO:teuthology.orchestra.run.smithi039.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-24T11:38:03.585 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-24T11:38:03.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:03.586 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-05-24T11:38:03.608 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:ceph-fuse[107557]: starting fuse 2024-05-24T11:38:03.625 INFO:teuthology.orchestra.run.smithi039.stdout:124 2024-05-24T11:38:03.625 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> import glob 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> import re 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> import os 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> import subprocess 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> def _find_admin_socket(client_name): 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> files = glob.glob(asok_path) 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> mountpoint = "/tmp/cephfs" 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> 2024-05-24T11:38:03.626 DEBUG:teuthology.orchestra.run.smithi039:> # Given a non-glob path, it better be there 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> if "*" not in asok_path: 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> assert(len(files) == 1) 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> return files[0] 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> for f in files: 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> contents = proc_f.read() 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> if mountpoint in contents: 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> return f 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> print(_find_admin_socket("client.0")) 2024-05-24T11:38:03.627 DEBUG:teuthology.orchestra.run.smithi039:> ' 2024-05-24T11:38:03.753 INFO:teuthology.orchestra.run.smithi039.stdout:/var/run/ceph/ceph-client.0.107557.asok 2024-05-24T11:38:03.935 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107557.asok 2024-05-24T11:38:03.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:03.936 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.107557.asok status 2024-05-24T11:38:04.099 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:38:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "metadata": { 2024-05-24T11:38:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_sha1": "b9b067bc6d30ec00e391f0fa43f50affe981ba2f", 2024-05-24T11:38:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 18.2.2-1349-gb9b067bc (b9b067bc6d30ec00e391f0fa43f50affe981ba2f) reef (stable)", 2024-05-24T11:38:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "entity_id": "0", 2024-05-24T11:38:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": "smithi039", 2024-05-24T11:38:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "mount_point": "/tmp/cephfs", 2024-05-24T11:38:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "pid": "107557", 2024-05-24T11:38:04.099 INFO:teuthology.orchestra.run.smithi039.stdout: "root": "/volumes/_nogroup/sv_0/650282f3-ff17-4113-9a25-acb8b2736148" 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_count": 0, 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_pinned_count": 0, 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 15345, 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "inst": { 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": { 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "client", 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "num": 15345 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1868022804 2024-05-24T11:38:04.100 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1868022804 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "inst_str": "client.15345 192.168.0.1:0/1868022804", 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "addr_str": "192.168.0.1:0/1868022804", 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "inode_count": 5, 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_epoch": 27, 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch": 79, 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch_barrier": 77, 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklisted": false, 2024-05-24T11:38:04.101 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs" 2024-05-24T11:38:04.102 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:38:04.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:04.109 DEBUG:teuthology.orchestra.run.smithi039:> stat --file-system '--printf=%T 2024-05-24T11:38:04.109 DEBUG:teuthology.orchestra.run.smithi039:> ' -- /tmp/cephfs 2024-05-24T11:38:04.127 INFO:teuthology.orchestra.run.smithi039.stdout:fuseblk 2024-05-24T11:38:04.127 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-05-24T11:38:04.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:38:04.127 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /tmp/cephfs 2024-05-24T11:38:04.198 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-24T11:38:04.210 INFO:teuthology.run_tasks:Running task exec... 2024-05-24T11:38:04.219 INFO:teuthology.task.exec:Executing custom commands... 2024-05-24T11:38:04.219 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:38:04.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-24T11:38:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.360+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/2043763743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d78101950 msgr2=0x7f8d78108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.360+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/2043763743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d78101950 0x7f8d78108270 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000b0a0 tx=0x7f8d7000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:38:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.360+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/2043763743 shutdown_connections 2024-05-24T11:38:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.360+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/2043763743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d780ffe40 0x7f8d78100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.360+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/2043763743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d78101950 0x7f8d78108270 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.360+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/2043763743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d78100f90 0x7f8d78101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.360+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/2043763743 >> 172.21.15.39:0/2043763743 conn(0x7f8d780fb3b0 msgr2=0x7f8d780fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.360+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/2043763743 shutdown_connections 2024-05-24T11:38:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.360+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/2043763743 wait complete. 2024-05-24T11:38:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7fa7d700 1 Processor -- start 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7fa7d700 1 -- start start 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7fa7d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d780ffe40 0x7f8d78069c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7fa7d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d78100f90 0x7f8d7819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7fa7d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d78101950 0x7f8d7819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7fa7d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7819e8d0 con 0x7f8d780ffe40 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7fa7d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7819ea40 con 0x7f8d78101950 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7fa7d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7819ec00 con 0x7f8d78100f90 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7d819700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d780ffe40 0x7f8d78069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7d819700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d780ffe40 0x7f8d78069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46112/0 (socket says 172.21.15.39:46112) 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7d819700 1 -- 172.21.15.39:0/4217633973 learned_addr learned my addr 172.21.15.39:0/4217633973 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7d819700 1 -- 172.21.15.39:0/4217633973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d78100f90 msgr2=0x7f8d7819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7d819700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d78100f90 0x7f8d7819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7e01a700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d78101950 0x7f8d7819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7d819700 1 -- 172.21.15.39:0/4217633973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d78101950 msgr2=0x7f8d7819e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7d819700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d78101950 0x7f8d7819e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7d819700 1 -- 172.21.15.39:0/4217633973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d70009d00 con 0x7f8d780ffe40 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d7d819700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d780ffe40 0x7f8d78069c30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c9b0 tx=0x7f8d6800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.361+0000 7f8d6effd700 1 -- 172.21.15.39:0/4217633973 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68007b00 con 0x7f8d780ffe40 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.362+0000 7f8d6effd700 1 -- 172.21.15.39:0/4217633973 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d68007c60 con 0x7f8d780ffe40 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.362+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7819eee0 con 0x7f8d780ffe40 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.362+0000 7f8d6effd700 1 -- 172.21.15.39:0/4217633973 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68018660 con 0x7f8d780ffe40 2024-05-24T11:38:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.362+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7819f360 con 0x7f8d780ffe40 2024-05-24T11:38:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.362+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d781071e0 con 0x7f8d780ffe40 2024-05-24T11:38:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.364+0000 7f8d6effd700 1 -- 172.21.15.39:0/4217633973 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8d6800f450 con 0x7f8d780ffe40 2024-05-24T11:38:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.364+0000 7f8d6effd700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d640a11b0 0x7f8d640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.364+0000 7f8d7d018700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d640a11b0 0x7f8d640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.364+0000 7f8d6effd700 1 -- 172.21.15.39:0/4217633973 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8d680ccc70 con 0x7f8d780ffe40 2024-05-24T11:38:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.365+0000 7f8d7d018700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d640a11b0 0x7f8d640a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d74005950 tx=0x7f8d740058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:04.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.368+0000 7f8d6effd700 1 -- 172.21.15.39:0/4217633973 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d68096430 con 0x7f8d780ffe40 2024-05-24T11:38:04.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:04.635+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f8d7810b4f0 con 0x7f8d780ffe40 2024-05-24T11:38:05.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.028+0000 7f8d6effd700 1 -- 172.21.15.39:0/4217633973 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f8d680a00f0 con 0x7f8d780ffe40 2024-05-24T11:38:05.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.029+0000 7f8d6effd700 1 -- 172.21.15.39:0/4217633973 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8d680ce460 con 0x7f8d780ffe40 2024-05-24T11:38:05.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d640a11b0 msgr2=0x7f8d640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d640a11b0 0x7f8d640a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d74005950 tx=0x7f8d740058e0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d780ffe40 msgr2=0x7f8d78069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d780ffe40 0x7f8d78069c30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c9b0 tx=0x7f8d6800cd70 comp rx=0 tx=0).stop 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 shutdown_connections 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f8d640a11b0 0x7f8d640a3670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d78101950 0x7f8d7819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d780ffe40 0x7f8d78069c30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 --2- 172.21.15.39:0/4217633973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d78100f90 0x7f8d7819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.031+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 >> 172.21.15.39:0/4217633973 conn(0x7f8d780fb3b0 msgr2=0x7f8d78104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.032+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 shutdown_connections 2024-05-24T11:38:05.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.032+0000 7f8d7fa7d700 1 -- 172.21.15.39:0/4217633973 wait complete. 2024-05-24T11:38:05.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-24T11:38:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.187+0000 7f15c0890700 1 -- 172.21.15.39:0/148944654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc106c20 msgr2=0x7f15bc109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.187+0000 7f15c0890700 1 --2- 172.21.15.39:0/148944654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc106c20 0x7f15bc109090 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f15b400b0a0 tx=0x7f15b400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.187+0000 7f15c0890700 1 -- 172.21.15.39:0/148944654 shutdown_connections 2024-05-24T11:38:05.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.187+0000 7f15c0890700 1 --2- 172.21.15.39:0/148944654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15bc1019c0 0x7f15bc103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.187+0000 7f15c0890700 1 --2- 172.21.15.39:0/148944654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc106c20 0x7f15bc109090 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.187+0000 7f15c0890700 1 --2- 172.21.15.39:0/148944654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15bc1042f0 0x7f15bc1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.187+0000 7f15c0890700 1 -- 172.21.15.39:0/148944654 >> 172.21.15.39:0/148944654 conn(0x7f15bc0fb3b0 msgr2=0x7f15bc0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:05.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.187+0000 7f15c0890700 1 -- 172.21.15.39:0/148944654 shutdown_connections 2024-05-24T11:38:05.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.187+0000 7f15c0890700 1 -- 172.21.15.39:0/148944654 wait complete. 2024-05-24T11:38:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.188+0000 7f15c0890700 1 Processor -- start 2024-05-24T11:38:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.188+0000 7f15c0890700 1 -- start start 2024-05-24T11:38:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15c0890700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15bc1019c0 0x7f15bc196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15c0890700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc1042f0 0x7f15bc197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15c0890700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15bc106c20 0x7f15bc19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15c0890700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc19e980 con 0x7f15bc1042f0 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15c0890700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc19eac0 con 0x7f15bc1019c0 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15c0890700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc19ec50 con 0x7f15bc106c20 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15b9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc1042f0 0x7f15bc197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15b9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc1042f0 0x7f15bc197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46160/0 (socket says 172.21.15.39:46160) 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15b9d9b700 1 -- 172.21.15.39:0/1495557511 learned_addr learned my addr 172.21.15.39:0/1495557511 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15ba59c700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15bc1019c0 0x7f15bc196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15b9d9b700 1 -- 172.21.15.39:0/1495557511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15bc106c20 msgr2=0x7f15bc19e3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15b9d9b700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15bc106c20 0x7f15bc19e3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15b9d9b700 1 -- 172.21.15.39:0/1495557511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15bc1019c0 msgr2=0x7f15bc196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15b9d9b700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15bc1019c0 0x7f15bc196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15b9d9b700 1 -- 172.21.15.39:0/1495557511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac009520 con 0x7f15bc1042f0 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15b9d9b700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc1042f0 0x7f15bc197390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f15b000e9a0 tx=0x7f15b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15ab7fe700 1 -- 172.21.15.39:0/1495557511 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0009790 con 0x7f15bc1042f0 2024-05-24T11:38:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b4009d00 con 0x7f15bc1042f0 2024-05-24T11:38:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc19f270 con 0x7f15bc1042f0 2024-05-24T11:38:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15ab7fe700 1 -- 172.21.15.39:0/1495557511 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15b000cbd0 con 0x7f15bc1042f0 2024-05-24T11:38:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.189+0000 7f15ab7fe700 1 -- 172.21.15.39:0/1495557511 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0018680 con 0x7f15bc1042f0 2024-05-24T11:38:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.191+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc0fcf30 con 0x7f15bc1042f0 2024-05-24T11:38:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.192+0000 7f15ab7fe700 1 -- 172.21.15.39:0/1495557511 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f15b00187e0 con 0x7f15bc1042f0 2024-05-24T11:38:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.192+0000 7f15ab7fe700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f15a40a1280 0x7f15a40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.192+0000 7f15ba59c700 1 -- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f15a40a1280 msgr2=0x7f15a40a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.193+0000 7f15ab7fe700 1 -- 172.21.15.39:0/1495557511 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f15b0014070 con 0x7f15bc1042f0 2024-05-24T11:38:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.193+0000 7f15ba59c700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f15a40a1280 0x7f15a40a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:38:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.197+0000 7f15ab7fe700 1 -- 172.21.15.39:0/1495557511 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f15b0052d60 con 0x7f15bc1042f0 2024-05-24T11:38:05.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:05 smithi039 ceph-mon[42919]: pgmap v260: 97 pgs: 97 active+clean; 640 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-24T11:38:05.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:05 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4217633973' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-24T11:38:05.328 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: ignoring --setuser ceph since I am not root 2024-05-24T11:38:05.328 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: ignoring --setgroup ceph since I am not root 2024-05-24T11:38:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.394+0000 7f15ba59c700 1 -- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f15a40a1280 msgr2=0x7f15a40a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.394+0000 7f15ba59c700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f15a40a1280 0x7f15a40a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-24T11:38:05.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.406+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f15bc0675a0 con 0x7f15bc1042f0 2024-05-24T11:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.412+0000 7f15ab7fe700 1 -- 172.21.15.39:0/1495557511 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f15b00962a0 con 0x7f15bc1042f0 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f15a40a1280 msgr2=0x7f15a40a3740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f15a40a1280 0x7f15a40a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc1042f0 msgr2=0x7f15bc197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc1042f0 0x7f15bc197390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f15b000e9a0 tx=0x7f15b000ecb0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 shutdown_connections 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f15a40a1280 0x7f15a40a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15bc106c20 0x7f15bc19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15bc1019c0 0x7f15bc196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 --2- 172.21.15.39:0/1495557511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15bc1042f0 0x7f15bc197390 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 >> 172.21.15.39:0/1495557511 conn(0x7f15bc0fb3b0 msgr2=0x7f15bc102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:05.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 shutdown_connections 2024-05-24T11:38:05.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.415+0000 7f15c0890700 1 -- 172.21.15.39:0/1495557511 wait complete. 2024-05-24T11:38:05.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-24T11:38:05.470 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:05 smithi137 ceph-mon[47907]: pgmap v260: 97 pgs: 97 active+clean; 640 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-24T11:38:05.470 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:05 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4217633973' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-24T11:38:05.471 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: ignoring --setuser ceph since I am not root 2024-05-24T11:38:05.471 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: ignoring --setgroup ceph since I am not root 2024-05-24T11:38:05.471 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:05.465+0000 7f3488ad3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-24T11:38:05.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:05 smithi053 ceph-mon[47448]: pgmap v260: 97 pgs: 97 active+clean; 640 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-24T11:38:05.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:05 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4217633973' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-24T11:38:05.489 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: ignoring --setuser ceph since I am not root 2024-05-24T11:38:05.489 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: ignoring --setgroup ceph since I am not root 2024-05-24T11:38:05.490 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:05 smithi053 ceph-mgr[48962]: -- 172.21.15.53:0/463414934 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558b92e921a0 con 0x558b92e68800 2024-05-24T11:38:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.557+0000 7f72a7182700 1 -- 172.21.15.39:0/1532737533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a00ffe00 msgr2=0x7f72a0100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.557+0000 7f72a7182700 1 --2- 172.21.15.39:0/1532737533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a00ffe00 0x7f72a0100220 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f72940099c0 tx=0x7f7294009cd0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.557+0000 7f72a7182700 1 -- 172.21.15.39:0/1532737533 shutdown_connections 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.557+0000 7f72a7182700 1 --2- 172.21.15.39:0/1532737533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a0101990 0x7f72a0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.557+0000 7f72a7182700 1 --2- 172.21.15.39:0/1532737533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a00ffe00 0x7f72a0100220 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.557+0000 7f72a7182700 1 --2- 172.21.15.39:0/1532737533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a0100ff0 0x7f72a0101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.557+0000 7f72a7182700 1 -- 172.21.15.39:0/1532737533 >> 172.21.15.39:0/1532737533 conn(0x7f72a00fb390 msgr2=0x7f72a00fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.557+0000 7f72a7182700 1 -- 172.21.15.39:0/1532737533 shutdown_connections 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.557+0000 7f72a7182700 1 -- 172.21.15.39:0/1532737533 wait complete. 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a7182700 1 Processor -- start 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a7182700 1 -- start start 2024-05-24T11:38:05.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a7182700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a00ffe00 0x7f72a0068d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a7182700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a0100ff0 0x7f72a019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a7182700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a0101990 0x7f72a019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a7182700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a019e8f0 con 0x7f72a0101990 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a7182700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a019ea60 con 0x7f72a0100ff0 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a7182700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a019ec20 con 0x7f72a00ffe00 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a571f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a0101990 0x7f72a019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a4f1e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a00ffe00 0x7f72a0068d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a571f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a0101990 0x7f72a019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46184/0 (socket says 172.21.15.39:46184) 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f72a571f700 1 -- 172.21.15.39:0/2043705924 learned_addr learned my addr 172.21.15.39:0/2043705924 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.558+0000 7f729ffff700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a0100ff0 0x7f72a019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:05.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f72a571f700 1 -- 172.21.15.39:0/2043705924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a00ffe00 msgr2=0x7f72a0068d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f72a571f700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a00ffe00 0x7f72a0068d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f72a571f700 1 -- 172.21.15.39:0/2043705924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a0100ff0 msgr2=0x7f72a019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f72a571f700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a0100ff0 0x7f72a019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f72a571f700 1 -- 172.21.15.39:0/2043705924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940095f0 con 0x7f72a0101990 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f72a571f700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a0101990 0x7f72a019e150 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f729800ef30 tx=0x7f729800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f729dffb700 1 -- 172.21.15.39:0/2043705924 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298018070 con 0x7f72a0101990 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f729dffb700 1 -- 172.21.15.39:0/2043705924 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f729800f040 con 0x7f72a0101990 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a019eea0 con 0x7f72a0101990 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f729dffb700 1 -- 172.21.15.39:0/2043705924 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729800cbc0 con 0x7f72a0101990 2024-05-24T11:38:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.559+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a019f2d0 con 0x7f72a0101990 2024-05-24T11:38:05.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.560+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a0107370 con 0x7f72a0101990 2024-05-24T11:38:05.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.561+0000 7f729dffb700 1 -- 172.21.15.39:0/2043705924 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f7298007500 con 0x7f72a0101990 2024-05-24T11:38:05.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.562+0000 7f729dffb700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72900a1200 0x7f72900a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.562+0000 7f72a4f1e700 1 -- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72900a1200 msgr2=0x7f72900a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:05.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.562+0000 7f729dffb700 1 -- 172.21.15.39:0/2043705924 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f72980cc580 con 0x7f72a0101990 2024-05-24T11:38:05.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.562+0000 7f72a4f1e700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72900a1200 0x7f72900a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:38:05.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.567+0000 7f729dffb700 1 -- 172.21.15.39:0/2043705924 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7298095c40 con 0x7f72a0101990 2024-05-24T11:38:05.689 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:05.490+0000 7efc5151d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-24T11:38:05.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:05.591+0000 7efc5151d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-24T11:38:05.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:05.684+0000 7efc5151d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-24T11:38:05.738 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:05.558+0000 7f3488ad3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-24T11:38:05.738 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:05.647+0000 7f3488ad3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-24T11:38:05.738 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:05.733+0000 7f3488ad3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-24T11:38:05.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.763+0000 7f72a4f1e700 1 -- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72900a1200 msgr2=0x7f72900a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:05.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.763+0000 7f72a4f1e700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72900a1200 0x7f72900a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-24T11:38:05.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.776+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f72a010b510 con 0x7f72a0101990 2024-05-24T11:38:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.782+0000 7f729dffb700 1 -- 172.21.15.39:0/2043705924 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7298099260 con 0x7f72a0101990 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72900a1200 msgr2=0x7f72900a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72900a1200 0x7f72900a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a0101990 msgr2=0x7f72a019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a0101990 0x7f72a019e150 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f729800ef30 tx=0x7f729800c3a0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 shutdown_connections 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f72900a1200 0x7f72900a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72a00ffe00 0x7f72a0068d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a0101990 0x7f72a019e150 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 --2- 172.21.15.39:0/2043705924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72a0100ff0 0x7f72a019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 >> 172.21.15.39:0/2043705924 conn(0x7f72a00fb390 msgr2=0x7f72a01044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 shutdown_connections 2024-05-24T11:38:05.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.785+0000 7f72a7182700 1 -- 172.21.15.39:0/2043705924 wait complete. 2024-05-24T11:38:05.803 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-24T11:38:05.818 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:05.513+0000 7fdeb6ced200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-24T11:38:05.818 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:05.618+0000 7fdeb6ced200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-24T11:38:05.818 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:05.717+0000 7fdeb6ced200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-24T11:38:05.818 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:05.813+0000 7fdeb6ced200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-24T11:38:05.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.929+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/2914734836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba41019c0 msgr2=0x7f7ba4103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.929+0000 7f7ba9b2e700 1 --2- 172.21.15.39:0/2914734836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba41019c0 0x7f7ba4103db0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009990 tx=0x7f7b94009ca0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.929+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/2914734836 shutdown_connections 2024-05-24T11:38:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.929+0000 7f7ba9b2e700 1 --2- 172.21.15.39:0/2914734836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba4106c20 0x7f7ba4109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.929+0000 7f7ba9b2e700 1 --2- 172.21.15.39:0/2914734836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba41019c0 0x7f7ba4103db0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.929+0000 7f7ba9b2e700 1 --2- 172.21.15.39:0/2914734836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba41042f0 0x7f7ba41066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.929+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/2914734836 >> 172.21.15.39:0/2914734836 conn(0x7f7ba40fb3b0 msgr2=0x7f7ba40fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.929+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/2914734836 shutdown_connections 2024-05-24T11:38:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.929+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/2914734836 wait complete. 2024-05-24T11:38:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.930+0000 7f7ba9b2e700 1 Processor -- start 2024-05-24T11:38:05.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.930+0000 7f7ba9b2e700 1 -- start start 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.930+0000 7f7ba9b2e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba41019c0 0x7f7ba4196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.930+0000 7f7ba9b2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba41042f0 0x7f7ba41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.930+0000 7f7ba9b2e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba4106c20 0x7f7ba419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.930+0000 7f7ba9b2e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba419e990 con 0x7f7ba41042f0 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.930+0000 7f7ba9b2e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba419ead0 con 0x7f7ba41019c0 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba9b2e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba419ec60 con 0x7f7ba4106c20 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba41042f0 0x7f7ba41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba41042f0 0x7f7ba41973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46192/0 (socket says 172.21.15.39:46192) 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba2ffd700 1 -- 172.21.15.39:0/501787846 learned_addr learned my addr 172.21.15.39:0/501787846 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba37fe700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba41019c0 0x7f7ba4196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:05.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba3fff700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba4106c20 0x7f7ba419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba2ffd700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba4106c20 msgr2=0x7f7ba419e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba2ffd700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba4106c20 0x7f7ba419e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba2ffd700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba41019c0 msgr2=0x7f7ba4196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba2ffd700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba41019c0 0x7f7ba4196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba2ffd700 1 -- 172.21.15.39:0/501787846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b98009520 con 0x7f7ba41042f0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba2ffd700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba41042f0 0x7f7ba41973a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800eb30 tx=0x7f7b9800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98009880 con 0x7f7ba41042f0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b940095f0 con 0x7f7ba41042f0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b980044b0 con 0x7f7ba41042f0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98003ce0 con 0x7f7ba41042f0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.931+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba419f280 con 0x7f7ba41042f0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.932+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba40fcf30 con 0x7f7ba41042f0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.933+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f7b98003e40 con 0x7f7ba41042f0 2024-05-24T11:38:05.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.934+0000 7f7ba0ff9700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 0x7f7b8c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:05.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.934+0000 7f7ba37fe700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 msgr2=0x7f7b8c0a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:05.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.934+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7b98014070 con 0x7f7ba41042f0 2024-05-24T11:38:05.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.934+0000 7f7ba37fe700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 0x7f7b8c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-24T11:38:05.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:05.938+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7b980966c0 con 0x7f7ba41042f0 2024-05-24T11:38:06.024 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:05.773+0000 7efc5151d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-24T11:38:06.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:06.136+0000 7f7ba37fe700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 msgr2=0x7f7b8c0a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:06.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:06.136+0000 7f7ba37fe700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 0x7f7b8c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-24T11:38:06.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:06.203+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 --> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ba402ce10 con 0x7f7b8c0a1200 2024-05-24T11:38:06.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:06 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4217633973' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-24T11:38:06.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:06 smithi053 ceph-mon[47448]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-24T11:38:06.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:06 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1495557511' entity='client.admin' 2024-05-24T11:38:06.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:06 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2043705924' entity='client.admin' 2024-05-24T11:38:06.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:06 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4217633973' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-24T11:38:06.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:06 smithi039 ceph-mon[42919]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-24T11:38:06.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:06 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1495557511' entity='client.admin' 2024-05-24T11:38:06.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:06 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2043705924' entity='client.admin' 2024-05-24T11:38:06.367 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:06 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4217633973' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-24T11:38:06.367 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:06 smithi137 ceph-mon[47907]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-24T11:38:06.367 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:06 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1495557511' entity='client.admin' 2024-05-24T11:38:06.367 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:06 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2043705924' entity='client.admin' 2024-05-24T11:38:06.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:06.537+0000 7f7ba37fe700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 msgr2=0x7f7b8c0a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:06.537+0000 7f7ba37fe700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 0x7f7b8c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-24T11:38:06.621 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:06.362+0000 7f3488ad3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:38:06.733 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:06 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:06.462+0000 7efc5151d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:38:06.826 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:06.534+0000 7fdeb6ced200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:38:06.826 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:06.820+0000 7fdeb6ced200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-24T11:38:06.978 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:06.616+0000 7f3488ad3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-24T11:38:06.978 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:06.704+0000 7f3488ad3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-24T11:38:06.978 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:06.814+0000 7f3488ad3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-24T11:38:06.978 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:06.972+0000 7f3488ad3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-24T11:38:07.078 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:06 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:06.728+0000 7efc5151d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-24T11:38:07.078 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:06 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:06.818+0000 7efc5151d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-24T11:38:07.078 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:06 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:06.933+0000 7efc5151d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-24T11:38:07.227 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:06.918+0000 7fdeb6ced200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-24T11:38:07.227 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:07.042+0000 7fdeb6ced200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-24T11:38:07.228 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:07.222+0000 7fdeb6ced200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-24T11:38:07.245 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:07.101+0000 7f3488ad3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-24T11:38:07.245 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:07.186+0000 7f3488ad3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-24T11:38:07.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:07.338+0000 7f7ba37fe700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 msgr2=0x7f7b8c0a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:07.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:07.338+0000 7f7ba37fe700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 0x7f7b8c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-24T11:38:07.489 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:07.367+0000 7fdeb6ced200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-24T11:38:07.489 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:07.463+0000 7fdeb6ced200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-24T11:38:07.578 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:07.099+0000 7efc5151d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-24T11:38:07.578 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:07.234+0000 7efc5151d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-24T11:38:07.578 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:07.324+0000 7efc5151d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-24T11:38:07.952 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:07.555+0000 7f3488ad3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-24T11:38:08.078 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:07.714+0000 7efc5151d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-24T11:38:08.239 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:07.886+0000 7fdeb6ced200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-24T11:38:08.241 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:07.947+0000 7f3488ad3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-24T11:38:08.241 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:08 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:08.143+0000 7f3488ad3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-24T11:38:08.241 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:08 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:08.236+0000 7f3488ad3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-24T11:38:08.434 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:08 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:08.125+0000 7efc5151d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-24T11:38:08.435 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:08 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:08.331+0000 7efc5151d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-24T11:38:08.662 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:08.329+0000 7fdeb6ced200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-24T11:38:08.662 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:08.551+0000 7fdeb6ced200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-24T11:38:08.662 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:08.657+0000 7fdeb6ced200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-24T11:38:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:08.939+0000 7f7ba37fe700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 msgr2=0x7f7b8c0a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:08.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:08.939+0000 7f7ba37fe700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 0x7f7b8c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-24T11:38:09.495 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:09.003+0000 7f3488ad3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-24T11:38:09.495 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:09.100+0000 7f3488ad3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-24T11:38:09.495 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:09.248+0000 7f3488ad3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-24T11:38:09.507 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:08 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:08.429+0000 7efc5151d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-24T11:38:09.507 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:09 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:09.241+0000 7efc5151d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-24T11:38:09.507 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:09 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:09.343+0000 7efc5151d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-24T11:38:09.507 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:09 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:09.502+0000 7efc5151d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-24T11:38:09.811 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:09 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:09.528+0000 7fdeb6ced200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-24T11:38:09.811 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:09 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:09.636+0000 7fdeb6ced200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-24T11:38:09.811 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:09 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:09.805+0000 7fdeb6ced200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-24T11:38:10.245 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:09.878+0000 7f3488ad3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-24T11:38:10.245 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:10.060+0000 7f3488ad3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-24T11:38:10.576 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:10 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:10.167+0000 7efc5151d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-24T11:38:10.576 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:10 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:10.362+0000 7efc5151d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-24T11:38:10.576 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:10 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:10.571+0000 7efc5151d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-24T11:38:10.745 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:10.253+0000 7f3488ad3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-24T11:38:10.957 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:10 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:10.529+0000 7fdeb6ced200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-24T11:38:10.958 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:10 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:10.734+0000 7fdeb6ced200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-24T11:38:10.958 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:10 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:10.952+0000 7fdeb6ced200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-24T11:38:11.471 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:11.182+0000 7f3488ad3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-24T11:38:11.745 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:11.466+0000 7f3488ad3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:38:11.745 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:11.555+0000 7f3488ad3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-24T11:38:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:11.821+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 19) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f7b98096a90 con 0x7f7ba41042f0 2024-05-24T11:38:11.828 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:11 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:11.559+0000 7efc5151d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-24T11:38:12.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:12.143+0000 7f7ba37fe700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 msgr2=0x7f7b8c0a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2090515453 2024-05-24T11:38:12.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:12.143+0000 7f7ba37fe700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 0x7f7b8c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-24T11:38:12.175 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:11 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:11.869+0000 7efc5151d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:38:12.175 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:11 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:11.964+0000 7efc5151d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-24T11:38:12.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:11 smithi039 ceph-mon[42919]: Standby manager daemon z restarted 2024-05-24T11:38:12.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:11 smithi039 ceph-mon[42919]: Standby manager daemon z started 2024-05-24T11:38:12.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:11 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-24T11:38:12.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:11 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:38:12.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:11 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-24T11:38:12.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:11 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:38:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:12.195+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mgrmap(e 20) v1 ==== 135558+0+0 (secure 0 0 0) 0x7f7b980173d0 con 0x7f7ba41042f0 2024-05-24T11:38:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:12.196+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 msgr2=0x7f7b8c0a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:38:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:12.196+0000 7f7ba0ff9700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/2090515453,v1:172.21.15.39:6801/2090515453] conn(0x7f7b8c0a1200 0x7f7b8c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:12.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:11 smithi053 ceph-mon[47448]: Standby manager daemon z restarted 2024-05-24T11:38:12.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:11 smithi053 ceph-mon[47448]: Standby manager daemon z started 2024-05-24T11:38:12.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:11 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-24T11:38:12.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:11 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:38:12.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:11 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-24T11:38:12.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:11 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:38:12.241 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:12.011+0000 7fdeb6ced200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-24T11:38:12.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:11 smithi137 ceph-mon[47907]: Standby manager daemon z restarted 2024-05-24T11:38:12.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:11 smithi137 ceph-mon[47907]: Standby manager daemon z started 2024-05-24T11:38:12.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:11 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-24T11:38:12.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:11 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:38:12.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:11 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-24T11:38:12.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:11 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.137:0/1482494793' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:38:12.246 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:38:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z[49762]: 2024-05-24T11:38:11.747+0000 7f3488ad3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-24T11:38:12.450 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.170+0000 7efc5151d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-24T11:38:12.700 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:12.363+0000 7fdeb6ced200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-24T11:38:12.700 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:12.473+0000 7fdeb6ced200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-24T11:38:12.700 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:38:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y[48939]: 2024-05-24T11:38:12.694+0000 7fdeb6ced200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-24T11:38:12.828 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb282f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.828 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb282f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.828 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb002a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.828 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb002a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.829 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb002a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.829 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb002a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.829 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb002a700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.829 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb282f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.829 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb282f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.829 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:38:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:38:12.458+0000 7efbb282f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:38:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: mgrmap e19: x(active, since 6m), standbys: y, z 2024-05-24T11:38:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: Active manager daemon x restarted 2024-05-24T11:38:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: Activating manager daemon x 2024-05-24T11:38:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: osdmap e80: 12 total, 12 up, 12 in 2024-05-24T11:38:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: mgrmap e20: x(active, starting, since 0.0130205s), standbys: y, z 2024-05-24T11:38:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:38:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:38:12.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-24T11:38:12.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-24T11:38:12.992 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: Manager daemon x is now available 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: Standby manager daemon y restarted 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: Standby manager daemon y started 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-24T11:38:12.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:12 smithi053 ceph-mon[47448]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:38:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.200+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mon.0 v2:172.21.15.39:3300/0 9 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f7b98056eb0 con 0x7f7ba41042f0 2024-05-24T11:38:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.201+0000 7f7ba0ff9700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c01bb90 0x7f7b8c01bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.201+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 --> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ba402ce10 con 0x7f7b8c01bb90 2024-05-24T11:38:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.203+0000 7f7ba37fe700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c01bb90 0x7f7b8c01bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:13.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.204+0000 7f7ba37fe700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c01bb90 0x7f7b8c01bf70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b94005910 tx=0x7f7b9400b610 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: mgrmap e19: x(active, since 6m), standbys: y, z 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: Active manager daemon x restarted 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: Activating manager daemon x 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: osdmap e80: 12 total, 12 up, 12 in 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: mgrmap e20: x(active, starting, since 0.0130205s), standbys: y, z 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-24T11:38:13.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-24T11:38:13.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-24T11:38:13.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: Manager daemon x is now available 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: Standby manager daemon y restarted 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: Standby manager daemon y started 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-24T11:38:13.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:12 smithi137 ceph-mon[47907]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:38:13.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.274+0000 7f7ba0ff9700 1 -- 172.21.15.39:0/501787846 <== mgr.15360 v2:172.21.15.39:6800/1882625876 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7ba402ce10 con 0x7f7b8c01bb90 2024-05-24T11:38:13.279 INFO:teuthology.orchestra.run.smithi039.stdout:Schedule set for path / 2024-05-24T11:38:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.276+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c01bb90 msgr2=0x7f7b8c01bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.276+0000 7f7ba9b2e700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c01bb90 0x7f7b8c01bf70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b94005910 tx=0x7f7b9400b610 comp rx=0 tx=0).stop 2024-05-24T11:38:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.276+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba41042f0 msgr2=0x7f7ba41973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.276+0000 7f7ba9b2e700 1 --2- 172.21.15.39:0/501787846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba41042f0 0x7f7ba41973a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800eb30 tx=0x7f7b9800ee40 comp rx=0 tx=0).stop 2024-05-24T11:38:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.276+0000 7f7ba3fff700 1 -- 172.21.15.39:0/501787846 reap_dead start 2024-05-24T11:38:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.276+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 shutdown_connections 2024-05-24T11:38:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.276+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 >> 172.21.15.39:0/501787846 conn(0x7f7ba40fb3b0 msgr2=0x7f7ba4103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.276+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 shutdown_connections 2024-05-24T11:38:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.276+0000 7f7ba9b2e700 1 -- 172.21.15.39:0/501787846 wait complete. 2024-05-24T11:38:13.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: mgrmap e19: x(active, since 6m), standbys: y, z 2024-05-24T11:38:13.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: Active manager daemon x restarted 2024-05-24T11:38:13.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: Activating manager daemon x 2024-05-24T11:38:13.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: osdmap e80: 12 total, 12 up, 12 in 2024-05-24T11:38:13.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: mgrmap e20: x(active, starting, since 0.0130205s), standbys: y, z 2024-05-24T11:38:13.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-24T11:38:13.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-24T11:38:13.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: Manager daemon x is now available 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: Standby manager daemon y restarted 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: Standby manager daemon y started 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-24T11:38:13.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-24T11:38:13.297 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-24T11:38:13.297 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:12 smithi039 ceph-mon[42919]: from='mgr.? 172.21.15.53:0/3633952224' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-24T11:38:13.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-24T11:38:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.428+0000 7f09d064d700 1 -- 172.21.15.39:0/453001114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 msgr2=0x7f09c81066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.428+0000 7f09d064d700 1 --2- 172.21.15.39:0/453001114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 0x7f09c81066b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009990 tx=0x7f09b8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 -- 172.21.15.39:0/453001114 shutdown_connections 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 --2- 172.21.15.39:0/453001114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09c8101990 0x7f09c8103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 --2- 172.21.15.39:0/453001114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c8106bf0 0x7f09c8109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 --2- 172.21.15.39:0/453001114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 0x7f09c81066b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 -- 172.21.15.39:0/453001114 >> 172.21.15.39:0/453001114 conn(0x7f09c80fb380 msgr2=0x7f09c80fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 -- 172.21.15.39:0/453001114 shutdown_connections 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 -- 172.21.15.39:0/453001114 wait complete. 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 Processor -- start 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 -- start start 2024-05-24T11:38:13.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c8101990 0x7f09c81012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 0x7f09c80ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.429+0000 7f09d064d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09c8106bf0 0x7f09c80ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09d064d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c8100620 con 0x7f09c81042c0 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09d064d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c8100790 con 0x7f09c8101990 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09d064d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c81a0e30 con 0x7f09c8106bf0 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cdbe8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 0x7f09c80ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cdbe8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 0x7f09c80ff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33118/0 (socket says 172.21.15.39:33118) 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cdbe8700 1 -- 172.21.15.39:0/3198032411 learned_addr learned my addr 172.21.15.39:0/3198032411 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cebea700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09c8106bf0 0x7f09c80ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09ce3e9700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c8101990 0x7f09c81012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cdbe8700 1 -- 172.21.15.39:0/3198032411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09c8106bf0 msgr2=0x7f09c80ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cdbe8700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09c8106bf0 0x7f09c80ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cdbe8700 1 -- 172.21.15.39:0/3198032411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c8101990 msgr2=0x7f09c81012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cdbe8700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c8101990 0x7f09c81012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cdbe8700 1 -- 172.21.15.39:0/3198032411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80095f0 con 0x7f09c81042c0 2024-05-24T11:38:13.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09cdbe8700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 0x7f09c80ff940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f09b8006010 tx=0x7f09b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:13.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09bf7fe700 1 -- 172.21.15.39:0/3198032411 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b801d070 con 0x7f09c81042c0 2024-05-24T11:38:13.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c81a1060 con 0x7f09c81042c0 2024-05-24T11:38:13.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09bf7fe700 1 -- 172.21.15.39:0/3198032411 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09b800bdb0 con 0x7f09c81042c0 2024-05-24T11:38:13.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09bf7fe700 1 -- 172.21.15.39:0/3198032411 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b800fda0 con 0x7f09c81042c0 2024-05-24T11:38:13.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.430+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c81a1480 con 0x7f09c81042c0 2024-05-24T11:38:13.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.432+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c80fcf00 con 0x7f09c81042c0 2024-05-24T11:38:13.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.433+0000 7f09bf7fe700 1 -- 172.21.15.39:0/3198032411 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f09b800f460 con 0x7f09c81042c0 2024-05-24T11:38:13.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.434+0000 7f09bf7fe700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f09b40a0f20 0x7f09b40a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:13.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.434+0000 7f09ce3e9700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f09b40a0f20 0x7f09b40a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:13.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.434+0000 7f09bf7fe700 1 -- 172.21.15.39:0/3198032411 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f09b80cd920 con 0x7f09c81042c0 2024-05-24T11:38:13.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.434+0000 7f09ce3e9700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f09b40a0f20 0x7f09b40a33e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009c60 tx=0x7f09c4009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:13.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.438+0000 7f09bf7fe700 1 -- 172.21.15.39:0/3198032411 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f09b8097350 con 0x7f09c81042c0 2024-05-24T11:38:13.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.701+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 --> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f09c80613e0 con 0x7f09b40a0f20 2024-05-24T11:38:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.767+0000 7f09bf7fe700 1 -- 172.21.15.39:0/3198032411 <== mgr.15360 v2:172.21.15.39:6800/1882625876 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f09c80613e0 con 0x7f09b40a0f20 2024-05-24T11:38:13.773 INFO:teuthology.orchestra.run.smithi039.stdout:Retention added to path / 2024-05-24T11:38:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.771+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f09b40a0f20 msgr2=0x7f09b40a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.771+0000 7f09d064d700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f09b40a0f20 0x7f09b40a33e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009c60 tx=0x7f09c4009260 comp rx=0 tx=0).stop 2024-05-24T11:38:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.771+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 msgr2=0x7f09c80ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.771+0000 7f09d064d700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 0x7f09c80ff940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f09b8006010 tx=0x7f09b8004730 comp rx=0 tx=0).stop 2024-05-24T11:38:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.772+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 shutdown_connections 2024-05-24T11:38:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.772+0000 7f09d064d700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09c8106bf0 0x7f09c80ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.772+0000 7f09d064d700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f09b40a0f20 0x7f09b40a33e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.772+0000 7f09d064d700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09c8101990 0x7f09c81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.772+0000 7f09d064d700 1 --2- 172.21.15.39:0/3198032411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c81042c0 0x7f09c80ff940 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.772+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 >> 172.21.15.39:0/3198032411 conn(0x7f09c80fb380 msgr2=0x7f09c81038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.774+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 shutdown_connections 2024-05-24T11:38:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.774+0000 7f09d064d700 1 -- 172.21.15.39:0/3198032411 wait complete. 2024-05-24T11:38:13.794 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-24T11:38:13.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.924+0000 7f15ba5ac700 1 -- 172.21.15.39:0/366713509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 msgr2=0x7f15b41082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.924+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/366713509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 0x7f15b41082e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00b0a0 tx=0x7f15ac00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.924+0000 7f15ba5ac700 1 -- 172.21.15.39:0/366713509 shutdown_connections 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.924+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/366713509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b41013e0 0x7f15b4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.924+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/366713509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 0x7f15b41082e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.924+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/366713509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b4100080 0x7f15b41004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.924+0000 7f15ba5ac700 1 -- 172.21.15.39:0/366713509 >> 172.21.15.39:0/366713509 conn(0x7f15b40faaf0 msgr2=0x7f15b40fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.924+0000 7f15ba5ac700 1 -- 172.21.15.39:0/366713509 shutdown_connections 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.924+0000 7f15ba5ac700 1 -- 172.21.15.39:0/366713509 wait complete. 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15ba5ac700 1 Processor -- start 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15ba5ac700 1 -- start start 2024-05-24T11:38:13.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15ba5ac700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b4100080 0x7f15b4196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15ba5ac700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 0x7f15b4197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15ba5ac700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b41013e0 0x7f15b419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15ba5ac700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b419e980 con 0x7f15b4100a20 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15ba5ac700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b419eac0 con 0x7f15b4100080 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15ba5ac700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b419ec50 con 0x7f15b41013e0 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 0x7f15b4197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 0x7f15b4197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33132/0 (socket says 172.21.15.39:33132) 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15b37fe700 1 -- 172.21.15.39:0/2546225342 learned_addr learned my addr 172.21.15.39:0/2546225342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15b8b49700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b41013e0 0x7f15b419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.925+0000 7f15b3fff700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b4100080 0x7f15b4196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b37fe700 1 -- 172.21.15.39:0/2546225342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b41013e0 msgr2=0x7f15b419e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b37fe700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b41013e0 0x7f15b419e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b37fe700 1 -- 172.21.15.39:0/2546225342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b4100080 msgr2=0x7f15b4196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b37fe700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b4100080 0x7f15b4196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b37fe700 1 -- 172.21.15.39:0/2546225342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac009d00 con 0x7f15b4100a20 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b8b49700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b41013e0 0x7f15b419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b37fe700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 0x7f15b4197390 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9a0 tx=0x7f15a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b17fa700 1 -- 172.21.15.39:0/2546225342 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8009790 con 0x7f15b4100a20 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b419ef30 con 0x7f15b4100a20 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b17fa700 1 -- 172.21.15.39:0/2546225342 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15a8010430 con 0x7f15b4100a20 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15b17fa700 1 -- 172.21.15.39:0/2546225342 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800cea0 con 0x7f15b4100a20 2024-05-24T11:38:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.926+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b419f3b0 con 0x7f15b4100a20 2024-05-24T11:38:13.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.927+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b40675a0 con 0x7f15b4100a20 2024-05-24T11:38:13.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.929+0000 7f15b17fa700 1 -- 172.21.15.39:0/2546225342 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f15a80105a0 con 0x7f15b4100a20 2024-05-24T11:38:13.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.930+0000 7f15b17fa700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f159c0a0f20 0x7f159c0a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:13.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.930+0000 7f15b3fff700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f159c0a0f20 0x7f159c0a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:13.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.930+0000 7f15b17fa700 1 -- 172.21.15.39:0/2546225342 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f15a8014070 con 0x7f15b4100a20 2024-05-24T11:38:13.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.930+0000 7f15b3fff700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f159c0a0f20 0x7f159c0a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15b4068040 tx=0x7f15a4009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:13.935+0000 7f15b17fa700 1 -- 172.21.15.39:0/2546225342 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f15a8052d60 con 0x7f15b4100a20 2024-05-24T11:38:14.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:13 smithi039 ceph-mon[42919]: [24/May/2024:11:38:12] ENGINE Bus STARTING 2024-05-24T11:38:14.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:13 smithi039 ceph-mon[42919]: [24/May/2024:11:38:12] ENGINE Serving on https://172.21.15.39:7150 2024-05-24T11:38:14.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:13 smithi039 ceph-mon[42919]: [24/May/2024:11:38:12] ENGINE Serving on http://172.21.15.39:8765 2024-05-24T11:38:14.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:13 smithi039 ceph-mon[42919]: [24/May/2024:11:38:12] ENGINE Bus STARTED 2024-05-24T11:38:14.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:13 smithi039 ceph-mon[42919]: mgrmap e21: x(active, since 1.01823s), standbys: z, y 2024-05-24T11:38:14.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:13 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:13 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:13 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:13 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.180+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 --> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f15b40ff6d0 con 0x7f159c0a0f20 2024-05-24T11:38:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.188+0000 7f15b17fa700 1 -- 172.21.15.39:0/2546225342 <== mgr.15360 v2:172.21.15.39:6800/1882625876 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f15b40ff6d0 con 0x7f159c0a0f20 2024-05-24T11:38:14.193 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-24T00:00:00", "created": "2024-05-24T11:38:13", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-24T11:38:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f159c0a0f20 msgr2=0x7f159c0a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f159c0a0f20 0x7f159c0a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15b4068040 tx=0x7f15a4009260 comp rx=0 tx=0).stop 2024-05-24T11:38:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 msgr2=0x7f15b4197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 0x7f15b4197390 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f15a800e9a0 tx=0x7f15a800ecb0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 shutdown_connections 2024-05-24T11:38:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b41013e0 0x7f15b419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f159c0a0f20 0x7f159c0a33e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b4100080 0x7f15b4196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 --2- 172.21.15.39:0/2546225342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4100a20 0x7f15b4197390 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.190+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 >> 172.21.15.39:0/2546225342 conn(0x7f15b40faaf0 msgr2=0x7f15b40fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.191+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 shutdown_connections 2024-05-24T11:38:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.191+0000 7f15ba5ac700 1 -- 172.21.15.39:0/2546225342 wait complete. 2024-05-24T11:38:14.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-24T11:38:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:13 smithi053 ceph-mon[47448]: [24/May/2024:11:38:12] ENGINE Bus STARTING 2024-05-24T11:38:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:13 smithi053 ceph-mon[47448]: [24/May/2024:11:38:12] ENGINE Serving on https://172.21.15.39:7150 2024-05-24T11:38:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:13 smithi053 ceph-mon[47448]: [24/May/2024:11:38:12] ENGINE Serving on http://172.21.15.39:8765 2024-05-24T11:38:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:13 smithi053 ceph-mon[47448]: [24/May/2024:11:38:12] ENGINE Bus STARTED 2024-05-24T11:38:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:13 smithi053 ceph-mon[47448]: mgrmap e21: x(active, since 1.01823s), standbys: z, y 2024-05-24T11:38:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:13 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:13 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:13 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:13 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:13 smithi137 ceph-mon[47907]: [24/May/2024:11:38:12] ENGINE Bus STARTING 2024-05-24T11:38:14.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:13 smithi137 ceph-mon[47907]: [24/May/2024:11:38:12] ENGINE Serving on https://172.21.15.39:7150 2024-05-24T11:38:14.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:13 smithi137 ceph-mon[47907]: [24/May/2024:11:38:12] ENGINE Serving on http://172.21.15.39:8765 2024-05-24T11:38:14.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:13 smithi137 ceph-mon[47907]: [24/May/2024:11:38:12] ENGINE Bus STARTED 2024-05-24T11:38:14.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:13 smithi137 ceph-mon[47907]: mgrmap e21: x(active, since 1.01823s), standbys: z, y 2024-05-24T11:38:14.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:13 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:13 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:13 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:13 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.341+0000 7f4741536700 1 -- 172.21.15.39:0/1372084959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 msgr2=0x7f473c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:14.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.341+0000 7f4741536700 1 --2- 172.21.15.39:0/1372084959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 0x7f473c1024f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4730009990 tx=0x7f4730009ca0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.342+0000 7f4741536700 1 -- 172.21.15.39:0/1372084959 shutdown_connections 2024-05-24T11:38:14.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.342+0000 7f4741536700 1 --2- 172.21.15.39:0/1372084959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f473c102a30 0x7f473c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.342+0000 7f4741536700 1 --2- 172.21.15.39:0/1372084959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f473c100f20 0x7f473c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.342+0000 7f4741536700 1 --2- 172.21.15.39:0/1372084959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 0x7f473c1024f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.342+0000 7f4741536700 1 -- 172.21.15.39:0/1372084959 >> 172.21.15.39:0/1372084959 conn(0x7f473c0fc490 msgr2=0x7f473c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:14.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.342+0000 7f4741536700 1 -- 172.21.15.39:0/1372084959 shutdown_connections 2024-05-24T11:38:14.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.342+0000 7f4741536700 1 -- 172.21.15.39:0/1372084959 wait complete. 2024-05-24T11:38:14.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.342+0000 7f4741536700 1 Processor -- start 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f4741536700 1 -- start start 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f4741536700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f473c100f20 0x7f473c196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f4741536700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 0x7f473c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f4741536700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f473c102a30 0x7f473c19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f4741536700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c19e860 con 0x7f473c102070 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f4741536700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c19e9d0 con 0x7f473c102a30 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f4741536700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c19eb90 con 0x7f473c100f20 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 0x7f473c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 0x7f473c1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33162/0 (socket says 172.21.15.39:33162) 2024-05-24T11:38:14.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473a7fc700 1 -- 172.21.15.39:0/2175279206 learned_addr learned my addr 172.21.15.39:0/2175279206 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473b7fe700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f473c102a30 0x7f473c19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473affd700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f473c100f20 0x7f473c196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473a7fc700 1 -- 172.21.15.39:0/2175279206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f473c100f20 msgr2=0x7f473c196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473a7fc700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f473c100f20 0x7f473c196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473a7fc700 1 -- 172.21.15.39:0/2175279206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f473c102a30 msgr2=0x7f473c19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473a7fc700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f473c102a30 0x7f473c19e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473a7fc700 1 -- 172.21.15.39:0/2175279206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47300095f0 con 0x7f473c102070 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.343+0000 7f473affd700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f473c100f20 0x7f473c196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.344+0000 7f473b7fe700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f473c102a30 0x7f473c19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.344+0000 7f473a7fc700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 0x7f473c1972b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f47300059d0 tx=0x7f4730004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.344+0000 7f4723fff700 1 -- 172.21.15.39:0/2175279206 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473001d070 con 0x7f473c102070 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.344+0000 7f4723fff700 1 -- 172.21.15.39:0/2175279206 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f473000bdd0 con 0x7f473c102070 2024-05-24T11:38:14.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.344+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c19ee10 con 0x7f473c102070 2024-05-24T11:38:14.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.344+0000 7f4723fff700 1 -- 172.21.15.39:0/2175279206 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730022da0 con 0x7f473c102070 2024-05-24T11:38:14.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.344+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c19f180 con 0x7f473c102070 2024-05-24T11:38:14.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.345+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c1083e0 con 0x7f473c102070 2024-05-24T11:38:14.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.346+0000 7f4723fff700 1 -- 172.21.15.39:0/2175279206 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f4730022570 con 0x7f473c102070 2024-05-24T11:38:14.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.347+0000 7f4723fff700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f47240a0f20 0x7f47240a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:14.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.347+0000 7f4723fff700 1 -- 172.21.15.39:0/2175279206 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f47300ce0b0 con 0x7f473c102070 2024-05-24T11:38:14.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.347+0000 7f473affd700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f47240a0f20 0x7f47240a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:14.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.347+0000 7f473affd700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f47240a0f20 0x7f47240a33e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f473c1981f0 tx=0x7f472c009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:14.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.350+0000 7f4723fff700 1 -- 172.21.15.39:0/2175279206 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4730097b70 con 0x7f473c102070 2024-05-24T11:38:14.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.605+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 --> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f473c02ce10 con 0x7f47240a0f20 2024-05-24T11:38:14.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.612+0000 7f4723fff700 1 -- 172.21.15.39:0/2175279206 <== mgr.15360 v2:172.21.15.39:6800/1882625876 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f473c02ce10 con 0x7f47240a0f20 2024-05-24T11:38:14.617 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1m 6m3h 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f47240a0f20 msgr2=0x7f47240a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f47240a0f20 0x7f47240a33e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f473c1981f0 tx=0x7f472c009260 comp rx=0 tx=0).stop 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 msgr2=0x7f473c1972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 0x7f473c1972b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f47300059d0 tx=0x7f4730004730 comp rx=0 tx=0).stop 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 shutdown_connections 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f473c100f20 0x7f473c196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f47240a0f20 0x7f47240a33e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f473c102a30 0x7f473c19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 --2- 172.21.15.39:0/2175279206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f473c102070 0x7f473c1972b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 >> 172.21.15.39:0/2175279206 conn(0x7f473c0fc490 msgr2=0x7f473c105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 shutdown_connections 2024-05-24T11:38:14.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:38:14.615+0000 7f4741536700 1 -- 172.21.15.39:0/2175279206 wait complete. 2024-05-24T11:38:14.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-24T11:38:14.663 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-24T11:38:14.672 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-24T11:38:14.685 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:38:14.685 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-24T11:38:14.705 INFO:teuthology.run_tasks:Running task exec... 2024-05-24T11:38:14.713 INFO:teuthology.task.exec:Executing custom commands... 2024-05-24T11:38:14.714 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:38:14.714 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-24T11:38:14.759 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-05-24T11:38:14.773 INFO:teuthology.orchestra.run.smithi039.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-24T11:38:14.773 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-fuse 340G 0 340G 0% /tmp/cephfs 2024-05-24T11:38:14.775 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-24T11:38:14.836 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-24T11:38:14.883 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-24T11:38:14.886 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.887 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:38:14.888 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:14 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:38:15.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:15.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:38:15.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:14 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:38:15.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:15.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:38:15.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:15.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:15.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:38:15.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:38:15.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:14 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:38:16.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:16.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi137 to 182.4M 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Unable to set osd_memory_target on smithi137 to 191356211: error parsing value: Value '191356211' is below minimum 939524096 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi039 to 182.5M 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Unable to set osd_memory_target on smithi039 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Updating smithi137:/etc/ceph/ceph.conf 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:15 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:16.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:38:16.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi137 to 182.4M 2024-05-24T11:38:16.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Unable to set osd_memory_target on smithi137 to 191356211: error parsing value: Value '191356211' is below minimum 939524096 2024-05-24T11:38:16.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi039 to 182.5M 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Unable to set osd_memory_target on smithi039 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Updating smithi137:/etc/ceph/ceph.conf 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-05-24T11:38:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:15 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:16.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:38:16.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi137 to 182.4M 2024-05-24T11:38:16.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Unable to set osd_memory_target on smithi137 to 191356211: error parsing value: Value '191356211' is below minimum 939524096 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi039 to 182.5M 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Unable to set osd_memory_target on smithi039 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Updating smithi053:/etc/ceph/ceph.conf 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Updating smithi137:/etc/ceph/ceph.conf 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-05-24T11:38:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:16.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:15 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:17.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:38:17.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:38:17.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:38:17.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:38:17.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:38:17.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:38:17.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:38:17.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:38:17.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:38:17.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:16 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:17.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:38:17.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:38:17.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:38:17.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:38:17.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:38:17.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:38:17.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:38:17.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:38:17.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:38:17.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:16 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:17.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:38:17.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:38:17.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-05-24T11:38:17.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: Updating smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:38:17.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: Updating smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:38:17.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: Updating smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-24T11:38:17.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:38:17.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:38:17.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:38:17.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:16 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:38:17.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-24T11:38:17.721 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-24T11:38:17.724 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-24T11:38:17.806 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-24T11:38:17.891 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/: 2024-05-24T11:38:17.892 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-24T11:38:17.892 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:17.892 INFO:teuthology.orchestra.run.smithi039.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 .. 2024-05-24T11:38:17.892 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 data 2024-05-24T11:38:17.892 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:17.892 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data: 2024-05-24T11:38:17.892 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:17.892 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:17.892 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 .. 2024-05-24T11:38:17.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-24T11:38:17.942 INFO:teuthology.orchestra.run.smithi039.stdout:[Service] 2024-05-24T11:38:17.943 INFO:teuthology.orchestra.run.smithi039.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-24T11:38:17.943 INFO:teuthology.orchestra.run.smithi039.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-24T11:38:17.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-24T11:38:18.027 INFO:teuthology.orchestra.run.smithi039.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-24T11:38:18.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:17 smithi053 ceph-mon[47448]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:38:18.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:17 smithi053 ceph-mon[47448]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-24T11:38:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:17 smithi137 ceph-mon[47907]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:38:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:17 smithi137 ceph-mon[47907]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-24T11:38:18.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:17 smithi039 ceph-mon[42919]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-24T11:38:18.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:17 smithi039 ceph-mon[42919]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-24T11:38:20.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:19 smithi053 ceph-mon[47448]: pgmap v6: 97 pgs: 97 active+clean; 1.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 7.8 KiB/s wr, 4 op/s 2024-05-24T11:38:20.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:19 smithi137 ceph-mon[47907]: pgmap v6: 97 pgs: 97 active+clean; 1.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 7.8 KiB/s wr, 4 op/s 2024-05-24T11:38:20.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:19 smithi039 ceph-mon[42919]: pgmap v6: 97 pgs: 97 active+clean; 1.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 7.8 KiB/s wr, 4 op/s 2024-05-24T11:38:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:21 smithi053 ceph-mon[47448]: pgmap v7: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 154 KiB/s wr, 82 op/s 2024-05-24T11:38:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:21 smithi137 ceph-mon[47907]: pgmap v7: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 154 KiB/s wr, 82 op/s 2024-05-24T11:38:22.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:21 smithi039 ceph-mon[42919]: pgmap v7: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 154 KiB/s wr, 82 op/s 2024-05-24T11:38:24.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:23 smithi053 ceph-mon[47448]: pgmap v8: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 120 KiB/s wr, 63 op/s 2024-05-24T11:38:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:23 smithi137 ceph-mon[47907]: pgmap v8: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 120 KiB/s wr, 63 op/s 2024-05-24T11:38:24.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:23 smithi039 ceph-mon[42919]: pgmap v8: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 120 KiB/s wr, 63 op/s 2024-05-24T11:38:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:25 smithi053 ceph-mon[47448]: pgmap v9: 97 pgs: 97 active+clean; 33 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 3.7 MiB/s wr, 136 op/s 2024-05-24T11:38:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:25 smithi137 ceph-mon[47907]: pgmap v9: 97 pgs: 97 active+clean; 33 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 3.7 MiB/s wr, 136 op/s 2024-05-24T11:38:26.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:25 smithi039 ceph-mon[42919]: pgmap v9: 97 pgs: 97 active+clean; 33 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 3.7 MiB/s wr, 136 op/s 2024-05-24T11:38:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 -- 172.21.15.39:0/3012627793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81010f0 msgr2=0x7f3fd81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/3012627793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81010f0 0x7f3fd81014d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009990 tx=0x7f3fd4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 -- 172.21.15.39:0/3012627793 shutdown_connections 2024-05-24T11:38:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/3012627793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd8106210 0x7f3fd810dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/3012627793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81010f0 0x7f3fd81014d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/3012627793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8101aa0 0x7f3fd8105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 -- 172.21.15.39:0/3012627793 >> 172.21.15.39:0/3012627793 conn(0x7f3fd80fc9d0 msgr2=0x7f3fd80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 -- 172.21.15.39:0/3012627793 shutdown_connections 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 -- 172.21.15.39:0/3012627793 wait complete. 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.875+0000 7f3fdf6db700 1 Processor -- start 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdf6db700 1 -- start start 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdf6db700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81010f0 0x7f3fd81972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdf6db700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8106210 0x7f3fd8197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdf6db700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd819bb70 0x7f3fd819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdf6db700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd819c5f0 con 0x7f3fd8106210 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdf6db700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd819c760 con 0x7f3fd81010f0 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdf6db700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd819c920 con 0x7f3fd819bb70 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdcc76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8106210 0x7f3fd8197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdcc76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8106210 0x7f3fd8197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49074/0 (socket says 172.21.15.39:49074) 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdcc76700 1 -- 172.21.15.39:0/1920364955 learned_addr learned my addr 172.21.15.39:0/1920364955 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdd477700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81010f0 0x7f3fd81972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fddc78700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd819bb70 0x7f3fd819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdcc76700 1 -- 172.21.15.39:0/1920364955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd819bb70 msgr2=0x7f3fd819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd819bb70 0x7f3fd819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdcc76700 1 -- 172.21.15.39:0/1920364955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81010f0 msgr2=0x7f3fd81972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81010f0 0x7f3fd81972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.876+0000 7f3fdcc76700 1 -- 172.21.15.39:0/1920364955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd40095f0 con 0x7f3fd8106210 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.877+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8106210 0x7f3fd8197820 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00d980 tx=0x7f3fcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.877+0000 7f3fca7fc700 1 -- 172.21.15.39:0/1920364955 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc00f890 con 0x7f3fd8106210 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.877+0000 7f3fca7fc700 1 -- 172.21.15.39:0/1920364955 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fcc00fe80 con 0x7f3fd8106210 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.877+0000 7f3fdf6db700 1 -- 172.21.15.39:0/1920364955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd81a3e80 con 0x7f3fd8106210 2024-05-24T11:38:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.877+0000 7f3fca7fc700 1 -- 172.21.15.39:0/1920364955 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc00e3b0 con 0x7f3fd8106210 2024-05-24T11:38:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.877+0000 7f3fdf6db700 1 -- 172.21.15.39:0/1920364955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd81a4350 con 0x7f3fd8106210 2024-05-24T11:38:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.879+0000 7f3fca7fc700 1 -- 172.21.15.39:0/1920364955 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f3fcc00f9f0 con 0x7f3fd8106210 2024-05-24T11:38:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.880+0000 7f3fca7fc700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f3fc40a13a0 0x7f3fc40a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.881+0000 7f3fdd477700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f3fc40a13a0 0x7f3fc40a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.881+0000 7f3fca7fc700 1 -- 172.21.15.39:0/1920364955 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3fcc0cc840 con 0x7f3fd8106210 2024-05-24T11:38:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.881+0000 7f3fdd477700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f3fc40a13a0 0x7f3fc40a3860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400b5c0 tx=0x7f3fd4005fb0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3feb896b80 1 Processor -- start 2024-05-24T11:38:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3feb896b80 1 -- start start 2024-05-24T11:38:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d462a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d4629bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 0x559d4629c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3feb896b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d461f30f0 con 0x559d462a2270 2024-05-24T11:38:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3feb896b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d461f0ab0 con 0x559d4629c480 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3feb896b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d461ec3a0 con 0x559d462a1950 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3fdcc76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d4629bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3fdd477700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d462a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3fdcc76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d4629bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49086/0 (socket says 172.21.15.39:49086) 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3fdcc76700 1 -- 172.21.15.39:0/3322259552 learned_addr learned my addr 172.21.15.39:0/3322259552 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.882+0000 7f3fddc78700 1 --2- 172.21.15.39:0/3322259552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 0x559d4629c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fdcc76700 1 -- 172.21.15.39:0/3322259552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 msgr2=0x559d462a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/3322259552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d462a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fdcc76700 1 -- 172.21.15.39:0/3322259552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 msgr2=0x559d4629c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/3322259552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 0x559d4629c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fdcc76700 1 -- 172.21.15.39:0/3322259552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d4606d350 con 0x559d462a2270 2024-05-24T11:38:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fdd477700 1 --2- 172.21.15.39:0/3322259552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d462a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/3322259552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d4629bf40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0c9fa0 tx=0x7f3fcc099220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f3e116b0ae3c2e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fc37fe700 1 -- 172.21.15.39:0/3322259552 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc099f20 con 0x559d462a2270 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fc37fe700 1 -- 172.21.15.39:0/3322259552 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fcc099820 con 0x559d462a2270 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.883+0000 7f3fc37fe700 1 -- 172.21.15.39:0/3322259552 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc099a80 con 0x559d462a2270 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 -- 172.21.15.39:0/3322259552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 msgr2=0x559d4629bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 --2- 172.21.15.39:0/3322259552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d4629bf40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0c9fa0 tx=0x7f3fcc099220 comp rx=0 tx=0).stop 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 -- 172.21.15.39:0/3322259552 shutdown_connections 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 --2- 172.21.15.39:0/3322259552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d462a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 --2- 172.21.15.39:0/3322259552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 0x559d4629c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 --2- 172.21.15.39:0/3322259552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d4629bf40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 -- 172.21.15.39:0/3322259552 >> 172.21.15.39:0/3322259552 conn(0x559d461fa1b0 msgr2=0x559d46296840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 -- 172.21.15.39:0/3322259552 shutdown_connections 2024-05-24T11:38:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 -- 172.21.15.39:0/3322259552 wait complete. 2024-05-24T11:38:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 -- start start 2024-05-24T11:38:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.884+0000 7f3feb896b80 1 Processor -- start 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4629c480 0x559d462a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a1950 0x559d462a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 0x559d462a6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3feb896b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d46188500 con 0x559d462a1950 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3feb896b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d461881a0 con 0x559d462a2270 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3feb896b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4606f640 con 0x559d4629c480 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3feb896b80 10 client.0 _ll_register_callbacks cb 0x559d46191ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fdcc76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a1950 0x559d462a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fdcc76700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a1950 0x559d462a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49094/0 (socket says 172.21.15.39:49094) 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fdcc76700 1 -- 172.21.15.39:0/2013632668 learned_addr learned my addr 172.21.15.39:0/2013632668 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fddc78700 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 0x559d462a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fdd477700 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4629c480 0x559d462a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fdcc76700 1 -- 172.21.15.39:0/2013632668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4629c480 msgr2=0x559d462a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4629c480 0x559d462a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fdcc76700 1 -- 172.21.15.39:0/2013632668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 msgr2=0x559d462a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 0x559d462a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.885+0000 7f3fdcc76700 1 -- 172.21.15.39:0/2013632668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd810e600 con 0x559d462a1950 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a1950 0x559d462a5af0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc099520 tx=0x7f3fcc095c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5650d929528061a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3fc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2013632668 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc09d650 con 0x559d462a1950 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2013632668 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3fcc0a1ca0 con 0x559d462a1950 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3feb896b80 1 -- 172.21.15.39:0/2013632668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d46145210 con 0x559d462a1950 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2013632668 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fcc0b90c0 con 0x559d462a1950 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3feb896b80 10 client.15459 fetch_fsmap learned FSMap version 27 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3feb896b80 1 -- 172.21.15.39:0/2013632668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3fd81a4350 con 0x559d462a1950 2024-05-24T11:38:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2013632668 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d46145210 con 0x559d462a1950 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2013632668 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3fcc099a80 con 0x559d462a1950 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3feb896b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3feb896b80 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3feb896b80 20 client.15459 populate_metadata read hostname 'smithi039' 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.886+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559d462b5780 0x559d462b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.887+0000 7f3feb896b80 4 client.15459 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.887+0000 7f3feb896b80 1 -- 172.21.15.39:0/2013632668 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d4626b7b0 con 0x559d462b5780 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.887+0000 7f3fdd477700 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559d462b5780 0x559d462b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.887+0000 7f3fdd477700 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559d462b5780 0x559d462b7ba0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4222cb14ec329e5 server_cookie=c490a5da56b1d921 in_seq=0 out_seq=0 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.887+0000 7f3fc0ff9700 10 client.15459 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2013632668 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d4626b7b0 con 0x559d462b5780 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3fc0ff9700 10 client.15459 handle_command_reply: tid=0 2024-05-24T11:38:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 client.15459 shutdown 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 -- 172.21.15.39:0/2013632668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a1950 msgr2=0x559d462a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a1950 0x559d462a5af0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc099520 tx=0x7f3fcc095c50 comp rx=0 tx=0).stop 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 -- 172.21.15.39:0/2013632668 shutdown_connections 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d4629c480 0x559d462a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559d462b5780 0x559d462b7ba0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 0x559d462a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2013632668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a1950 0x559d462a5af0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3fc0ff9700 0 client.15459 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 -- 172.21.15.39:0/2013632668 >> 172.21.15.39:0/2013632668 conn(0x559d461fa1b0 msgr2=0x559d462a04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 -- 172.21.15.39:0/2013632668 shutdown_connections 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 1 -- 172.21.15.39:0/2013632668 wait complete. 2024-05-24T11:38:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.889+0000 7f3feb896b80 20 client.15459 trim_cache size 0 max 16384 2024-05-24T11:38:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.907+0000 7f3feb896b80 1 Processor -- start 2024-05-24T11:38:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.907+0000 7f3feb896b80 1 -- start start 2024-05-24T11:38:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.907+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 0x559d462a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.907+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d46296910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.907+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d46298540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.907+0000 7f3feb896b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d462a3b00 con 0x559d462a2270 2024-05-24T11:38:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.907+0000 7f3feb896b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4629b1a0 con 0x559d4629c480 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.907+0000 7f3feb896b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4629af00 con 0x559d462a1950 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fddc78700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d46298540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fddc78700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d46298540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49100/0 (socket says 172.21.15.39:49100) 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fddc78700 1 -- 172.21.15.39:0/4127201397 learned_addr learned my addr 172.21.15.39:0/4127201397 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fdd477700 1 --2- 172.21.15.39:0/4127201397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 0x559d462a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/4127201397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d46296910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fddc78700 1 -- 172.21.15.39:0/4127201397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 msgr2=0x559d46296910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fddc78700 1 --2- 172.21.15.39:0/4127201397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d46296910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fddc78700 1 -- 172.21.15.39:0/4127201397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 msgr2=0x559d462a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fddc78700 1 --2- 172.21.15.39:0/4127201397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 0x559d462a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fddc78700 1 -- 172.21.15.39:0/4127201397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d46019b00 con 0x559d462a2270 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fddc78700 1 --2- 172.21.15.39:0/4127201397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d46298540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000a780 tx=0x7f3fd00078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3110603b0ffab7ae server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fc3fff700 1 -- 172.21.15.39:0/4127201397 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000f040 con 0x559d462a2270 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fc3fff700 1 -- 172.21.15.39:0/4127201397 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fd0010040 con 0x559d462a2270 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.908+0000 7f3fc3fff700 1 -- 172.21.15.39:0/4127201397 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0017400 con 0x559d462a2270 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 -- 172.21.15.39:0/4127201397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 msgr2=0x559d46298540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 --2- 172.21.15.39:0/4127201397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d46298540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000a780 tx=0x7f3fd00078d0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 -- 172.21.15.39:0/4127201397 shutdown_connections 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 --2- 172.21.15.39:0/4127201397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d462a2270 0x559d46298540 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 --2- 172.21.15.39:0/4127201397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d4629c480 0x559d462a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 --2- 172.21.15.39:0/4127201397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d46296910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 -- 172.21.15.39:0/4127201397 >> 172.21.15.39:0/4127201397 conn(0x559d461fa1b0 msgr2=0x559d4629b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 -- 172.21.15.39:0/4127201397 shutdown_connections 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 -- 172.21.15.39:0/4127201397 wait complete. 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 -- start start 2024-05-24T11:38:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.909+0000 7f3feb896b80 1 Processor -- start 2024-05-24T11:38:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d4629c480 0x559d462acf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d462a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3feb896b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 0x559d462a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3feb896b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d462a97a0 con 0x559d4629c480 2024-05-24T11:38:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3feb896b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d462a9910 con 0x559d462a2270 2024-05-24T11:38:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3feb896b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4630ac30 con 0x559d462a1950 2024-05-24T11:38:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3feb896b80 10 client.0 _ll_register_callbacks cb 0x559d46191ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:38:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdd477700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d4629c480 0x559d462acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdd477700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d4629c480 0x559d462acf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49104/0 (socket says 172.21.15.39:49104) 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdd477700 1 -- 172.21.15.39:0/2168736034 learned_addr learned my addr 172.21.15.39:0/2168736034 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fddc78700 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 0x559d462a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d462a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdd477700 1 -- 172.21.15.39:0/2168736034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 msgr2=0x559d462a8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdd477700 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d462a8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdd477700 1 -- 172.21.15.39:0/2168736034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 msgr2=0x559d462a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdd477700 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 0x559d462a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdd477700 1 -- 172.21.15.39:0/2168736034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd000a430 con 0x559d4629c480 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fdd477700 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d4629c480 0x559d462acf10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4015000 tx=0x7f3fd4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7cb282c16500b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.910+0000 7f3fc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2168736034 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400be80 con 0x559d4629c480 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2168736034 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f3fd4004890 con 0x559d4629c480 2024-05-24T11:38:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3feb896b80 1 -- 172.21.15.39:0/2168736034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fcc099820 con 0x559d4629c480 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2168736034 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd4004a90 con 0x559d4629c480 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3feb896b80 10 client.15465 fetch_fsmap learned FSMap version 27 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3feb896b80 1 -- 172.21.15.39:0/2168736034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559d461b2500 con 0x559d4629c480 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2168736034 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fcc099820 con 0x559d4629c480 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2168736034 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3fd403b020 con 0x559d4629c480 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3feb896b80 10 client.15465 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3feb896b80 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3feb896b80 20 client.15465 populate_metadata read hostname 'smithi039' 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559d4630c1f0 0x559d4630e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3feb896b80 4 client.15465 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3feb896b80 1 -- 172.21.15.39:0/2168736034 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d461e4e60 con 0x559d4630c1f0 2024-05-24T11:38:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.911+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559d4630c1f0 0x559d4630e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.912+0000 7f3fc0ff9700 10 client.15465 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:27.912+0000 7f3fdcc76700 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559d4630c1f0 0x559d4630e610 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f40403c9171cbf89 server_cookie=14fdfaab88b08f42 in_seq=0 out_seq=0 2024-05-24T11:38:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.127+0000 7f3fc0ff9700 1 -- 172.21.15.39:0/2168736034 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d461e4e60 con 0x559d4630c1f0 2024-05-24T11:38:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.127+0000 7f3fc0ff9700 10 client.15465 handle_command_reply: tid=0 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 client.15465 shutdown 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 -- 172.21.15.39:0/2168736034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d4629c480 msgr2=0x559d462acf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d4629c480 0x559d462acf10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4015000 tx=0x7f3fd4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 -- 172.21.15.39:0/2168736034 shutdown_connections 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559d4630c1f0 0x559d4630e610 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559d462a2270 0x559d462a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d4629c480 0x559d462acf10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 --2- 172.21.15.39:0/2168736034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d462a1950 0x559d462a8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3fc0ff9700 0 client.15465 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 -- 172.21.15.39:0/2168736034 >> 172.21.15.39:0/2168736034 conn(0x559d461fa1b0 msgr2=0x559d462a3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 -- 172.21.15.39:0/2168736034 shutdown_connections 2024-05-24T11:38:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 1 -- 172.21.15.39:0/2168736034 wait complete. 2024-05-24T11:38:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.128+0000 7f3feb896b80 20 client.15465 trim_cache size 0 max 16384 2024-05-24T11:38:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:38:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:38:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:38:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:38:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 -- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f3fc40a13a0 msgr2=0x7f3fc40a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f3fc40a13a0 0x7f3fc40a3860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400b5c0 tx=0x7f3fd4005fb0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 -- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8106210 msgr2=0x7f3fd8197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8106210 0x7f3fd8197820 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00d980 tx=0x7f3fcc00dc90 comp rx=0 tx=0).stop 2024-05-24T11:38:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 -- 172.21.15.39:0/1920364955 shutdown_connections 2024-05-24T11:38:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd819bb70 0x7f3fd819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f3fc40a13a0 0x7f3fc40a3860 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd81010f0 0x7f3fd81972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 --2- 172.21.15.39:0/1920364955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd8106210 0x7f3fd8197820 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 -- 172.21.15.39:0/1920364955 >> 172.21.15.39:0/1920364955 conn(0x7f3fd80fc9d0 msgr2=0x7f3fd80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 -- 172.21.15.39:0/1920364955 shutdown_connections 2024-05-24T11:38:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:28.129+0000 7f3fdf6db700 1 -- 172.21.15.39:0/1920364955 wait complete. 2024-05-24T11:38:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:27 smithi053 ceph-mon[47448]: pgmap v10: 97 pgs: 97 active+clean; 33 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.4 MiB/s wr, 125 op/s 2024-05-24T11:38:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:27 smithi137 ceph-mon[47907]: pgmap v10: 97 pgs: 97 active+clean; 33 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.4 MiB/s wr, 125 op/s 2024-05-24T11:38:28.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:27 smithi039 ceph-mon[42919]: pgmap v10: 97 pgs: 97 active+clean; 33 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.4 MiB/s wr, 125 op/s 2024-05-24T11:38:30.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:29 smithi053 ceph-mon[47448]: pgmap v11: 97 pgs: 97 active+clean; 37 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-05-24T11:38:30.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:29 smithi137 ceph-mon[47907]: pgmap v11: 97 pgs: 97 active+clean; 37 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-05-24T11:38:30.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:29 smithi039 ceph-mon[42919]: pgmap v11: 97 pgs: 97 active+clean; 37 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-05-24T11:38:32.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:31 smithi053 ceph-mon[47448]: pgmap v12: 97 pgs: 97 active+clean; 39 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 4.3 MiB/s wr, 301 op/s 2024-05-24T11:38:32.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:31 smithi137 ceph-mon[47907]: pgmap v12: 97 pgs: 97 active+clean; 39 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 4.3 MiB/s wr, 301 op/s 2024-05-24T11:38:32.287 INFO:teuthology.orchestra.run.smithi039.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-24T11:38:32.288 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:31 smithi039 ceph-mon[42919]: pgmap v12: 97 pgs: 97 active+clean; 39 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 4.3 MiB/s wr, 301 op/s 2024-05-24T11:38:32.290 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-24T11:38:32.636 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/: 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 . 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 data 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data: 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:total 40 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 . 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 May 24 11:38 base 2024-05-24T11:38:32.637 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 565760 May 24 11:38 global 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 log 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_commit_ts 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_dynshmem 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 May 24 11:38 pg_hba.conf 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 24 11:38 pg_ident.conf 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:38 pg_logical 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 pg_multixact 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 pg_notify 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_replslot 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_serial 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_snapshots 2024-05-24T11:38:32.638 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_stat 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_stat_tmp 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 pg_subtrans 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_tblspc 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_twophase 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 24 11:38 PG_VERSION 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:38 pg_wal 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 pg_xact 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 24 11:38 postgresql.auto.conf 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 May 24 11:38 postgresql.conf 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base: 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:total 3 2024-05-24T11:38:32.639 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 May 24 11:38 . 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 1 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 12346 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 12347 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:total 7202 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 . 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 May 24 11:38 .. 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 112 2024-05-24T11:38:32.640 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 113 2024-05-24T11:38:32.641 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 24 11:38 12182 2024-05-24T11:38:32.641 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12182_fsm 2024-05-24T11:38:32.641 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12182_vm 2024-05-24T11:38:32.641 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12184 2024-05-24T11:38:32.641 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12186 2024-05-24T11:38:32.641 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187 2024-05-24T11:38:32.641 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12187_fsm 2024-05-24T11:38:32.641 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187_vm 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12189 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12191 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12192_fsm 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192_vm 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12194 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12196 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12197_fsm 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197_vm 2024-05-24T11:38:32.642 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12199 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12201 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12202_fsm 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202_vm 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12204 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12206 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12207_fsm 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207_vm 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12209 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12211 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12212 2024-05-24T11:38:32.643 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12214 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12216 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 24 11:38 1247 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1247_fsm 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1247_vm 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 24 11:38 1249 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1249_fsm 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1249_vm 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 24 11:38 1255 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1255_fsm 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1255_vm 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 24 11:38 1259 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1259_fsm 2024-05-24T11:38:32.644 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1259_vm 2024-05-24T11:38:32.645 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417 2024-05-24T11:38:32.645 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417_vm 2024-05-24T11:38:32.645 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418 2024-05-24T11:38:32.645 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418_vm 2024-05-24T11:38:32.645 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 174 2024-05-24T11:38:32.645 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 175 2024-05-24T11:38:32.646 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2187 2024-05-24T11:38:32.646 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224 2024-05-24T11:38:32.646 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224_vm 2024-05-24T11:38:32.646 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328 2024-05-24T11:38:32.646 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328_vm 2024-05-24T11:38:32.646 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336 2024-05-24T11:38:32.647 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336_vm 2024-05-24T11:38:32.647 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2337 2024-05-24T11:38:32.647 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2600 2024-05-24T11:38:32.647 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2600_fsm 2024-05-24T11:38:32.647 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2600_vm 2024-05-24T11:38:32.647 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601 2024-05-24T11:38:32.648 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2601_fsm 2024-05-24T11:38:32.648 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601_vm 2024-05-24T11:38:32.648 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2602 2024-05-24T11:38:32.648 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2602_fsm 2024-05-24T11:38:32.648 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2602_vm 2024-05-24T11:38:32.648 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2603 2024-05-24T11:38:32.649 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2603_fsm 2024-05-24T11:38:32.649 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2603_vm 2024-05-24T11:38:32.649 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604 2024-05-24T11:38:32.649 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604_vm 2024-05-24T11:38:32.649 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2605 2024-05-24T11:38:32.649 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2605_fsm 2024-05-24T11:38:32.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2605_vm 2024-05-24T11:38:32.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606 2024-05-24T11:38:32.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2606_fsm 2024-05-24T11:38:32.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606_vm 2024-05-24T11:38:32.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607 2024-05-24T11:38:32.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607_fsm 2024-05-24T11:38:32.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2607_vm 2024-05-24T11:38:32.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 24 11:38 2608 2024-05-24T11:38:32.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2608_fsm 2024-05-24T11:38:32.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2608_vm 2024-05-24T11:38:32.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 24 11:38 2609 2024-05-24T11:38:32.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2609_fsm 2024-05-24T11:38:32.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2609_vm 2024-05-24T11:38:32.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610 2024-05-24T11:38:32.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610_fsm 2024-05-24T11:38:32.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2610_vm 2024-05-24T11:38:32.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611 2024-05-24T11:38:32.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611_vm 2024-05-24T11:38:32.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612 2024-05-24T11:38:32.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2612_fsm 2024-05-24T11:38:32.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612_vm 2024-05-24T11:38:32.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613 2024-05-24T11:38:32.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613_vm 2024-05-24T11:38:32.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615 2024-05-24T11:38:32.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2615_fsm 2024-05-24T11:38:32.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615_vm 2024-05-24T11:38:32.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616 2024-05-24T11:38:32.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616_fsm 2024-05-24T11:38:32.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2616_vm 2024-05-24T11:38:32.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 24 11:38 2617 2024-05-24T11:38:32.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2617_fsm 2024-05-24T11:38:32.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2617_vm 2024-05-24T11:38:32.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 24 11:38 2618 2024-05-24T11:38:32.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2618_fsm 2024-05-24T11:38:32.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2618_vm 2024-05-24T11:38:32.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 24 11:38 2619 2024-05-24T11:38:32.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2619_fsm 2024-05-24T11:38:32.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2619_vm 2024-05-24T11:38:32.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620 2024-05-24T11:38:32.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620_vm 2024-05-24T11:38:32.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2650 2024-05-24T11:38:32.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2651 2024-05-24T11:38:32.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2652 2024-05-24T11:38:32.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2653 2024-05-24T11:38:32.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2654 2024-05-24T11:38:32.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2655 2024-05-24T11:38:32.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2656 2024-05-24T11:38:32.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2657 2024-05-24T11:38:32.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 24 11:38 2658 2024-05-24T11:38:32.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2659 2024-05-24T11:38:32.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2660 2024-05-24T11:38:32.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2661 2024-05-24T11:38:32.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2662 2024-05-24T11:38:32.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2663 2024-05-24T11:38:32.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2664 2024-05-24T11:38:32.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2665 2024-05-24T11:38:32.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2666 2024-05-24T11:38:32.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2667 2024-05-24T11:38:32.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2668 2024-05-24T11:38:32.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2669 2024-05-24T11:38:32.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2670 2024-05-24T11:38:32.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 May 24 11:38 2673 2024-05-24T11:38:32.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 May 24 11:38 2674 2024-05-24T11:38:32.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 24 11:38 2675 2024-05-24T11:38:32.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2678 2024-05-24T11:38:32.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2679 2024-05-24T11:38:32.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2680 2024-05-24T11:38:32.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2681 2024-05-24T11:38:32.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2682 2024-05-24T11:38:32.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2683 2024-05-24T11:38:32.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2684 2024-05-24T11:38:32.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2685 2024-05-24T11:38:32.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2686 2024-05-24T11:38:32.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2687 2024-05-24T11:38:32.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2688 2024-05-24T11:38:32.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2689 2024-05-24T11:38:32.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2690 2024-05-24T11:38:32.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 24 11:38 2691 2024-05-24T11:38:32.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2692 2024-05-24T11:38:32.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2693 2024-05-24T11:38:32.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2696 2024-05-24T11:38:32.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2699 2024-05-24T11:38:32.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2701 2024-05-24T11:38:32.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2702 2024-05-24T11:38:32.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2703 2024-05-24T11:38:32.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2704 2024-05-24T11:38:32.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2753 2024-05-24T11:38:32.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2753_fsm 2024-05-24T11:38:32.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2753_vm 2024-05-24T11:38:32.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2754 2024-05-24T11:38:32.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2755 2024-05-24T11:38:32.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2756 2024-05-24T11:38:32.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2757 2024-05-24T11:38:32.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830 2024-05-24T11:38:32.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830_vm 2024-05-24T11:38:32.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2831 2024-05-24T11:38:32.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832 2024-05-24T11:38:32.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832_vm 2024-05-24T11:38:32.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2833 2024-05-24T11:38:32.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834 2024-05-24T11:38:32.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834_vm 2024-05-24T11:38:32.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2835 2024-05-24T11:38:32.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836 2024-05-24T11:38:32.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836_vm 2024-05-24T11:38:32.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2837 2024-05-24T11:38:32.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 24 11:38 2838 2024-05-24T11:38:32.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2838_fsm 2024-05-24T11:38:32.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2838_vm 2024-05-24T11:38:32.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2839 2024-05-24T11:38:32.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840 2024-05-24T11:38:32.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840_fsm 2024-05-24T11:38:32.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2840_vm 2024-05-24T11:38:32.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2841 2024-05-24T11:38:32.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995 2024-05-24T11:38:32.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995_vm 2024-05-24T11:38:32.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2996 2024-05-24T11:38:32.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079 2024-05-24T11:38:32.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3079_fsm 2024-05-24T11:38:32.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079_vm 2024-05-24T11:38:32.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3080 2024-05-24T11:38:32.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3081 2024-05-24T11:38:32.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3085 2024-05-24T11:38:32.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118 2024-05-24T11:38:32.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118_vm 2024-05-24T11:38:32.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3119 2024-05-24T11:38:32.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3164 2024-05-24T11:38:32.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256 2024-05-24T11:38:32.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256_vm 2024-05-24T11:38:32.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3257 2024-05-24T11:38:32.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3258 2024-05-24T11:38:32.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350 2024-05-24T11:38:32.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350_vm 2024-05-24T11:38:32.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3351 2024-05-24T11:38:32.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3379 2024-05-24T11:38:32.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3380 2024-05-24T11:38:32.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381 2024-05-24T11:38:32.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381_vm 2024-05-24T11:38:32.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3394 2024-05-24T11:38:32.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3394_fsm 2024-05-24T11:38:32.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3394_vm 2024-05-24T11:38:32.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3395 2024-05-24T11:38:32.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439 2024-05-24T11:38:32.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439_vm 2024-05-24T11:38:32.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3440 2024-05-24T11:38:32.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 3455 2024-05-24T11:38:32.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3456 2024-05-24T11:38:32.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3456_fsm 2024-05-24T11:38:32.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3456_vm 2024-05-24T11:38:32.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466 2024-05-24T11:38:32.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466_vm 2024-05-24T11:38:32.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3467 2024-05-24T11:38:32.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3468 2024-05-24T11:38:32.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501 2024-05-24T11:38:32.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501_vm 2024-05-24T11:38:32.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3502 2024-05-24T11:38:32.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3503 2024-05-24T11:38:32.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3534 2024-05-24T11:38:32.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541 2024-05-24T11:38:32.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3541_fsm 2024-05-24T11:38:32.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541_vm 2024-05-24T11:38:32.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3542 2024-05-24T11:38:32.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3574 2024-05-24T11:38:32.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3575 2024-05-24T11:38:32.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576 2024-05-24T11:38:32.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576_vm 2024-05-24T11:38:32.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596 2024-05-24T11:38:32.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596_vm 2024-05-24T11:38:32.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3597 2024-05-24T11:38:32.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598 2024-05-24T11:38:32.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598_vm 2024-05-24T11:38:32.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3599 2024-05-24T11:38:32.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600 2024-05-24T11:38:32.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3600_fsm 2024-05-24T11:38:32.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600_vm 2024-05-24T11:38:32.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601 2024-05-24T11:38:32.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3601_fsm 2024-05-24T11:38:32.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601_vm 2024-05-24T11:38:32.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602 2024-05-24T11:38:32.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3602_fsm 2024-05-24T11:38:32.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602_vm 2024-05-24T11:38:32.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3603 2024-05-24T11:38:32.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3603_fsm 2024-05-24T11:38:32.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3603_vm 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3604 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3605 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3606 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3607 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3608 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 3609 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3712 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3764_fsm 2024-05-24T11:38:32.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764_vm 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3766 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3767 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3997 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 5002 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 548 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 549 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102_vm 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104_vm 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106 2024-05-24T11:38:32.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106_vm 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6110 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6111 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6112 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6113 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6117 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826_vm 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 827 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 828 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 24 11:38 pg_filenode.map 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 24 11:38 PG_VERSION 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:32.915 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/12346: 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:total 7202 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 . 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 May 24 11:38 .. 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 112 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 113 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 24 11:38 12182 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12182_fsm 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12182_vm 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12184 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12186 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12187_fsm 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187_vm 2024-05-24T11:38:32.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12189 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12191 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12192_fsm 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192_vm 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12194 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12196 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12197_fsm 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197_vm 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12199 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12201 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12202_fsm 2024-05-24T11:38:32.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202_vm 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12204 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12206 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12207_fsm 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207_vm 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12209 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12211 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12212 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12214 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12216 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 24 11:38 1247 2024-05-24T11:38:32.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1247_fsm 2024-05-24T11:38:32.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1247_vm 2024-05-24T11:38:32.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 24 11:38 1249 2024-05-24T11:38:32.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1249_fsm 2024-05-24T11:38:32.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1249_vm 2024-05-24T11:38:32.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 24 11:38 1255 2024-05-24T11:38:32.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1255_fsm 2024-05-24T11:38:32.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1255_vm 2024-05-24T11:38:32.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 24 11:38 1259 2024-05-24T11:38:32.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1259_fsm 2024-05-24T11:38:32.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1259_vm 2024-05-24T11:38:32.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417 2024-05-24T11:38:32.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417_vm 2024-05-24T11:38:32.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418 2024-05-24T11:38:32.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418_vm 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 174 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 175 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2187 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224_vm 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328_vm 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336_vm 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2337 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2600 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2600_fsm 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2600_vm 2024-05-24T11:38:32.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2601_fsm 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601_vm 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2602 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2602_fsm 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2602_vm 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2603 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2603_fsm 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2603_vm 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604_vm 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2605 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2605_fsm 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2605_vm 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606 2024-05-24T11:38:32.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2606_fsm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606_vm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607_fsm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2607_vm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 24 11:38 2608 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2608_fsm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2608_vm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 24 11:38 2609 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2609_fsm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2609_vm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610_fsm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2610_vm 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611 2024-05-24T11:38:32.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611_vm 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2612_fsm 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612_vm 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613_vm 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2615_fsm 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615_vm 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616_fsm 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2616_vm 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 24 11:38 2617 2024-05-24T11:38:32.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2617_fsm 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2617_vm 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 24 11:38 2618 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2618_fsm 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2618_vm 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 24 11:38 2619 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2619_fsm 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2619_vm 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620_vm 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2650 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2651 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2652 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2653 2024-05-24T11:38:32.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2654 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2655 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2656 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2657 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 24 11:38 2658 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2659 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2660 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2661 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2662 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2663 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2664 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2665 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2666 2024-05-24T11:38:32.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2667 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2668 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2669 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2670 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 May 24 11:38 2673 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 May 24 11:38 2674 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 24 11:38 2675 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2678 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2679 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2680 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2681 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2682 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2683 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2684 2024-05-24T11:38:32.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2685 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2686 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2687 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2688 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2689 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2690 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 24 11:38 2691 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2692 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2693 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2696 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2699 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2701 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2702 2024-05-24T11:38:32.930 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2703 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2704 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2753 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2753_fsm 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2753_vm 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2754 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2755 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2756 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2757 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830_vm 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2831 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832_vm 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2833 2024-05-24T11:38:32.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834_vm 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2835 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836_vm 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2837 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 24 11:38 2838 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2838_fsm 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2838_vm 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2839 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840_fsm 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2840_vm 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2841 2024-05-24T11:38:32.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995_vm 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2996 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3079_fsm 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079_vm 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3080 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3081 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3085 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118_vm 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3119 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3164 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256_vm 2024-05-24T11:38:32.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3257 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3258 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350_vm 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3351 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3379 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3380 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381_vm 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3394 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3394_fsm 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3394_vm 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3395 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439 2024-05-24T11:38:32.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439_vm 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3440 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 3455 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3456 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3456_fsm 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3456_vm 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466_vm 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3467 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3468 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501_vm 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3502 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3503 2024-05-24T11:38:32.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3534 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3541_fsm 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541_vm 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3542 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3574 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3575 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576_vm 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596_vm 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3597 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598_vm 2024-05-24T11:38:32.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3599 2024-05-24T11:38:32.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600 2024-05-24T11:38:32.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3600_fsm 2024-05-24T11:38:32.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600_vm 2024-05-24T11:38:32.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601 2024-05-24T11:38:32.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3601_fsm 2024-05-24T11:38:32.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601_vm 2024-05-24T11:38:32.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3602_fsm 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602_vm 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3603 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3603_fsm 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3603_vm 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3604 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3605 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3606 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3607 2024-05-24T11:38:33.192 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3608 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 3609 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3712 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3764_fsm 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764_vm 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3766 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3767 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3997 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 5002 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 548 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 549 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102 2024-05-24T11:38:33.193 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102_vm 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104_vm 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106_vm 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6110 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6111 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6112 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6113 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6117 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826_vm 2024-05-24T11:38:33.194 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 827 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 828 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 24 11:38 pg_filenode.map 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 24 11:38 PG_VERSION 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/12347: 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:total 7202 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 . 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 May 24 11:38 .. 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 112 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 113 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 24 11:38 12182 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12182_fsm 2024-05-24T11:38:33.195 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12182_vm 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12184 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12186 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12187_fsm 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187_vm 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12189 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12191 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12192_fsm 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192_vm 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12194 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12196 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12197_fsm 2024-05-24T11:38:33.196 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197_vm 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12199 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12201 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12202_fsm 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202_vm 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12204 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12206 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12207_fsm 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207_vm 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12209 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12211 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12212 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12214 2024-05-24T11:38:33.197 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12216 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 24 11:38 1247 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1247_fsm 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1247_vm 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 24 11:38 1249 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1249_fsm 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1249_vm 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 24 11:38 1255 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1255_fsm 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1255_vm 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 24 11:38 1259 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1259_fsm 2024-05-24T11:38:33.198 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1259_vm 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417_vm 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418_vm 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 174 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 175 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2187 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224_vm 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328 2024-05-24T11:38:33.199 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328_vm 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336_vm 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2337 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2600 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2600_fsm 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2600_vm 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2601_fsm 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601_vm 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2602 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2602_fsm 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2602_vm 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2603 2024-05-24T11:38:33.200 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2603_fsm 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2603_vm 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604_vm 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2605 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2605_fsm 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2605_vm 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2606_fsm 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606_vm 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607_fsm 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2607_vm 2024-05-24T11:38:33.201 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 24 11:38 2608 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2608_fsm 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2608_vm 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 24 11:38 2609 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2609_fsm 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2609_vm 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610_fsm 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2610_vm 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611_vm 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612 2024-05-24T11:38:33.202 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2612_fsm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612_vm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613_vm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2615_fsm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615_vm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616_fsm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2616_vm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 24 11:38 2617 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2617_fsm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2617_vm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 24 11:38 2618 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2618_fsm 2024-05-24T11:38:33.203 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2618_vm 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 24 11:38 2619 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2619_fsm 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2619_vm 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620_vm 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2650 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2651 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2652 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2653 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2654 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2655 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2656 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2657 2024-05-24T11:38:33.204 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 24 11:38 2658 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2659 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2660 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2661 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2662 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2663 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2664 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2665 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2666 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2667 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2668 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2669 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2670 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 May 24 11:38 2673 2024-05-24T11:38:33.205 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 May 24 11:38 2674 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 24 11:38 2675 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2678 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2679 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2680 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2681 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2682 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2683 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2684 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2685 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2686 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2687 2024-05-24T11:38:33.206 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2688 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2689 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2690 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 24 11:38 2691 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2692 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2693 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2696 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2699 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2701 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2702 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2703 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2704 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2753 2024-05-24T11:38:33.207 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2753_fsm 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2753_vm 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2754 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2755 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2756 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2757 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830_vm 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2831 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832_vm 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2833 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834_vm 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2835 2024-05-24T11:38:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836_vm 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2837 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 24 11:38 2838 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2838_fsm 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2838_vm 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2839 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840_fsm 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2840_vm 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2841 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995_vm 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2996 2024-05-24T11:38:33.209 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3079_fsm 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079_vm 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3080 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3081 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3085 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118_vm 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3119 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3164 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256_vm 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3257 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3258 2024-05-24T11:38:33.210 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350_vm 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3351 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3379 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3380 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381_vm 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3394 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3394_fsm 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3394_vm 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3395 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439_vm 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3440 2024-05-24T11:38:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 3455 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3456 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3456_fsm 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3456_vm 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466_vm 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3467 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3468 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501_vm 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3502 2024-05-24T11:38:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3503 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3534 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3541_fsm 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541_vm 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3542 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3574 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3575 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576_vm 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596_vm 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3597 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598 2024-05-24T11:38:33.213 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598_vm 2024-05-24T11:38:33.214 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3599 2024-05-24T11:38:33.214 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600 2024-05-24T11:38:33.214 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3600_fsm 2024-05-24T11:38:33.214 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600_vm 2024-05-24T11:38:33.214 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601 2024-05-24T11:38:33.214 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3601_fsm 2024-05-24T11:38:33.214 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601_vm 2024-05-24T11:38:33.214 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3602_fsm 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602_vm 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3603 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3603_fsm 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3603_vm 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3604 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3605 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3606 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3607 2024-05-24T11:38:33.240 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3608 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 3609 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3712 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3764_fsm 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764_vm 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3766 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3767 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3997 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 5002 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 548 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 549 2024-05-24T11:38:33.241 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102_vm 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104_vm 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106_vm 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6110 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6111 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6112 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6113 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6117 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826_vm 2024-05-24T11:38:33.242 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 827 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 828 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 24 11:38 pg_filenode.map 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 24 11:38 PG_VERSION 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/global: 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:total 554 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 565760 May 24 11:38 . 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1136 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1136_fsm 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1136_vm 2024-05-24T11:38:33.243 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 1137 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1213 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1213_fsm 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1213_vm 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1214 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1214_fsm 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1214_vm 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 1232 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 1233 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1260 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1260_fsm 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1260_vm 2024-05-24T11:38:33.244 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1261 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1261_fsm 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1261_vm 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1262 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1262_fsm 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1262_vm 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2396 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2396_fsm 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2396_vm 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2397 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2671 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2672 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2676 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2677 2024-05-24T11:38:33.245 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2694 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2695 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2697 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2698 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2846 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2846_vm 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2847 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2964 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2964_vm 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2965 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2966 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2966_vm 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2967 2024-05-24T11:38:33.246 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3592 2024-05-24T11:38:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3592_vm 2024-05-24T11:38:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3593 2024-05-24T11:38:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 4060 2024-05-24T11:38:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 4060_vm 2024-05-24T11:38:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 4061 2024-05-24T11:38:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6000 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6000_vm 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6001 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6002 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6100 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6100_vm 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6114 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6115 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 pg_control 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 24 11:38 pg_filenode.map 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/log: 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.278 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:total 3 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:38 . 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 mappings 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:38 replorigin_checkpoint 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 snapshots 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:38 .. 2024-05-24T11:38:33.279 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:38 .. 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 . 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 members 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 offsets 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-24T11:38:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-24T11:38:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 . 2024-05-24T11:38:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 .. 2024-05-24T11:38:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 0000 2024-05-24T11:38:33.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-24T11:38:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-24T11:38:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 . 2024-05-24T11:38:33.316 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 .. 2024-05-24T11:38:33.316 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 0000 2024-05-24T11:38:33.316 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 . 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 0000 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-24T11:38:33.317 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.318 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 . 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 0000 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.319 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:total 16386 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:38 . 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:38 000000010000000000000001 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 archive_status 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.320 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-24T11:38:33.321 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:38:33.321 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:38:33.321 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:38 .. 2024-05-24T11:38:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:38:33.321 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-24T11:38:33.321 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-24T11:38:33.321 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 . 2024-05-24T11:38:33.321 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 May 24 11:38 .. 2024-05-24T11:38:33.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 0000 2024-05-24T11:38:33.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-24T11:38:33.529 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-24T11:38:33.619 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-24T11:38:33.665 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-24T11:38:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-24T11:38:33.674 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-24T11:38:33.697 INFO:teuthology.orchestra.run.smithi039.stderr:creating tables... 2024-05-24T11:38:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.52 s, remaining 16.08 s) 2024-05-24T11:38:34.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:33 smithi053 ceph-mon[47448]: pgmap v13: 97 pgs: 97 active+clean; 39 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 4.2 MiB/s wr, 255 op/s 2024-05-24T11:38:34.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:33 smithi137 ceph-mon[47907]: pgmap v13: 97 pgs: 97 active+clean; 39 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 4.2 MiB/s wr, 255 op/s 2024-05-24T11:38:34.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:33 smithi039 ceph-mon[42919]: pgmap v13: 97 pgs: 97 active+clean; 39 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 4.2 MiB/s wr, 255 op/s 2024-05-24T11:38:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:200000 of 3200000 tuples (6%) done (elapsed 1.78 s, remaining 26.74 s) 2024-05-24T11:38:36.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:35 smithi053 ceph-mon[47448]: pgmap v14: 97 pgs: 97 active+clean; 47 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 5.1 MiB/s wr, 330 op/s 2024-05-24T11:38:36.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:35 smithi137 ceph-mon[47907]: pgmap v14: 97 pgs: 97 active+clean; 47 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 5.1 MiB/s wr, 330 op/s 2024-05-24T11:38:36.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:35 smithi039 ceph-mon[42919]: pgmap v14: 97 pgs: 97 active+clean; 47 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 5.1 MiB/s wr, 330 op/s 2024-05-24T11:38:36.367 INFO:teuthology.orchestra.run.smithi039.stderr:300000 of 3200000 tuples (9%) done (elapsed 2.67 s, remaining 25.78 s) 2024-05-24T11:38:37.609 INFO:teuthology.orchestra.run.smithi039.stderr:400000 of 3200000 tuples (12%) done (elapsed 3.91 s, remaining 27.36 s) 2024-05-24T11:38:38.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:37 smithi053 ceph-mon[47448]: pgmap v15: 97 pgs: 97 active+clean; 47 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-05-24T11:38:38.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:37 smithi137 ceph-mon[47907]: pgmap v15: 97 pgs: 97 active+clean; 47 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-05-24T11:38:38.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:37 smithi039 ceph-mon[42919]: pgmap v15: 97 pgs: 97 active+clean; 47 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-05-24T11:38:38.893 INFO:teuthology.orchestra.run.smithi039.stderr:500000 of 3200000 tuples (15%) done (elapsed 5.19 s, remaining 28.04 s) 2024-05-24T11:38:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:600000 of 3200000 tuples (18%) done (elapsed 6.08 s, remaining 26.33 s) 2024-05-24T11:38:40.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:39 smithi053 ceph-mon[47448]: pgmap v16: 97 pgs: 97 active+clean; 104 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 11 MiB/s wr, 270 op/s 2024-05-24T11:38:40.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:39 smithi137 ceph-mon[47907]: pgmap v16: 97 pgs: 97 active+clean; 104 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 11 MiB/s wr, 270 op/s 2024-05-24T11:38:40.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:39 smithi039 ceph-mon[42919]: pgmap v16: 97 pgs: 97 active+clean; 104 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 11 MiB/s wr, 270 op/s 2024-05-24T11:38:40.958 INFO:teuthology.orchestra.run.smithi039.stderr:700000 of 3200000 tuples (21%) done (elapsed 7.26 s, remaining 25.92 s) 2024-05-24T11:38:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.265+0000 7f46cf18f700 1 -- 172.21.15.39:0/3016313476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c81009c0 msgr2=0x7f46c8104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.265+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3016313476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c81009c0 0x7f46c8104a00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009990 tx=0x7f46b4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.265+0000 7f46cf18f700 1 -- 172.21.15.39:0/3016313476 shutdown_connections 2024-05-24T11:38:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.265+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3016313476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c8100010 0x7f46c81003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.265+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3016313476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c8105130 0x7f46c810ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.265+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3016313476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c81009c0 0x7f46c8104a00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.265+0000 7f46cf18f700 1 -- 172.21.15.39:0/3016313476 >> 172.21.15.39:0/3016313476 conn(0x7f46c8076f40 msgr2=0x7f46c8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.265+0000 7f46cf18f700 1 -- 172.21.15.39:0/3016313476 shutdown_connections 2024-05-24T11:38:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.265+0000 7f46cf18f700 1 -- 172.21.15.39:0/3016313476 wait complete. 2024-05-24T11:38:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46cf18f700 1 Processor -- start 2024-05-24T11:38:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46cf18f700 1 -- start start 2024-05-24T11:38:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46cf18f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c8100010 0x7f46c81972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46cf18f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8105130 0x7f46c81977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46cf18f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c819bb90 0x7f46c819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46cf18f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c819c610 con 0x7f46c8105130 2024-05-24T11:38:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46cf18f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c819c780 con 0x7f46c819bb90 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46cf18f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c819c940 con 0x7f46c8100010 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8105130 0x7f46c81977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8105130 0x7f46c81977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54614/0 (socket says 172.21.15.39:54614) 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46c7fff700 1 -- 172.21.15.39:0/3976665847 learned_addr learned my addr 172.21.15.39:0/3976665847 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c8100010 0x7f46c81972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.266+0000 7f46cd72c700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c819bb90 0x7f46c819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46c7fff700 1 -- 172.21.15.39:0/3976665847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c8100010 msgr2=0x7f46c81972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46c7fff700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c8100010 0x7f46c81972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46c7fff700 1 -- 172.21.15.39:0/3976665847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c819bb90 msgr2=0x7f46c819c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46c7fff700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c819bb90 0x7f46c819c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46c7fff700 1 -- 172.21.15.39:0/3976665847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b40095f0 con 0x7f46c8105130 2024-05-24T11:38:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c8100010 0x7f46c81972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:38:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46c7fff700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8105130 0x7f46c81977f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f46b40036c0 tx=0x7f46b4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46c5ffb700 1 -- 172.21.15.39:0/3976665847 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b401d070 con 0x7f46c8105130 2024-05-24T11:38:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46c5ffb700 1 -- 172.21.15.39:0/3976665847 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46b40059c0 con 0x7f46c8105130 2024-05-24T11:38:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46cf18f700 1 -- 172.21.15.39:0/3976665847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c81a3e40 con 0x7f46c8105130 2024-05-24T11:38:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46c5ffb700 1 -- 172.21.15.39:0/3976665847 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4022a50 con 0x7f46c8105130 2024-05-24T11:38:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.267+0000 7f46cf18f700 1 -- 172.21.15.39:0/3976665847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c81a4260 con 0x7f46c8105130 2024-05-24T11:38:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.270+0000 7f46c5ffb700 1 -- 172.21.15.39:0/3976665847 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f46b400f670 con 0x7f46c8105130 2024-05-24T11:38:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.271+0000 7f46db34ab80 1 Processor -- start 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.271+0000 7f46db34ab80 1 -- start start 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0540 0x55d9de3f0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 0x55d9de3f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c5ffb700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46b80a13a0 0x7f46b80a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46db34ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3430f0 con 0x55d9de3f4e40 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46db34ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de340ab0 con 0x55d9de3f0540 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46db34ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de33c3a0 con 0x55d9de3f0e60 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c5ffb700 1 -- 172.21.15.39:0/3976665847 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f46b40cdf50 con 0x7f46c8105130 2024-05-24T11:38:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46cd72c700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46b80a13a0 0x7f46b80a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54620/0 (socket says 172.21.15.39:54620) 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c7fff700 1 -- 172.21.15.39:0/1735217279 learned_addr learned my addr 172.21.15.39:0/1735217279 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46cd72c700 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0540 0x55d9de3f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 0x55d9de3f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c7fff700 1 -- 172.21.15.39:0/1735217279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 msgr2=0x55d9de3f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c7fff700 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 0x55d9de3f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c7fff700 1 -- 172.21.15.39:0/1735217279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0540 msgr2=0x55d9de3f0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c7fff700 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0540 0x55d9de3f0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c7fff700 1 -- 172.21.15.39:0/1735217279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9de1bd350 con 0x55d9de3f4e40 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46cd72c700 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0540 0x55d9de3f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46c7fff700 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f52a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f46b409af50 tx=0x7f46b4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c685a484c5d4b7ec server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.272+0000 7f46cd72c700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46b80a13a0 0x7f46b80a3860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009550 tx=0x7f46c000b040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46b2ffd700 1 -- 172.21.15.39:0/1735217279 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b409b6c0 con 0x55d9de3f4e40 2024-05-24T11:38:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46b2ffd700 1 -- 172.21.15.39:0/1735217279 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46b409afc0 con 0x55d9de3f4e40 2024-05-24T11:38:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46b2ffd700 1 -- 172.21.15.39:0/1735217279 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b409b220 con 0x55d9de3f4e40 2024-05-24T11:38:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46db34ab80 1 -- 172.21.15.39:0/1735217279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 msgr2=0x55d9de3f52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46db34ab80 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f52a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f46b409af50 tx=0x7f46b4009310 comp rx=0 tx=0).stop 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46db34ab80 1 -- 172.21.15.39:0/1735217279 shutdown_connections 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46db34ab80 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f52a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46db34ab80 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0540 0x55d9de3f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46db34ab80 1 --2- 172.21.15.39:0/1735217279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 0x55d9de3f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.273+0000 7f46db34ab80 1 -- 172.21.15.39:0/1735217279 >> 172.21.15.39:0/1735217279 conn(0x55d9de34a1b0 msgr2=0x55d9de3e6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.274+0000 7f46db34ab80 1 -- 172.21.15.39:0/1735217279 shutdown_connections 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.274+0000 7f46db34ab80 1 -- 172.21.15.39:0/1735217279 wait complete. 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.274+0000 7f46db34ab80 1 -- start start 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.274+0000 7f46db34ab80 1 Processor -- start 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0e60 0x55d9de3f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46db34ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de2d8500 con 0x55d9de3f4e40 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46db34ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de2d81a0 con 0x55d9de3f0e60 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46db34ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de1bf640 con 0x55d9de3f0540 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46db34ab80 10 client.0 _ll_register_callbacks cb 0x55d9de2e1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46cd72c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0e60 0x55d9de3f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0e60 0x55d9de3f98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:41212/0 (socket says 172.21.15.39:41212) 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46c7fff700 1 -- 172.21.15.39:0/3052816678 learned_addr learned my addr 172.21.15.39:0/3052816678 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46cd72c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54622/0 (socket says 172.21.15.39:54622) 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46cd72c700 1 -- 172.21.15.39:0/3052816678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 msgr2=0x55d9de3fc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46cd72c700 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3fc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46cd72c700 1 -- 172.21.15.39:0/3052816678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0e60 msgr2=0x55d9de3f98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46cd72c700 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0e60 0x55d9de3f98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46cd72c700 1 -- 172.21.15.39:0/3052816678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c81a4260 con 0x55d9de3f4e40 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46c7fff700 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0e60 0x55d9de3f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46cd72c700 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f9ed0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46c001ab40 tx=0x7f46c0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5fd15e07d6f20f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.275+0000 7f46a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46a7fff700 1 -- 172.21.15.39:0/3052816678 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c001f070 con 0x55d9de3f4e40 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46a7fff700 1 -- 172.21.15.39:0/3052816678 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f46c0015700 con 0x55d9de3f4e40 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46db34ab80 1 -- 172.21.15.39:0/3052816678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9de295210 con 0x55d9de3f4e40 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46a7fff700 1 -- 172.21.15.39:0/3052816678 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46c00095e0 con 0x55d9de3f4e40 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46db34ab80 10 client.15477 fetch_fsmap learned FSMap version 27 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46db34ab80 1 -- 172.21.15.39:0/3052816678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d9de169b00 con 0x55d9de3f4e40 2024-05-24T11:38:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46a7fff700 1 -- 172.21.15.39:0/3052816678 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d9de295210 con 0x55d9de3f4e40 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46a7fff700 1 -- 172.21.15.39:0/3052816678 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f46c002d070 con 0x55d9de3f4e40 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46db34ab80 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46db34ab80 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46db34ab80 20 client.15477 populate_metadata read hostname 'smithi039' 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d9de3ff810 0x55d9de401c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46db34ab80 4 client.15477 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.276+0000 7f46db34ab80 1 -- 172.21.15.39:0/3052816678 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9de3bb7b0 con 0x55d9de3ff810 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.277+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d9de3ff810 0x55d9de401c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.277+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d9de3ff810 0x55d9de401c30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc829c8093dab2f8 server_cookie=b085a73ceb3950e4 in_seq=0 out_seq=0 2024-05-24T11:38:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.277+0000 7f46a7fff700 10 client.15477 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46a7fff700 1 -- 172.21.15.39:0/3052816678 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9de3bb7b0 con 0x55d9de3ff810 2024-05-24T11:38:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46a7fff700 10 client.15477 handle_command_reply: tid=0 2024-05-24T11:38:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 client.15477 shutdown 2024-05-24T11:38:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 -- 172.21.15.39:0/3052816678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 msgr2=0x55d9de3f9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f9ed0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46c001ab40 tx=0x7f46c0004b40 comp rx=0 tx=0).stop 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 -- 172.21.15.39:0/3052816678 shutdown_connections 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d9de3ff810 0x55d9de401c30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f4e40 0x55d9de3f9ed0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3052816678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f0e60 0x55d9de3f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 -- 172.21.15.39:0/3052816678 >> 172.21.15.39:0/3052816678 conn(0x55d9de34a1b0 msgr2=0x55d9de3eaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46a7fff700 0 client.15477 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 -- 172.21.15.39:0/3052816678 shutdown_connections 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.278+0000 7f46db34ab80 1 -- 172.21.15.39:0/3052816678 wait complete. 2024-05-24T11:38:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.279+0000 7f46db34ab80 20 client.15477 trim_cache size 0 max 16384 2024-05-24T11:38:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.298+0000 7f46db34ab80 1 Processor -- start 2024-05-24T11:38:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.298+0000 7f46db34ab80 1 -- start start 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.298+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3e6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.298+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0e60 0x55d9de420590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.298+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 0x55d9de406590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.298+0000 7f46db34ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3f0040 con 0x55d9de3f0e60 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.298+0000 7f46db34ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3fa7b0 con 0x55d9de3f4e40 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.298+0000 7f46db34ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3f3a30 con 0x55d9de3f0540 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0e60 0x55d9de420590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0e60 0x55d9de420590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54634/0 (socket says 172.21.15.39:54634) 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46c7fff700 1 -- 172.21.15.39:0/3565639250 learned_addr learned my addr 172.21.15.39:0/3565639250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46cd72c700 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 0x55d9de406590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46c7fff700 1 -- 172.21.15.39:0/3565639250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 msgr2=0x55d9de3e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46c7fff700 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3e6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46c7fff700 1 -- 172.21.15.39:0/3565639250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 msgr2=0x55d9de406590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46c7fff700 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 0x55d9de406590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46c7fff700 1 -- 172.21.15.39:0/3565639250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9de3f54f0 con 0x55d9de3f0e60 2024-05-24T11:38:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:38:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46c7fff700 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0e60 0x55d9de420590 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46b409af50 tx=0x7f46b4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648ec40a1f977848 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46b37fe700 1 -- 172.21.15.39:0/3565639250 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4097300 con 0x55d9de3f0e60 2024-05-24T11:38:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46b37fe700 1 -- 172.21.15.39:0/3565639250 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46b409b6c0 con 0x55d9de3f0e60 2024-05-24T11:38:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.299+0000 7f46b37fe700 1 -- 172.21.15.39:0/3565639250 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b409b220 con 0x55d9de3f0e60 2024-05-24T11:38:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.300+0000 7f46db34ab80 1 -- 172.21.15.39:0/3565639250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0e60 msgr2=0x55d9de420590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.300+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0e60 0x55d9de420590 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46b409af50 tx=0x7f46b4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.300+0000 7f46db34ab80 1 -- 172.21.15.39:0/3565639250 shutdown_connections 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.300+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0540 0x55d9de3e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.300+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 0x55d9de406590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.300+0000 7f46db34ab80 1 --2- 172.21.15.39:0/3565639250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0e60 0x55d9de420590 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.300+0000 7f46db34ab80 1 -- 172.21.15.39:0/3565639250 >> 172.21.15.39:0/3565639250 conn(0x55d9de34a1b0 msgr2=0x55d9de3f8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.300+0000 7f46db34ab80 1 -- 172.21.15.39:0/3565639250 shutdown_connections 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.300+0000 7f46db34ab80 1 -- 172.21.15.39:0/3565639250 wait complete. 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46db34ab80 1 -- start start 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46db34ab80 1 Processor -- start 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0540 0x55d9de4065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 0x55d9de406ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46db34ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 0x55d9de3ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46db34ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3eee00 con 0x55d9de3f0540 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46db34ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3eef70 con 0x55d9de3f4e40 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46db34ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3ef130 con 0x55d9de3f0e60 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46db34ab80 10 client.0 _ll_register_callbacks cb 0x55d9de2e1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46ccf2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0540 0x55d9de4065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46ccf2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0540 0x55d9de4065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54638/0 (socket says 172.21.15.39:54638) 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46ccf2b700 1 -- 172.21.15.39:0/4280851464 learned_addr learned my addr 172.21.15.39:0/4280851464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46cd72c700 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 0x55d9de3ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.301+0000 7f46c7fff700 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 0x55d9de406ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46ccf2b700 1 -- 172.21.15.39:0/4280851464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 msgr2=0x55d9de406ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 0x55d9de406ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46ccf2b700 1 -- 172.21.15.39:0/4280851464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 msgr2=0x55d9de3ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 0x55d9de3ee6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46ccf2b700 1 -- 172.21.15.39:0/4280851464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46bc012040 con 0x55d9de3f0540 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46ccf2b700 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0540 0x55d9de4065a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f46bc005950 tx=0x7f46bc0129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965985bcc552c566 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46a7fff700 1 -- 172.21.15.39:0/4280851464 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc017040 con 0x55d9de3f0540 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46db34ab80 1 -- 172.21.15.39:0/4280851464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46b409afc0 con 0x55d9de3f0540 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46a7fff700 1 -- 172.21.15.39:0/4280851464 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f46bc01c780 con 0x55d9de3f0540 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46a7fff700 1 -- 172.21.15.39:0/4280851464 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46bc004020 con 0x55d9de3f0540 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46db34ab80 10 client.15483 fetch_fsmap learned FSMap version 27 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46db34ab80 1 -- 172.21.15.39:0/4280851464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d9de1bd350 con 0x55d9de3f0540 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.302+0000 7f46a7fff700 1 -- 172.21.15.39:0/4280851464 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46b409afc0 con 0x55d9de3f0540 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46a7fff700 1 -- 172.21.15.39:0/4280851464 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f46bc003d10 con 0x55d9de3f0540 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46db34ab80 10 client.15483 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46db34ab80 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:38:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46db34ab80 20 client.15483 populate_metadata read hostname 'smithi039' 2024-05-24T11:38:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46db34ab80 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d9de46d020 0x55d9de46f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46db34ab80 4 client.15483 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:38:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46db34ab80 1 -- 172.21.15.39:0/4280851464 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d9de334e60 con 0x55d9de46d020 2024-05-24T11:38:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46c7fff700 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d9de46d020 0x55d9de46f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46a7fff700 10 client.15483 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.303+0000 7f46c7fff700 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d9de46d020 0x55d9de46f440 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0fd4fdd291ae5cd server_cookie=e282fcf781538086 in_seq=0 out_seq=0 2024-05-24T11:38:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.381+0000 7f46a7fff700 1 -- 172.21.15.39:0/4280851464 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d9de334e60 con 0x55d9de46d020 2024-05-24T11:38:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.381+0000 7f46a7fff700 10 client.15483 handle_command_reply: tid=0 2024-05-24T11:38:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.381+0000 7f46db34ab80 1 client.15483 shutdown 2024-05-24T11:38:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.381+0000 7f46db34ab80 1 -- 172.21.15.39:0/4280851464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0540 msgr2=0x55d9de4065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.381+0000 7f46db34ab80 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0540 0x55d9de4065a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f46bc005950 tx=0x7f46bc0129f0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46db34ab80 1 -- 172.21.15.39:0/4280851464 shutdown_connections 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46db34ab80 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d9de46d020 0x55d9de46f440 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46db34ab80 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9de3f4e40 0x55d9de3ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46db34ab80 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d9de3f0540 0x55d9de4065a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46a7fff700 0 client.15483 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46db34ab80 1 --2- 172.21.15.39:0/4280851464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de3f0e60 0x55d9de406ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46db34ab80 1 -- 172.21.15.39:0/4280851464 >> 172.21.15.39:0/4280851464 conn(0x55d9de34a1b0 msgr2=0x55d9de3f2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46db34ab80 1 -- 172.21.15.39:0/4280851464 shutdown_connections 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46db34ab80 1 -- 172.21.15.39:0/4280851464 wait complete. 2024-05-24T11:38:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.382+0000 7f46db34ab80 20 client.15483 trim_cache size 0 max 16384 2024-05-24T11:38:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:38:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:38:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:38:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:38:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 -- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46b80a13a0 msgr2=0x7f46b80a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46b80a13a0 0x7f46b80a3860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009550 tx=0x7f46c000b040 comp rx=0 tx=0).stop 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 -- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8105130 msgr2=0x7f46c81977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8105130 0x7f46c81977f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f46b40036c0 tx=0x7f46b4009580 comp rx=0 tx=0).stop 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 -- 172.21.15.39:0/3976665847 shutdown_connections 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c8100010 0x7f46c81972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46b80a13a0 0x7f46b80a3860 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46c819bb90 0x7f46c819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 --2- 172.21.15.39:0/3976665847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8105130 0x7f46c81977f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 -- 172.21.15.39:0/3976665847 >> 172.21.15.39:0/3976665847 conn(0x7f46c8076f40 msgr2=0x7f46c8069550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 -- 172.21.15.39:0/3976665847 shutdown_connections 2024-05-24T11:38:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:41.383+0000 7f46cf18f700 1 -- 172.21.15.39:0/3976665847 wait complete. 2024-05-24T11:38:42.224 INFO:teuthology.orchestra.run.smithi039.stderr:800000 of 3200000 tuples (25%) done (elapsed 8.52 s, remaining 25.57 s) 2024-05-24T11:38:42.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:41 smithi137 ceph-mon[47907]: pgmap v17: 97 pgs: 97 active+clean; 136 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 16 MiB/s wr, 161 op/s 2024-05-24T11:38:42.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:41 smithi039 ceph-mon[42919]: pgmap v17: 97 pgs: 97 active+clean; 136 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 16 MiB/s wr, 161 op/s 2024-05-24T11:38:42.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:41 smithi053 ceph-mon[47448]: pgmap v17: 97 pgs: 97 active+clean; 136 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 16 MiB/s wr, 161 op/s 2024-05-24T11:38:43.155 INFO:teuthology.orchestra.run.smithi039.stderr:900000 of 3200000 tuples (28%) done (elapsed 9.46 s, remaining 24.16 s) 2024-05-24T11:38:44.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:43 smithi137 ceph-mon[47907]: pgmap v18: 97 pgs: 97 active+clean; 136 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 15 MiB/s wr, 95 op/s 2024-05-24T11:38:44.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:43 smithi039 ceph-mon[42919]: pgmap v18: 97 pgs: 97 active+clean; 136 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 15 MiB/s wr, 95 op/s 2024-05-24T11:38:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:1000000 of 3200000 tuples (31%) done (elapsed 10.71 s, remaining 23.57 s) 2024-05-24T11:38:44.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:43 smithi053 ceph-mon[47448]: pgmap v18: 97 pgs: 97 active+clean; 136 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 15 MiB/s wr, 95 op/s 2024-05-24T11:38:45.641 INFO:teuthology.orchestra.run.smithi039.stderr:1100000 of 3200000 tuples (34%) done (elapsed 11.94 s, remaining 22.80 s) 2024-05-24T11:38:46.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:45 smithi039 ceph-mon[42919]: pgmap v19: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 33 MiB/s wr, 149 op/s 2024-05-24T11:38:46.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:45 smithi053 ceph-mon[47448]: pgmap v19: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 33 MiB/s wr, 149 op/s 2024-05-24T11:38:46.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:45 smithi137 ceph-mon[47907]: pgmap v19: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 33 MiB/s wr, 149 op/s 2024-05-24T11:38:46.511 INFO:teuthology.orchestra.run.smithi039.stderr:1200000 of 3200000 tuples (37%) done (elapsed 12.81 s, remaining 21.35 s) 2024-05-24T11:38:47.736 INFO:teuthology.orchestra.run.smithi039.stderr:1300000 of 3200000 tuples (40%) done (elapsed 14.04 s, remaining 20.51 s) 2024-05-24T11:38:48.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:47 smithi039 ceph-mon[42919]: pgmap v20: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 32 MiB/s wr, 74 op/s 2024-05-24T11:38:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:47 smithi053 ceph-mon[47448]: pgmap v20: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 32 MiB/s wr, 74 op/s 2024-05-24T11:38:48.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:47 smithi137 ceph-mon[47907]: pgmap v20: 97 pgs: 97 active+clean; 232 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 32 MiB/s wr, 74 op/s 2024-05-24T11:38:49.008 INFO:teuthology.orchestra.run.smithi039.stderr:1400000 of 3200000 tuples (43%) done (elapsed 15.31 s, remaining 19.68 s) 2024-05-24T11:38:49.843 INFO:teuthology.orchestra.run.smithi039.stderr:1500000 of 3200000 tuples (46%) done (elapsed 16.14 s, remaining 18.30 s) 2024-05-24T11:38:50.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:49 smithi039 ceph-mon[42919]: pgmap v21: 97 pgs: 97 active+clean; 292 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 42 MiB/s wr, 84 op/s 2024-05-24T11:38:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:49 smithi053 ceph-mon[47448]: pgmap v21: 97 pgs: 97 active+clean; 292 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 42 MiB/s wr, 84 op/s 2024-05-24T11:38:50.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:49 smithi137 ceph-mon[47907]: pgmap v21: 97 pgs: 97 active+clean; 292 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 42 MiB/s wr, 84 op/s 2024-05-24T11:38:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:1600000 of 3200000 tuples (50%) done (elapsed 17.44 s, remaining 17.44 s) 2024-05-24T11:38:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:1700000 of 3200000 tuples (53%) done (elapsed 18.37 s, remaining 16.21 s) 2024-05-24T11:38:52.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:52 smithi039 ceph-mon[42919]: pgmap v22: 97 pgs: 97 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 MiB/s wr, 71 op/s 2024-05-24T11:38:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:52 smithi053 ceph-mon[47448]: pgmap v22: 97 pgs: 97 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 MiB/s wr, 71 op/s 2024-05-24T11:38:52.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:52 smithi137 ceph-mon[47907]: pgmap v22: 97 pgs: 97 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 MiB/s wr, 71 op/s 2024-05-24T11:38:53.334 INFO:teuthology.orchestra.run.smithi039.stderr:1800000 of 3200000 tuples (56%) done (elapsed 19.63 s, remaining 15.27 s) 2024-05-24T11:38:54.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:54 smithi039 ceph-mon[42919]: pgmap v23: 97 pgs: 97 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 68 op/s 2024-05-24T11:38:54.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:54 smithi053 ceph-mon[47448]: pgmap v23: 97 pgs: 97 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 68 op/s 2024-05-24T11:38:54.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:54 smithi137 ceph-mon[47907]: pgmap v23: 97 pgs: 97 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 68 op/s 2024-05-24T11:38:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.511+0000 7f5bdb927700 1 -- 172.21.15.39:0/1925005210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 msgr2=0x7f5bd41073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.511+0000 7f5bdb927700 1 --2- 172.21.15.39:0/1925005210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 0x7f5bd41073e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009990 tx=0x7f5bd0009ca0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.511+0000 7f5bdb927700 1 -- 172.21.15.39:0/1925005210 shutdown_connections 2024-05-24T11:38:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.511+0000 7f5bdb927700 1 --2- 172.21.15.39:0/1925005210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd4068ca0 0x7f5bd410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.511+0000 7f5bdb927700 1 --2- 172.21.15.39:0/1925005210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 0x7f5bd41073e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.511+0000 7f5bdb927700 1 --2- 172.21.15.39:0/1925005210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40682e0 0x7f5bd4068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.511+0000 7f5bdb927700 1 -- 172.21.15.39:0/1925005210 >> 172.21.15.39:0/1925005210 conn(0x7f5bd4076f10 msgr2=0x7f5bd4077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.511+0000 7f5bdb927700 1 -- 172.21.15.39:0/1925005210 shutdown_connections 2024-05-24T11:38:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.511+0000 7f5bdb927700 1 -- 172.21.15.39:0/1925005210 wait complete. 2024-05-24T11:38:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bdb927700 1 Processor -- start 2024-05-24T11:38:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bdb927700 1 -- start start 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bdb927700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40682e0 0x7f5bd4197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bdb927700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd4068ca0 0x7f5bd41976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bdb927700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 0x7f5bd419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bdb927700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd419c080 con 0x7f5bd4107000 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bdb927700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd419c1f0 con 0x7f5bd4068ca0 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bdb927700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd419c3b0 con 0x7f5bd40682e0 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bd9ec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 0x7f5bd419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bd9ec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 0x7f5bd419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48536/0 (socket says 172.21.15.39:48536) 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/2375757758 learned_addr learned my addr 172.21.15.39:0/2375757758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40682e0 0x7f5bd4197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.512+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd4068ca0 0x7f5bd41976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/2375757758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40682e0 msgr2=0x7f5bd4197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40682e0 0x7f5bd4197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/2375757758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd4068ca0 msgr2=0x7f5bd41976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd4068ca0 0x7f5bd41976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/2375757758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00095f0 con 0x7f5bd4107000 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 0x7f5bd419ba40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc007d70 tx=0x7f5bcc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bca7fc700 1 -- 172.21.15.39:0/2375757758 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00ac90 con 0x7f5bd4107000 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bca7fc700 1 -- 172.21.15.39:0/2375757758 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bcc00adf0 con 0x7f5bd4107000 2024-05-24T11:38:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bdb927700 1 -- 172.21.15.39:0/2375757758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd419c690 con 0x7f5bd4107000 2024-05-24T11:38:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bca7fc700 1 -- 172.21.15.39:0/2375757758 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0115e0 con 0x7f5bd4107000 2024-05-24T11:38:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.513+0000 7f5bdb927700 1 -- 172.21.15.39:0/2375757758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd41a3ed0 con 0x7f5bd4107000 2024-05-24T11:38:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.516+0000 7f5bca7fc700 1 -- 172.21.15.39:0/2375757758 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f5bcc012420 con 0x7f5bd4107000 2024-05-24T11:38:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.516+0000 7f5bca7fc700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5bc00a1320 0x7f5bc00a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.517+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5bc00a1320 0x7f5bc00a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.517+0000 7f5bca7fc700 1 -- 172.21.15.39:0/2375757758 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5bcc0ccf00 con 0x7f5bd4107000 2024-05-24T11:38:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.517+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5bc00a1320 0x7f5bc00a37e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000b5c0 tx=0x7f5bd0005fb0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.517+0000 7f5be7ae2b80 1 Processor -- start 2024-05-24T11:38:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.517+0000 7f5be7ae2b80 1 -- start start 2024-05-24T11:38:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa53d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa4df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa4e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16f9a50f0 con 0x55a16fa54270 2024-05-24T11:38:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16f9a2ab0 con 0x55a16fa4e480 2024-05-24T11:38:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16f99e3a0 con 0x55a16fa53950 2024-05-24T11:38:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd8ec2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa4df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd8ec2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa4df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48542/0 (socket says 172.21.15.39:48542) 2024-05-24T11:38:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd8ec2700 1 -- 172.21.15.39:0/3277680920 learned_addr learned my addr 172.21.15.39:0/3277680920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa53d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa4e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd8ec2700 1 -- 172.21.15.39:0/3277680920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 msgr2=0x55a16fa53d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa53d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd8ec2700 1 -- 172.21.15.39:0/3277680920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 msgr2=0x55a16fa4e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa4e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd8ec2700 1 -- 172.21.15.39:0/3277680920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a16f81f350 con 0x55a16fa54270 2024-05-24T11:38:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa4e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:38:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.518+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa4df40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400ad10 tx=0x7f5bc400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735babf1417e06e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5bbf7fe700 1 -- 172.21.15.39:0/3277680920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc400ae60 con 0x55a16fa54270 2024-05-24T11:38:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5bbf7fe700 1 -- 172.21.15.39:0/3277680920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bc40044c0 con 0x55a16fa54270 2024-05-24T11:38:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5bbf7fe700 1 -- 172.21.15.39:0/3277680920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4018430 con 0x55a16fa54270 2024-05-24T11:38:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3277680920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 msgr2=0x55a16fa4df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa4df40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400ad10 tx=0x7f5bc400f6a0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3277680920 shutdown_connections 2024-05-24T11:38:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa53d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa4e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3277680920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa4df40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.519+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3277680920 >> 172.21.15.39:0/3277680920 conn(0x55a16f9ac1b0 msgr2=0x55a16fa48840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.520+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3277680920 shutdown_connections 2024-05-24T11:38:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.520+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3277680920 wait complete. 2024-05-24T11:38:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.520+0000 7f5be7ae2b80 1 -- start start 2024-05-24T11:38:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.520+0000 7f5be7ae2b80 1 Processor -- start 2024-05-24T11:38:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa4e480 0x55a16fa5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa53950 0x55a16fa574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16f93a500 con 0x55a16fa54270 2024-05-24T11:38:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16f93a1a0 con 0x55a16fa53950 2024-05-24T11:38:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16f821640 con 0x55a16fa4e480 2024-05-24T11:38:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5be7ae2b80 10 client.0 _ll_register_callbacks cb 0x55a16f943ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:38:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd9ec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd9ec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48546/0 (socket says 172.21.15.39:48546) 2024-05-24T11:38:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/113944864 learned_addr learned my addr 172.21.15.39:0/113944864 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa4e480 0x55a16fa5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa53950 0x55a16fa574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/113944864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa4e480 msgr2=0x55a16fa5a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa4e480 0x55a16fa5a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/113944864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa53950 msgr2=0x55a16fa574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa53950 0x55a16fa574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/113944864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc400a970 con 0x55a16fa54270 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa53950 0x55a16fa574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc099cf0 tx=0x7f5bcc09a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa49c01af6df1d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.521+0000 7f5bbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5bbcff9700 1 -- 172.21.15.39:0/113944864 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc09a450 con 0x55a16fa54270 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5bbcff9700 1 -- 172.21.15.39:0/113944864 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5bcc09de20 con 0x55a16fa54270 2024-05-24T11:38:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/113944864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a16f8f7210 con 0x55a16fa54270 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5bbcff9700 1 -- 172.21.15.39:0/113944864 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bcc05fea0 con 0x55a16fa54270 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5be7ae2b80 10 client.15495 fetch_fsmap learned FSMap version 27 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/113944864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a16f7cbb00 con 0x55a16fa54270 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5bbcff9700 1 -- 172.21.15.39:0/113944864 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bcc063080 con 0x55a16fa54270 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5bbcff9700 1 -- 172.21.15.39:0/113944864 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5bcc0a1fe0 con 0x55a16fa54270 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5be7ae2b80 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5be7ae2b80 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5be7ae2b80 20 client.15495 populate_metadata read hostname 'smithi039' 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a16fa67a10 0x55a16fa69e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5be7ae2b80 4 client.15495 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/113944864 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a16fa1d7b0 con 0x55a16fa67a10 2024-05-24T11:38:54.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.522+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a16fa67a10 0x55a16fa69e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.523+0000 7f5bbcff9700 10 client.15495 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.523+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a16fa67a10 0x55a16fa69e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c071a35bcacaa server_cookie=6704b556b029606b in_seq=0 out_seq=0 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5bbcff9700 1 -- 172.21.15.39:0/113944864 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a16fa1d7b0 con 0x55a16fa67a10 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5bbcff9700 10 client.15495 handle_command_reply: tid=0 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 client.15495 shutdown 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/113944864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 msgr2=0x55a16fa57a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc099cf0 tx=0x7f5bcc09a770 comp rx=0 tx=0).stop 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/113944864 shutdown_connections 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa4e480 0x55a16fa5a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a16fa67a10 0x55a16fa69e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/113944864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa53950 0x55a16fa574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5bbcff9700 0 client.15495 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/113944864 >> 172.21.15.39:0/113944864 conn(0x55a16f9ac1b0 msgr2=0x55a16fa524a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/113944864 shutdown_connections 2024-05-24T11:38:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/113944864 wait complete. 2024-05-24T11:38:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.524+0000 7f5be7ae2b80 20 client.15495 trim_cache size 0 max 16384 2024-05-24T11:38:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:1900000 of 3200000 tuples (59%) done (elapsed 20.84 s, remaining 14.26 s) 2024-05-24T11:38:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.543+0000 7f5be7ae2b80 1 Processor -- start 2024-05-24T11:38:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5be7ae2b80 1 -- start start 2024-05-24T11:38:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa4e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16fa55a20 con 0x55a16fa54270 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16fa4d140 con 0x55a16fa4e480 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16fa53f80 con 0x55a16fa53950 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd9ec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd9ec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48550/0 (socket says 172.21.15.39:48550) 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/207693319 learned_addr learned my addr 172.21.15.39:0/207693319 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa4e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/207693319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 msgr2=0x55a16fa4e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa4e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/207693319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 msgr2=0x55a16fa531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/207693319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a16f964500 con 0x55a16fa54270 2024-05-24T11:38:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.544+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa4e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:38:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.545+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc09a1b0 tx=0x7f5bcc018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e86b719e2a4f522 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.545+0000 7f5bbffff700 1 -- 172.21.15.39:0/207693319 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc05fea0 con 0x55a16fa54270 2024-05-24T11:38:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.545+0000 7f5bbffff700 1 -- 172.21.15.39:0/207693319 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a16f8f7210 con 0x55a16fa54270 2024-05-24T11:38:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.545+0000 7f5bbffff700 1 -- 172.21.15.39:0/207693319 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0994e0 con 0x55a16fa54270 2024-05-24T11:38:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/207693319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 msgr2=0x55a16fa57a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc09a1b0 tx=0x7f5bcc018fb0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/207693319 shutdown_connections 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa57a10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/207693319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa4e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/207693319 >> 172.21.15.39:0/207693319 conn(0x55a16f9ac1b0 msgr2=0x55a16fa63720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/207693319 shutdown_connections 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/207693319 wait complete. 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 -- start start 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.546+0000 7f5be7ae2b80 1 Processor -- start 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa5f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa5ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5be7ae2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa5b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16fa5b940 con 0x55a16fa54270 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16fa5bab0 con 0x55a16fa4e480 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5be7ae2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16fabcf30 con 0x55a16fa53950 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5be7ae2b80 10 client.0 _ll_register_callbacks cb 0x55a16f943ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd9ec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa5b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd9ec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa5b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48562/0 (socket says 172.21.15.39:48562) 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/3526761517 learned_addr learned my addr 172.21.15.39:0/3526761517 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa5f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa5ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/3526761517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 msgr2=0x55a16fa5ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa5ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/3526761517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 msgr2=0x55a16fa5f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa5f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd9ec4700 1 -- 172.21.15.39:0/3526761517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a16f81f350 con 0x55a16fa54270 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.547+0000 7f5bd8ec2700 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa5ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5bd9ec4700 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa5b270 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00a690 tx=0x7f5bcc099a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e72641eb64de361 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5bbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5bbcff9700 1 -- 172.21.15.39:0/3526761517 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc05fea0 con 0x55a16fa54270 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5bbcff9700 1 -- 172.21.15.39:0/3526761517 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5bcc09de20 con 0x55a16fa54270 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3526761517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bc40044c0 con 0x55a16fa54270 2024-05-24T11:38:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5bbcff9700 1 -- 172.21.15.39:0/3526761517 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bcc0b8380 con 0x55a16fa54270 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5be7ae2b80 10 client.15501 fetch_fsmap learned FSMap version 27 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3526761517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a16f7cbb00 con 0x55a16fa54270 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5bbcff9700 1 -- 172.21.15.39:0/3526761517 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bc40044c0 con 0x55a16fa54270 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5bbcff9700 1 -- 172.21.15.39:0/3526761517 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5bcc0126d0 con 0x55a16fa54270 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5be7ae2b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5be7ae2b80 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5be7ae2b80 20 client.15501 populate_metadata read hostname 'smithi039' 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a16fabe230 0x55a16fac0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5be7ae2b80 4 client.15501 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.548+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3526761517 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a16f996e60 con 0x55a16fabe230 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.549+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a16fabe230 0x55a16fac0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.549+0000 7f5bbcff9700 10 client.15501 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.549+0000 7f5bd96c3700 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a16fabe230 0x55a16fac0620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ec2feac0ce402 server_cookie=d4be04c2bb60942 in_seq=0 out_seq=0 2024-05-24T11:38:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5bbcff9700 1 -- 172.21.15.39:0/3526761517 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a16f996e60 con 0x55a16fabe230 2024-05-24T11:38:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5bbcff9700 10 client.15501 handle_command_reply: tid=0 2024-05-24T11:38:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 client.15501 shutdown 2024-05-24T11:38:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3526761517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 msgr2=0x55a16fa5b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa5b270 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00a690 tx=0x7f5bcc099a20 comp rx=0 tx=0).stop 2024-05-24T11:38:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3526761517 shutdown_connections 2024-05-24T11:38:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a16fabe230 0x55a16fac0620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a16fa54270 0x55a16fa5b270 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a16fa4e480 0x55a16fa5f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 --2- 172.21.15.39:0/3526761517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a16fa53950 0x55a16fa5ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3526761517 >> 172.21.15.39:0/3526761517 conn(0x55a16f9ac1b0 msgr2=0x55a16fa63720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5bbcff9700 0 client.15501 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:38:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3526761517 shutdown_connections 2024-05-24T11:38:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 1 -- 172.21.15.39:0/3526761517 wait complete. 2024-05-24T11:38:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.569+0000 7f5be7ae2b80 20 client.15501 trim_cache size 0 max 16384 2024-05-24T11:38:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:38:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:38:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:38:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:38:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 -- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5bc00a1320 msgr2=0x7f5bc00a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5bc00a1320 0x7f5bc00a37e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000b5c0 tx=0x7f5bd0005fb0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 -- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 msgr2=0x7f5bd419ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:38:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 0x7f5bd419ba40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc007d70 tx=0x7f5bcc00a340 comp rx=0 tx=0).stop 2024-05-24T11:38:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 -- 172.21.15.39:0/2375757758 shutdown_connections 2024-05-24T11:38:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40682e0 0x7f5bd4197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5bc00a1320 0x7f5bc00a37e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4107000 0x7f5bd419ba40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 --2- 172.21.15.39:0/2375757758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd4068ca0 0x7f5bd41976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:38:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 -- 172.21.15.39:0/2375757758 >> 172.21.15.39:0/2375757758 conn(0x7f5bd4076f10 msgr2=0x7f5bd40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:38:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 -- 172.21.15.39:0/2375757758 shutdown_connections 2024-05-24T11:38:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:38:54.570+0000 7f5bdb927700 1 -- 172.21.15.39:0/2375757758 wait complete. 2024-05-24T11:38:55.391 INFO:teuthology.orchestra.run.smithi039.stderr:2000000 of 3200000 tuples (62%) done (elapsed 21.69 s, remaining 13.01 s) 2024-05-24T11:38:56.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:56 smithi039 ceph-mon[42919]: pgmap v24: 97 pgs: 97 active+clean; 440 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 MiB/s wr, 80 op/s 2024-05-24T11:38:56.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:56 smithi053 ceph-mon[47448]: pgmap v24: 97 pgs: 97 active+clean; 440 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 MiB/s wr, 80 op/s 2024-05-24T11:38:56.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:56 smithi137 ceph-mon[47907]: pgmap v24: 97 pgs: 97 active+clean; 440 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 MiB/s wr, 80 op/s 2024-05-24T11:38:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2100000 of 3200000 tuples (65%) done (elapsed 22.92 s, remaining 12.01 s) 2024-05-24T11:38:57.846 INFO:teuthology.orchestra.run.smithi039.stderr:2200000 of 3200000 tuples (68%) done (elapsed 24.15 s, remaining 10.98 s) 2024-05-24T11:38:58.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:38:58 smithi039 ceph-mon[42919]: pgmap v25: 97 pgs: 97 active+clean; 440 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 25 op/s 2024-05-24T11:38:58.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:38:58 smithi053 ceph-mon[47448]: pgmap v25: 97 pgs: 97 active+clean; 440 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 25 op/s 2024-05-24T11:38:58.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:38:58 smithi137 ceph-mon[47907]: pgmap v25: 97 pgs: 97 active+clean; 440 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 25 op/s 2024-05-24T11:38:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2300000 of 3200000 tuples (71%) done (elapsed 25.07 s, remaining 9.81 s) 2024-05-24T11:39:00.030 INFO:teuthology.orchestra.run.smithi039.stderr:2400000 of 3200000 tuples (75%) done (elapsed 26.33 s, remaining 8.78 s) 2024-05-24T11:39:00.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:00 smithi039 ceph-mon[42919]: pgmap v26: 97 pgs: 97 active+clean; 500 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 39 op/s 2024-05-24T11:39:00.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:00 smithi053 ceph-mon[47448]: pgmap v26: 97 pgs: 97 active+clean; 500 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 39 op/s 2024-05-24T11:39:00.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:00 smithi137 ceph-mon[47907]: pgmap v26: 97 pgs: 97 active+clean; 500 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 39 op/s 2024-05-24T11:39:01.078 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:39:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:39:00.777+0000 7efb91bee700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:39:01.078 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:39:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:39:00.777+0000 7efb91bee700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:39:01.079 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:39:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:39:00.777+0000 7efb91bee700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:39:01.079 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:39:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:39:00.777+0000 7efb91bee700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:39:01.079 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:39:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:39:00.777+0000 7efb91bee700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-24T11:39:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2500000 of 3200000 tuples (78%) done (elapsed 27.60 s, remaining 7.73 s) 2024-05-24T11:39:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2600000 of 3200000 tuples (81%) done (elapsed 28.48 s, remaining 6.57 s) 2024-05-24T11:39:02.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:02 smithi039 ceph-mon[42919]: pgmap v27: 97 pgs: 97 active+clean; 548 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 42 MiB/s wr, 34 op/s 2024-05-24T11:39:02.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:02 smithi053 ceph-mon[47448]: pgmap v27: 97 pgs: 97 active+clean; 548 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 42 MiB/s wr, 34 op/s 2024-05-24T11:39:02.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:02 smithi137 ceph-mon[47907]: pgmap v27: 97 pgs: 97 active+clean; 548 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 42 MiB/s wr, 34 op/s 2024-05-24T11:39:03.398 INFO:teuthology.orchestra.run.smithi039.stderr:2700000 of 3200000 tuples (84%) done (elapsed 29.70 s, remaining 5.50 s) 2024-05-24T11:39:04.283 INFO:teuthology.orchestra.run.smithi039.stderr:2800000 of 3200000 tuples (87%) done (elapsed 30.58 s, remaining 4.37 s) 2024-05-24T11:39:04.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:04 smithi039 ceph-mon[42919]: pgmap v28: 97 pgs: 97 active+clean; 548 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-24T11:39:04.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:04 smithi039 ceph-mon[42919]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-05-24T11:39:04.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:04 smithi053 ceph-mon[47448]: pgmap v28: 97 pgs: 97 active+clean; 548 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-24T11:39:04.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:04 smithi053 ceph-mon[47448]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-05-24T11:39:04.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:04 smithi137 ceph-mon[47907]: pgmap v28: 97 pgs: 97 active+clean; 548 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 30 op/s 2024-05-24T11:39:04.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:04 smithi137 ceph-mon[47907]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-05-24T11:39:05.508 INFO:teuthology.orchestra.run.smithi039.stderr:2900000 of 3200000 tuples (90%) done (elapsed 31.81 s, remaining 3.29 s) 2024-05-24T11:39:06.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:06 smithi039 ceph-mon[42919]: pgmap v29: 97 pgs: 97 active+clean; 669 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 MiB/s wr, 59 op/s 2024-05-24T11:39:06.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:06 smithi053 ceph-mon[47448]: pgmap v29: 97 pgs: 97 active+clean; 669 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 MiB/s wr, 59 op/s 2024-05-24T11:39:06.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:06 smithi137 ceph-mon[47907]: pgmap v29: 97 pgs: 97 active+clean; 669 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 MiB/s wr, 59 op/s 2024-05-24T11:39:06.757 INFO:teuthology.orchestra.run.smithi039.stderr:3000000 of 3200000 tuples (93%) done (elapsed 33.06 s, remaining 2.20 s) 2024-05-24T11:39:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:3100000 of 3200000 tuples (96%) done (elapsed 33.95 s, remaining 1.10 s) 2024-05-24T11:39:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.696+0000 7fcb5799e700 1 -- 172.21.15.39:0/4011466366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb500682e0 msgr2=0x7fcb50068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.696+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4011466366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb500682e0 0x7fcb50068760 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009990 tx=0x7fcb40009ca0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.696+0000 7fcb5799e700 1 -- 172.21.15.39:0/4011466366 shutdown_connections 2024-05-24T11:39:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.696+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4011466366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50068ca0 0x7fcb5010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.696+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4011466366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb50107000 0x7fcb501073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.696+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4011466366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb500682e0 0x7fcb50068760 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.696+0000 7fcb5799e700 1 -- 172.21.15.39:0/4011466366 >> 172.21.15.39:0/4011466366 conn(0x7fcb50076f10 msgr2=0x7fcb50077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.696+0000 7fcb5799e700 1 -- 172.21.15.39:0/4011466366 shutdown_connections 2024-05-24T11:39:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.696+0000 7fcb5799e700 1 -- 172.21.15.39:0/4011466366 wait complete. 2024-05-24T11:39:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.697+0000 7fcb5799e700 1 Processor -- start 2024-05-24T11:39:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.697+0000 7fcb5799e700 1 -- start start 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.697+0000 7fcb5799e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb500682e0 0x7fcb500ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.697+0000 7fcb5799e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb50068ca0 0x7fcb50100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.697+0000 7fcb5799e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50107000 0x7fcb50100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.697+0000 7fcb5799e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb501a55d0 con 0x7fcb50068ca0 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.697+0000 7fcb5799e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb501a5740 con 0x7fcb500682e0 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.697+0000 7fcb5799e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb501a5900 con 0x7fcb50107000 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.697+0000 7fcb54f39700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb50068ca0 0x7fcb50100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb54f39700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb50068ca0 0x7fcb50100350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47700/0 (socket says 172.21.15.39:47700) 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb54f39700 1 -- 172.21.15.39:0/4007814219 learned_addr learned my addr 172.21.15.39:0/4007814219 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb5573a700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb500682e0 0x7fcb500ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50107000 0x7fcb50100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb55f3b700 1 -- 172.21.15.39:0/4007814219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb500682e0 msgr2=0x7fcb500ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb500682e0 0x7fcb500ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb55f3b700 1 -- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb50068ca0 msgr2=0x7fcb50100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb50068ca0 0x7fcb50100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb55f3b700 1 -- 172.21.15.39:0/4007814219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb400095f0 con 0x7fcb50107000 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb54f39700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb50068ca0 0x7fcb50100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb5573a700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb500682e0 0x7fcb500ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50107000 0x7fcb50100890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800efc0 tx=0x7fcb4800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb467fc700 1 -- 172.21.15.39:0/4007814219 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48018070 con 0x7fcb50107000 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb467fc700 1 -- 172.21.15.39:0/4007814219 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb4800f040 con 0x7fcb50107000 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.698+0000 7fcb5799e700 1 -- 172.21.15.39:0/4007814219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb501a5b80 con 0x7fcb50107000 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.699+0000 7fcb467fc700 1 -- 172.21.15.39:0/4007814219 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800cba0 con 0x7fcb50107000 2024-05-24T11:39:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.699+0000 7fcb5799e700 1 -- 172.21.15.39:0/4007814219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb501a6040 con 0x7fcb50107000 2024-05-24T11:39:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.701+0000 7fcb467fc700 1 -- 172.21.15.39:0/4007814219 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fcb48003a00 con 0x7fcb50107000 2024-05-24T11:39:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.702+0000 7fcb467fc700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcb3c0a13b0 0x7fcb3c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.702+0000 7fcb5573a700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcb3c0a13b0 0x7fcb3c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.702+0000 7fcb467fc700 1 -- 172.21.15.39:0/4007814219 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcb480cca40 con 0x7fcb50107000 2024-05-24T11:39:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.703+0000 7fcb5573a700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcb3c0a13b0 0x7fcb3c0a3870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c007f50 tx=0x7fcb4c0073d0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.703+0000 7fcb63b59b80 1 Processor -- start 2024-05-24T11:39:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.703+0000 7fcb63b59b80 1 -- start start 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.703+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd1712d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.703+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1713270 0x55bfd170cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.703+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd170d480 0x55bfd170d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.703+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd16640f0 con 0x55bfd1712950 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.703+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1661ab0 con 0x55bfd1713270 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.703+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd165d3a0 con 0x55bfd170d480 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb5573a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd1712d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb5573a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd1712d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47702/0 (socket says 172.21.15.39:47702) 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb5573a700 1 -- 172.21.15.39:0/2879981615 learned_addr learned my addr 172.21.15.39:0/2879981615 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd170d480 0x55bfd170d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb54f39700 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1713270 0x55bfd170cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb5573a700 1 -- 172.21.15.39:0/2879981615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd170d480 msgr2=0x55bfd170d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb5573a700 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd170d480 0x55bfd170d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb5573a700 1 -- 172.21.15.39:0/2879981615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1713270 msgr2=0x55bfd170cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb5573a700 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1713270 0x55bfd170cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb5573a700 1 -- 172.21.15.39:0/2879981615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd14de350 con 0x55bfd1712950 2024-05-24T11:39:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd170d480 0x55bfd170d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:39:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb5573a700 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd1712d30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c004380 tx=0x7fcb4c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8227654248365c20 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb3b7fe700 1 -- 172.21.15.39:0/2879981615 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c003900 con 0x55bfd1712950 2024-05-24T11:39:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb3b7fe700 1 -- 172.21.15.39:0/2879981615 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb4c011440 con 0x55bfd1712950 2024-05-24T11:39:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.704+0000 7fcb3b7fe700 1 -- 172.21.15.39:0/2879981615 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c01f690 con 0x55bfd1712950 2024-05-24T11:39:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.705+0000 7fcb63b59b80 1 -- 172.21.15.39:0/2879981615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 msgr2=0x55bfd1712d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.705+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd1712d30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c004380 tx=0x7fcb4c013000 comp rx=0 tx=0).stop 2024-05-24T11:39:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.705+0000 7fcb63b59b80 1 -- 172.21.15.39:0/2879981615 shutdown_connections 2024-05-24T11:39:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.705+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd170d480 0x55bfd170d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.705+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd1712d30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.705+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/2879981615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1713270 0x55bfd170cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.705+0000 7fcb63b59b80 1 -- 172.21.15.39:0/2879981615 >> 172.21.15.39:0/2879981615 conn(0x55bfd166b1b0 msgr2=0x55bfd1707840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.705+0000 7fcb63b59b80 1 -- 172.21.15.39:0/2879981615 shutdown_connections 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.705+0000 7fcb63b59b80 1 -- 172.21.15.39:0/2879981615 wait complete. 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.706+0000 7fcb63b59b80 1 -- start start 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.706+0000 7fcb63b59b80 1 Processor -- start 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.706+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 0x55bfd1719280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.706+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd17165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.706+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 0x55bfd1716d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.706+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd15f9500 con 0x55bfd1712950 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.706+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd15f91a0 con 0x55bfd170d480 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.706+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd14e0640 con 0x55bfd1713270 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.706+0000 7fcb63b59b80 10 client.0 _ll_register_callbacks cb 0x55bfd1602ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:39:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb54f39700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd17165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb54f39700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd17165c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47708/0 (socket says 172.21.15.39:47708) 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb54f39700 1 -- 172.21.15.39:0/3975059188 learned_addr learned my addr 172.21.15.39:0/3975059188 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb5573a700 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 0x55bfd1719280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 0x55bfd1716d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb54f39700 1 -- 172.21.15.39:0/3975059188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 msgr2=0x55bfd1716d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb54f39700 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 0x55bfd1716d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb54f39700 1 -- 172.21.15.39:0/3975059188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 msgr2=0x55bfd1719280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb54f39700 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 0x55bfd1719280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb54f39700 1 -- 172.21.15.39:0/3975059188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c004030 con 0x55bfd1712950 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb54f39700 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd17165c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcb40005270 tx=0x7fcb400052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca04f6baefc4cc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb38ff9700 1 -- 172.21.15.39:0/3975059188 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4001d070 con 0x55bfd1712950 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb38ff9700 1 -- 172.21.15.39:0/3975059188 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcb40004910 con 0x55bfd1712950 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.707+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3975059188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfd15b6210 con 0x55bfd1712950 2024-05-24T11:39:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb38ff9700 1 -- 172.21.15.39:0/3975059188 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb40004ad0 con 0x55bfd1712950 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb63b59b80 10 client.15519 fetch_fsmap learned FSMap version 27 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3975059188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bfd148ab00 con 0x55bfd1712950 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb38ff9700 1 -- 172.21.15.39:0/3975059188 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bfd15b6210 con 0x55bfd1712950 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb38ff9700 1 -- 172.21.15.39:0/3975059188 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb4000fb30 con 0x55bfd1712950 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb63b59b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb63b59b80 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb63b59b80 20 client.15519 populate_metadata read hostname 'smithi039' 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bfd1726ba0 0x55bfd1728fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb63b59b80 4 client.15519 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3975059188 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfd16dc7b0 con 0x55bfd1726ba0 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.708+0000 7fcb5573a700 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bfd1726ba0 0x55bfd1728fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.709+0000 7fcb38ff9700 10 client.15519 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.709+0000 7fcb5573a700 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bfd1726ba0 0x55bfd1728fc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9124e3bdd5366f59 server_cookie=c9962fd69fe7f6c5 in_seq=0 out_seq=0 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb38ff9700 1 -- 172.21.15.39:0/3975059188 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfd16dc7b0 con 0x55bfd1726ba0 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb38ff9700 10 client.15519 handle_command_reply: tid=0 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 client.15519 shutdown 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3975059188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 msgr2=0x55bfd17165c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd17165c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcb40005270 tx=0x7fcb400052a0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3975059188 shutdown_connections 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 0x55bfd1716d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bfd1726ba0 0x55bfd1728fc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 0x55bfd1719280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3975059188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1712950 0x55bfd17165c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb38ff9700 0 client.15519 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3975059188 >> 172.21.15.39:0/3975059188 conn(0x55bfd166b1b0 msgr2=0x55bfd170bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3975059188 shutdown_connections 2024-05-24T11:39:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3975059188 wait complete. 2024-05-24T11:39:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.710+0000 7fcb63b59b80 20 client.15519 trim_cache size 0 max 16384 2024-05-24T11:39:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb63b59b80 1 Processor -- start 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb63b59b80 1 -- start start 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 0x55bfd1716560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1712950 0x55bfd17226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1713270 0x55bfd1707910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd17130c0 con 0x55bfd1713270 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd172e1b0 con 0x55bfd170d480 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1722fe0 con 0x55bfd1712950 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb54f39700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1712950 0x55bfd17226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb54f39700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1712950 0x55bfd17226f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:54980/0 (socket says 172.21.15.39:54980) 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb5573a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 0x55bfd1716560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb54f39700 1 -- 172.21.15.39:0/3013739911 learned_addr learned my addr 172.21.15.39:0/3013739911 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.729+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/3013739911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1713270 0x55bfd1707910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.730+0000 7fcb5573a700 1 -- 172.21.15.39:0/3013739911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1712950 msgr2=0x55bfd17226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.730+0000 7fcb5573a700 1 --2- 172.21.15.39:0/3013739911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1712950 0x55bfd17226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.730+0000 7fcb5573a700 1 -- 172.21.15.39:0/3013739911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1713270 msgr2=0x55bfd1707910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.730+0000 7fcb5573a700 1 --2- 172.21.15.39:0/3013739911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1713270 0x55bfd1707910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.730+0000 7fcb5573a700 1 -- 172.21.15.39:0/3013739911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd1623500 con 0x55bfd170d480 2024-05-24T11:39:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.730+0000 7fcb5573a700 1 --2- 172.21.15.39:0/3013739911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 0x55bfd1716560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c022040 tx=0x7fcb4c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af158dc9a3cbd987 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.730+0000 7fcb3bfff700 1 -- 172.21.15.39:0/3013739911 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c031070 con 0x55bfd170d480 2024-05-24T11:39:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.730+0000 7fcb3bfff700 1 -- 172.21.15.39:0/3013739911 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb4c011be0 con 0x55bfd170d480 2024-05-24T11:39:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.730+0000 7fcb3bfff700 1 -- 172.21.15.39:0/3013739911 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c031730 con 0x55bfd170d480 2024-05-24T11:39:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3013739911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 msgr2=0x55bfd1716560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3013739911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 0x55bfd1716560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c022040 tx=0x7fcb4c005750 comp rx=0 tx=0).stop 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3013739911 shutdown_connections 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3013739911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd1713270 0x55bfd1707910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3013739911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd170d480 0x55bfd1716560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/3013739911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1712950 0x55bfd17226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3013739911 >> 172.21.15.39:0/3013739911 conn(0x55bfd166b1b0 msgr2=0x55bfd1715e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3013739911 shutdown_connections 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 -- 172.21.15.39:0/3013739911 wait complete. 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.731+0000 7fcb63b59b80 1 -- start start 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb63b59b80 1 Processor -- start 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd170d480 0x55bfd171bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1712950 0x55bfd171c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb63b59b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 0x55bfd171c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd177c250 con 0x55bfd170d480 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd177c3c0 con 0x55bfd1712950 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb63b59b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd177c580 con 0x55bfd1713270 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb63b59b80 10 client.0 _ll_register_callbacks cb 0x55bfd1602ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb5573a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd170d480 0x55bfd171bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb5573a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd170d480 0x55bfd171bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47724/0 (socket says 172.21.15.39:47724) 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb5573a700 1 -- 172.21.15.39:0/1948052303 learned_addr learned my addr 172.21.15.39:0/1948052303 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.732+0000 7fcb54f39700 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1712950 0x55bfd171c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb5573a700 1 -- 172.21.15.39:0/1948052303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 msgr2=0x55bfd171c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 0x55bfd171c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb5573a700 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 0x55bfd171c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb5573a700 1 -- 172.21.15.39:0/1948052303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1712950 msgr2=0x55bfd171c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb5573a700 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1712950 0x55bfd171c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb5573a700 1 -- 172.21.15.39:0/1948052303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfd14de350 con 0x55bfd170d480 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb55f3b700 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 0x55bfd171c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb5573a700 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd170d480 0x55bfd171bc20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005720 tx=0x7fcb4c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bedecf3f34033f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb38ff9700 1 -- 172.21.15.39:0/1948052303 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c031040 con 0x55bfd170d480 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb38ff9700 1 -- 172.21.15.39:0/1948052303 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fcb4c017070 con 0x55bfd170d480 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb63b59b80 1 -- 172.21.15.39:0/1948052303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb4c011440 con 0x55bfd170d480 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb38ff9700 1 -- 172.21.15.39:0/1948052303 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb4c011a70 con 0x55bfd170d480 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb63b59b80 10 client.15525 fetch_fsmap learned FSMap version 27 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.733+0000 7fcb63b59b80 1 -- 172.21.15.39:0/1948052303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bfd14e2c70 con 0x55bfd170d480 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.734+0000 7fcb38ff9700 1 -- 172.21.15.39:0/1948052303 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb4c011440 con 0x55bfd170d480 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.734+0000 7fcb38ff9700 1 -- 172.21.15.39:0/1948052303 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb4c03aae0 con 0x55bfd170d480 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.734+0000 7fcb63b59b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:39:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.734+0000 7fcb63b59b80 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:39:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.734+0000 7fcb63b59b80 20 client.15525 populate_metadata read hostname 'smithi039' 2024-05-24T11:39:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.734+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bfd177d8d0 0x55bfd177fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.734+0000 7fcb63b59b80 4 client.15525 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:39:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.734+0000 7fcb63b59b80 1 -- 172.21.15.39:0/1948052303 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bfd1655e60 con 0x55bfd177d8d0 2024-05-24T11:39:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.734+0000 7fcb54f39700 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bfd177d8d0 0x55bfd177fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.735+0000 7fcb38ff9700 10 client.15525 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.735+0000 7fcb54f39700 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bfd177d8d0 0x55bfd177fcc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb51c380dcd802c server_cookie=a6b073dffa4fd6ec in_seq=0 out_seq=0 2024-05-24T11:39:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb38ff9700 1 -- 172.21.15.39:0/1948052303 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bfd1655e60 con 0x55bfd177d8d0 2024-05-24T11:39:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb38ff9700 10 client.15525 handle_command_reply: tid=0 2024-05-24T11:39:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 client.15525 shutdown 2024-05-24T11:39:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 -- 172.21.15.39:0/1948052303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd170d480 msgr2=0x55bfd171bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd170d480 0x55bfd171bc20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005720 tx=0x7fcb4c005a00 comp rx=0 tx=0).stop 2024-05-24T11:39:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 -- 172.21.15.39:0/1948052303 shutdown_connections 2024-05-24T11:39:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfd1713270 0x55bfd171c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bfd177d8d0 0x55bfd177fcc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfd170d480 0x55bfd171bc20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 --2- 172.21.15.39:0/1948052303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bfd1712950 0x55bfd171c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb38ff9700 0 client.15525 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 -- 172.21.15.39:0/1948052303 >> 172.21.15.39:0/1948052303 conn(0x55bfd166b1b0 msgr2=0x55bfd170bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 -- 172.21.15.39:0/1948052303 shutdown_connections 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 1 -- 172.21.15.39:0/1948052303 wait complete. 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.761+0000 7fcb63b59b80 20 client.15525 trim_cache size 0 max 16384 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:39:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:39:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 -- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcb3c0a13b0 msgr2=0x7fcb3c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcb3c0a13b0 0x7fcb3c0a3870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c007f50 tx=0x7fcb4c0073d0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 -- 172.21.15.39:0/4007814219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50107000 msgr2=0x7fcb50100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50107000 0x7fcb50100890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800efc0 tx=0x7fcb4800c3a0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 -- 172.21.15.39:0/4007814219 shutdown_connections 2024-05-24T11:39:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50107000 0x7fcb50100890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcb3c0a13b0 0x7fcb3c0a3870 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb500682e0 0x7fcb500ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 --2- 172.21.15.39:0/4007814219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb50068ca0 0x7fcb50100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 -- 172.21.15.39:0/4007814219 >> 172.21.15.39:0/4007814219 conn(0x7fcb50076f10 msgr2=0x7fcb500fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 -- 172.21.15.39:0/4007814219 shutdown_connections 2024-05-24T11:39:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:07.762+0000 7fcb5799e700 1 -- 172.21.15.39:0/4007814219 wait complete. 2024-05-24T11:39:08.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:08 smithi039 ceph-mon[42919]: pgmap v30: 97 pgs: 97 active+clean; 669 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 MiB/s wr, 46 op/s 2024-05-24T11:39:08.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:08 smithi053 ceph-mon[47448]: pgmap v30: 97 pgs: 97 active+clean; 669 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 MiB/s wr, 46 op/s 2024-05-24T11:39:08.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:08 smithi137 ceph-mon[47907]: pgmap v30: 97 pgs: 97 active+clean; 669 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 MiB/s wr, 46 op/s 2024-05-24T11:39:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:3200000 of 3200000 tuples (100%) done (elapsed 35.24 s, remaining 0.00 s) 2024-05-24T11:39:09.003 INFO:teuthology.orchestra.run.smithi039.stderr:vacuum... 2024-05-24T11:39:10.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:10 smithi039 ceph-mon[42919]: pgmap v31: 97 pgs: 97 active+clean; 738 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 46 MiB/s wr, 61 op/s 2024-05-24T11:39:10.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:10 smithi053 ceph-mon[47448]: pgmap v31: 97 pgs: 97 active+clean; 738 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 46 MiB/s wr, 61 op/s 2024-05-24T11:39:10.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:10 smithi137 ceph-mon[47907]: pgmap v31: 97 pgs: 97 active+clean; 738 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 46 MiB/s wr, 61 op/s 2024-05-24T11:39:12.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:12 smithi039 ceph-mon[42919]: pgmap v32: 97 pgs: 97 active+clean; 778 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 MiB/s wr, 51 op/s 2024-05-24T11:39:12.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:12 smithi053 ceph-mon[47448]: pgmap v32: 97 pgs: 97 active+clean; 778 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 MiB/s wr, 51 op/s 2024-05-24T11:39:12.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:12 smithi137 ceph-mon[47907]: pgmap v32: 97 pgs: 97 active+clean; 778 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 MiB/s wr, 51 op/s 2024-05-24T11:39:14.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:14 smithi039 ceph-mon[42919]: pgmap v33: 97 pgs: 97 active+clean; 778 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 46 op/s 2024-05-24T11:39:14.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:14 smithi053 ceph-mon[47448]: pgmap v33: 97 pgs: 97 active+clean; 778 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 46 op/s 2024-05-24T11:39:14.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:14 smithi137 ceph-mon[47907]: pgmap v33: 97 pgs: 97 active+clean; 778 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 46 op/s 2024-05-24T11:39:16.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:16 smithi039 ceph-mon[42919]: pgmap v34: 97 pgs: 97 active+clean; 978 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 MiB/s wr, 61 op/s 2024-05-24T11:39:16.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:16 smithi053 ceph-mon[47448]: pgmap v34: 97 pgs: 97 active+clean; 978 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 MiB/s wr, 61 op/s 2024-05-24T11:39:16.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:16 smithi137 ceph-mon[47907]: pgmap v34: 97 pgs: 97 active+clean; 978 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 MiB/s wr, 61 op/s 2024-05-24T11:39:17.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:17 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:39:17.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:17 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:39:17.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:17 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:39:17.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:17 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:39:17.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:17 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:39:17.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:17 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:39:17.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:17 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:39:17.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:17 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:39:17.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:17 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:39:17.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:17 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:39:17.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:17 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:39:17.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:17 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:39:17.741 INFO:teuthology.orchestra.run.smithi039.stderr:set primary keys... 2024-05-24T11:39:18.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:18 smithi039 ceph-mon[42919]: pgmap v35: 97 pgs: 97 active+clean; 978 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 33 op/s 2024-05-24T11:39:18.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:18 smithi053 ceph-mon[47448]: pgmap v35: 97 pgs: 97 active+clean; 978 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 33 op/s 2024-05-24T11:39:18.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:18 smithi137 ceph-mon[47907]: pgmap v35: 97 pgs: 97 active+clean; 978 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 MiB/s wr, 33 op/s 2024-05-24T11:39:20.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:20 smithi039 ceph-mon[42919]: pgmap v36: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 MiB/s wr, 47 op/s 2024-05-24T11:39:20.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:20 smithi053 ceph-mon[47448]: pgmap v36: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 MiB/s wr, 47 op/s 2024-05-24T11:39:20.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:20 smithi137 ceph-mon[47907]: pgmap v36: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 MiB/s wr, 47 op/s 2024-05-24T11:39:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.884+0000 7f258c260700 1 -- 172.21.15.39:0/4160429746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 msgr2=0x7f25841014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.884+0000 7f258c260700 1 --2- 172.21.15.39:0/4160429746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 0x7f25841014f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f25800098f0 tx=0x7f2580009c00 comp rx=0 tx=0).stop 2024-05-24T11:39:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.885+0000 7f258c260700 1 -- 172.21.15.39:0/4160429746 shutdown_connections 2024-05-24T11:39:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.885+0000 7f258c260700 1 --2- 172.21.15.39:0/4160429746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2584106230 0x7f258410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.885+0000 7f258c260700 1 --2- 172.21.15.39:0/4160429746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 0x7f25841014f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.885+0000 7f258c260700 1 --2- 172.21.15.39:0/4160429746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584101ac0 0x7f2584105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.885+0000 7f258c260700 1 -- 172.21.15.39:0/4160429746 >> 172.21.15.39:0/4160429746 conn(0x7f25840fc9b0 msgr2=0x7f25840fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.885+0000 7f258c260700 1 -- 172.21.15.39:0/4160429746 shutdown_connections 2024-05-24T11:39:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.885+0000 7f258c260700 1 -- 172.21.15.39:0/4160429746 wait complete. 2024-05-24T11:39:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.885+0000 7f258c260700 1 Processor -- start 2024-05-24T11:39:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f258c260700 1 -- start start 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f258c260700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 0x7f25841994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f258c260700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2584106230 0x7f25841999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f258c260700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258419dd90 0x7f258419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f258c260700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258419e810 con 0x7f2584101110 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f258c260700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258419e980 con 0x7f2584106230 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f258c260700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258419eb40 con 0x7f258419dd90 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f2589ffc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 0x7f25841994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f2589ffc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 0x7f25841994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39872/0 (socket says 172.21.15.39:39872) 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f2589ffc700 1 -- 172.21.15.39:0/3546113283 learned_addr learned my addr 172.21.15.39:0/3546113283 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f25897fb700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2584106230 0x7f25841999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f2589ffc700 1 -- 172.21.15.39:0/3546113283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258419dd90 msgr2=0x7f258419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f2589ffc700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258419dd90 0x7f258419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f2589ffc700 1 -- 172.21.15.39:0/3546113283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2584106230 msgr2=0x7f25841999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f2589ffc700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2584106230 0x7f25841999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.886+0000 7f2589ffc700 1 -- 172.21.15.39:0/3546113283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740095f0 con 0x7f2584101110 2024-05-24T11:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.887+0000 7f2589ffc700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 0x7f25841994b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2580010f90 tx=0x7f2580010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.887+0000 7f257affd700 1 -- 172.21.15.39:0/3546113283 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258001c070 con 0x7f2584101110 2024-05-24T11:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.887+0000 7f257affd700 1 -- 172.21.15.39:0/3546113283 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f258000b990 con 0x7f2584101110 2024-05-24T11:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.887+0000 7f258c260700 1 -- 172.21.15.39:0/3546113283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2580009520 con 0x7f2584101110 2024-05-24T11:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.887+0000 7f257affd700 1 -- 172.21.15.39:0/3546113283 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25800046d0 con 0x7f2584101110 2024-05-24T11:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.887+0000 7f258c260700 1 -- 172.21.15.39:0/3546113283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25841a62d0 con 0x7f2584101110 2024-05-24T11:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.890+0000 7f257affd700 1 -- 172.21.15.39:0/3546113283 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f2580020070 con 0x7f2584101110 2024-05-24T11:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.891+0000 7f257affd700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f25700a13b0 0x7f25700a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.891+0000 7f257affd700 1 -- 172.21.15.39:0/3546113283 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f25800cdbd0 con 0x7f2584101110 2024-05-24T11:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.891+0000 7f25897fb700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f25700a13b0 0x7f25700a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.891+0000 7f259841bb80 1 Processor -- start 2024-05-24T11:39:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.891+0000 7f259841bb80 1 -- start start 2024-05-24T11:39:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.891+0000 7f25897fb700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f25700a13b0 0x7f25700a3870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f258419aa00 tx=0x7f2574009310 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f259841bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f259841bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f59a5270 0x5588f599ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f259841bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f599f480 0x5588f599f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f259841bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f58f60f0 con 0x5588f59a4950 2024-05-24T11:39:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f259841bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f58f3ab0 con 0x5588f599f480 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f259841bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f58ef3a0 con 0x5588f59a5270 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f2589ffc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f2589ffc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39874/0 (socket says 172.21.15.39:39874) 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f2589ffc700 1 -- 172.21.15.39:0/42981112 learned_addr learned my addr 172.21.15.39:0/42981112 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f25897fb700 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f59a5270 0x5588f599ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f258a7fd700 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f599f480 0x5588f599f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f2589ffc700 1 -- 172.21.15.39:0/42981112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f59a5270 msgr2=0x5588f599ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f2589ffc700 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f59a5270 0x5588f599ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f2589ffc700 1 -- 172.21.15.39:0/42981112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f599f480 msgr2=0x5588f599f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f2589ffc700 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f599f480 0x5588f599f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f2589ffc700 1 -- 172.21.15.39:0/42981112 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588f5770350 con 0x5588f59a4950 2024-05-24T11:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f258a7fd700 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f599f480 0x5588f599f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:39:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.892+0000 7f2589ffc700 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a4d30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f258009aaf0 tx=0x7f258009a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a35024993809f0e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.893+0000 7f256ffff700 1 -- 172.21.15.39:0/42981112 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258009e990 con 0x5588f59a4950 2024-05-24T11:39:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.893+0000 7f256ffff700 1 -- 172.21.15.39:0/42981112 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25800a2f50 con 0x5588f59a4950 2024-05-24T11:39:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.893+0000 7f256ffff700 1 -- 172.21.15.39:0/42981112 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258009a050 con 0x5588f59a4950 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.893+0000 7f259841bb80 1 -- 172.21.15.39:0/42981112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 msgr2=0x5588f59a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.893+0000 7f259841bb80 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a4d30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f258009aaf0 tx=0x7f258009a4c0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.894+0000 7f259841bb80 1 -- 172.21.15.39:0/42981112 shutdown_connections 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.894+0000 7f259841bb80 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f599f480 0x5588f599f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.894+0000 7f259841bb80 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a4d30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.894+0000 7f259841bb80 1 --2- 172.21.15.39:0/42981112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f59a5270 0x5588f599ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.894+0000 7f259841bb80 1 -- 172.21.15.39:0/42981112 >> 172.21.15.39:0/42981112 conn(0x5588f58fd1b0 msgr2=0x5588f5999840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.894+0000 7f259841bb80 1 -- 172.21.15.39:0/42981112 shutdown_connections 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.894+0000 7f259841bb80 1 -- 172.21.15.39:0/42981112 wait complete. 2024-05-24T11:39:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.894+0000 7f259841bb80 1 -- start start 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.894+0000 7f259841bb80 1 Processor -- start 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f259841bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f59ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f259841bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f259841bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f259841bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f588b500 con 0x5588f59a4950 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f259841bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f588b1a0 con 0x5588f59a5270 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f259841bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f5772640 con 0x5588f599f480 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f259841bb80 10 client.0 _ll_register_callbacks cb 0x5588f5894ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f25897fb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f258a7fd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f258a7fd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:53840/0 (socket says 172.21.15.39:53840) 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f258a7fd700 1 -- 172.21.15.39:0/2535346918 learned_addr learned my addr 172.21.15.39:0/2535346918 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.895+0000 7f25897fb700 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39876/0 (socket says 172.21.15.39:39876) 2024-05-24T11:39:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f2589ffc700 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f59ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f25897fb700 1 -- 172.21.15.39:0/2535346918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 msgr2=0x5588f59ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f25897fb700 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f59ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f25897fb700 1 -- 172.21.15.39:0/2535346918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 msgr2=0x5588f59a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f25897fb700 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f25897fb700 1 -- 172.21.15.39:0/2535346918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25841a62d0 con 0x5588f59a4950 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f258a7fd700 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f25897fb700 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a8400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5588f599a790 tx=0x7f25740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bc92f8fbb999a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f256d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f256d7fa700 1 -- 172.21.15.39:0/2535346918 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574022070 con 0x5588f59a4950 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f256d7fa700 1 -- 172.21.15.39:0/2535346918 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f257401d7e0 con 0x5588f59a4950 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f259841bb80 1 -- 172.21.15.39:0/2535346918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588f5848210 con 0x5588f59a4950 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f256d7fa700 1 -- 172.21.15.39:0/2535346918 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257401d9c0 con 0x5588f59a4950 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f259841bb80 10 client.15537 fetch_fsmap learned FSMap version 27 2024-05-24T11:39:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.896+0000 7f259841bb80 1 -- 172.21.15.39:0/2535346918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588f571cb00 con 0x5588f59a4950 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f256d7fa700 1 -- 172.21.15.39:0/2535346918 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5588f5848210 con 0x5588f59a4950 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f256d7fa700 1 -- 172.21.15.39:0/2535346918 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2574026b10 con 0x5588f59a4950 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f259841bb80 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f259841bb80 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f259841bb80 20 client.15537 populate_metadata read hostname 'smithi039' 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f259841bb80 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5588f59b88e0 0x5588f59bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f259841bb80 4 client.15537 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f259841bb80 1 -- 172.21.15.39:0/2535346918 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588f596e7b0 con 0x5588f59b88e0 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f2589ffc700 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5588f59b88e0 0x5588f59bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f2589ffc700 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5588f59b88e0 0x5588f59bad00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f3c3632d75edef server_cookie=af5ef487c8970864 in_seq=0 out_seq=0 2024-05-24T11:39:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.897+0000 7f256d7fa700 10 client.15537 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.898+0000 7f256d7fa700 1 -- 172.21.15.39:0/2535346918 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588f596e7b0 con 0x5588f59b88e0 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.898+0000 7f256d7fa700 10 client.15537 handle_command_reply: tid=0 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.898+0000 7f259841bb80 1 client.15537 shutdown 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.898+0000 7f259841bb80 1 -- 172.21.15.39:0/2535346918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 msgr2=0x5588f59a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.898+0000 7f259841bb80 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a8400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5588f599a790 tx=0x7f25740047c0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f259841bb80 1 -- 172.21.15.39:0/2535346918 shutdown_connections 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f259841bb80 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f59ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f259841bb80 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5588f59b88e0 0x5588f59bad00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f259841bb80 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f259841bb80 1 --2- 172.21.15.39:0/2535346918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f59a8400 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f256d7fa700 0 client.15537 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f259841bb80 1 -- 172.21.15.39:0/2535346918 >> 172.21.15.39:0/2535346918 conn(0x5588f58fd1b0 msgr2=0x5588f599df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f259841bb80 1 -- 172.21.15.39:0/2535346918 shutdown_connections 2024-05-24T11:39:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f259841bb80 1 -- 172.21.15.39:0/2535346918 wait complete. 2024-05-24T11:39:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.899+0000 7f259841bb80 20 client.15537 trim_cache size 0 max 16384 2024-05-24T11:39:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f259841bb80 1 Processor -- start 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f259841bb80 1 -- start start 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f259841bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f5999910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f259841bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a4950 0x5588f59a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f259841bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a5270 0x5588f59a2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f259841bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f5994330 con 0x5588f59a5270 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f259841bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f59a2e70 con 0x5588f59a4950 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f259841bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f599b4b0 con 0x5588f599f480 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f258a7fd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a5270 0x5588f59a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f2589ffc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f5999910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f258a7fd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a5270 0x5588f59a2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39890/0 (socket says 172.21.15.39:39890) 2024-05-24T11:39:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f258a7fd700 1 -- 172.21.15.39:0/2878409444 learned_addr learned my addr 172.21.15.39:0/2878409444 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.914+0000 7f25897fb700 1 --2- 172.21.15.39:0/2878409444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a4950 0x5588f59a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f258a7fd700 1 -- 172.21.15.39:0/2878409444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 msgr2=0x5588f5999910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f258a7fd700 1 --2- 172.21.15.39:0/2878409444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f5999910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f258a7fd700 1 -- 172.21.15.39:0/2878409444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a4950 msgr2=0x5588f59a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f258a7fd700 1 --2- 172.21.15.39:0/2878409444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a4950 0x5588f59a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f258a7fd700 1 -- 172.21.15.39:0/2878409444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588f58b5500 con 0x5588f59a5270 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f25897fb700 1 --2- 172.21.15.39:0/2878409444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a4950 0x5588f59a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f258a7fd700 1 --2- 172.21.15.39:0/2878409444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a5270 0x5588f59a2930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f257c0060b0 tx=0x7f257c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14e9626f79125ee server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f2578ff9700 1 -- 172.21.15.39:0/2878409444 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c010070 con 0x5588f59a5270 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f2578ff9700 1 -- 172.21.15.39:0/2878409444 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f257c011040 con 0x5588f59a5270 2024-05-24T11:39:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.915+0000 7f2578ff9700 1 -- 172.21.15.39:0/2878409444 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c017810 con 0x5588f59a5270 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 -- 172.21.15.39:0/2878409444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a5270 msgr2=0x5588f59a2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 --2- 172.21.15.39:0/2878409444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a5270 0x5588f59a2930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f257c0060b0 tx=0x7f257c00b040 comp rx=0 tx=0).stop 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 -- 172.21.15.39:0/2878409444 shutdown_connections 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 --2- 172.21.15.39:0/2878409444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f5999910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 --2- 172.21.15.39:0/2878409444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a5270 0x5588f59a2930 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 --2- 172.21.15.39:0/2878409444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a4950 0x5588f59a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 -- 172.21.15.39:0/2878409444 >> 172.21.15.39:0/2878409444 conn(0x5588f58fd1b0 msgr2=0x5588f59b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 -- 172.21.15.39:0/2878409444 shutdown_connections 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 -- 172.21.15.39:0/2878409444 wait complete. 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.916+0000 7f259841bb80 1 -- start start 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f259841bb80 1 Processor -- start 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f259841bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f5a09d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f259841bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f5a0a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f259841bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f259841bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f5a0aa50 con 0x5588f59a4950 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f259841bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f5a0abc0 con 0x5588f59a5270 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f259841bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f59add40 con 0x5588f599f480 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f259841bb80 10 client.0 _ll_register_callbacks cb 0x5588f5894ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f25897fb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f5a0a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f25897fb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f5a0a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39898/0 (socket says 172.21.15.39:39898) 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f25897fb700 1 -- 172.21.15.39:0/3837109251 learned_addr learned my addr 172.21.15.39:0/3837109251 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f2589ffc700 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f5a09d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f258a7fd700 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f25897fb700 1 -- 172.21.15.39:0/3837109251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 msgr2=0x5588f5a09d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f25897fb700 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f5a09d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f25897fb700 1 -- 172.21.15.39:0/3837109251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 msgr2=0x5588f59ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f25897fb700 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f25897fb700 1 -- 172.21.15.39:0/3837109251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c003a00 con 0x5588f59a4950 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.917+0000 7f258a7fd700 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f25897fb700 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f5a0a2c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2574026ef0 tx=0x7f257401def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab18cdc426798359 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f256d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f256d7fa700 1 -- 172.21.15.39:0/3837109251 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574022050 con 0x5588f59a4950 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f256d7fa700 1 -- 172.21.15.39:0/3837109251 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f257401c550 con 0x5588f59a4950 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f259841bb80 1 -- 172.21.15.39:0/3837109251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25800a2f50 con 0x5588f59a4950 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f256d7fa700 1 -- 172.21.15.39:0/3837109251 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257401cbc0 con 0x5588f59a4950 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f259841bb80 10 client.15543 fetch_fsmap learned FSMap version 27 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f259841bb80 1 -- 172.21.15.39:0/3837109251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588f5774c70 con 0x5588f59a4950 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f256d7fa700 1 -- 172.21.15.39:0/3837109251 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25800a2f50 con 0x5588f59a4950 2024-05-24T11:39:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f256d7fa700 1 -- 172.21.15.39:0/3837109251 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2574026b00 con 0x5588f59a4950 2024-05-24T11:39:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f259841bb80 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:39:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f259841bb80 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:39:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f259841bb80 20 client.15543 populate_metadata read hostname 'smithi039' 2024-05-24T11:39:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f259841bb80 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5588f5a20a50 0x5588f5a22e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f259841bb80 4 client.15543 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:39:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.918+0000 7f259841bb80 1 -- 172.21.15.39:0/3837109251 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588f58e7e60 con 0x5588f5a20a50 2024-05-24T11:39:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.919+0000 7f2589ffc700 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5588f5a20a50 0x5588f5a22e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.919+0000 7f256d7fa700 10 client.15543 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.919+0000 7f2589ffc700 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5588f5a20a50 0x5588f5a22e70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1244ee5d0de6d246 server_cookie=886238a42f62a2d8 in_seq=0 out_seq=0 2024-05-24T11:39:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f256d7fa700 1 -- 172.21.15.39:0/3837109251 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588f58e7e60 con 0x5588f5a20a50 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f256d7fa700 10 client.15543 handle_command_reply: tid=0 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 client.15543 shutdown 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 -- 172.21.15.39:0/3837109251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 msgr2=0x5588f5a0a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f5a0a2c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2574026ef0 tx=0x7f257401def0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 -- 172.21.15.39:0/3837109251 shutdown_connections 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5588f599f480 0x5588f5a09d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5588f5a20a50 0x5588f5a22e70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5588f59a5270 0x5588f59ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 --2- 172.21.15.39:0/3837109251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5588f59a4950 0x5588f5a0a2c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f256d7fa700 0 client.15543 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 -- 172.21.15.39:0/3837109251 >> 172.21.15.39:0/3837109251 conn(0x5588f58fd1b0 msgr2=0x5588f59a6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 -- 172.21.15.39:0/3837109251 shutdown_connections 2024-05-24T11:39:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 1 -- 172.21.15.39:0/3837109251 wait complete. 2024-05-24T11:39:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.943+0000 7f259841bb80 20 client.15543 trim_cache size 0 max 16384 2024-05-24T11:39:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:39:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:39:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:39:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 -- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f25700a13b0 msgr2=0x7f25700a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f25700a13b0 0x7f25700a3870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f258419aa00 tx=0x7f2574009310 comp rx=0 tx=0).stop 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 -- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 msgr2=0x7f25841994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 0x7f25841994b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2580010f90 tx=0x7f2580010fc0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 -- 172.21.15.39:0/3546113283 shutdown_connections 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258419dd90 0x7f258419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f25700a13b0 0x7f25700a3870 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2584101110 0x7f25841994b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 --2- 172.21.15.39:0/3546113283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2584106230 0x7f25841999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 -- 172.21.15.39:0/3546113283 >> 172.21.15.39:0/3546113283 conn(0x7f25840fc9b0 msgr2=0x7f25840fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 -- 172.21.15.39:0/3546113283 shutdown_connections 2024-05-24T11:39:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:20.944+0000 7f258c260700 1 -- 172.21.15.39:0/3546113283 wait complete. 2024-05-24T11:39:22.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:22 smithi039 ceph-mon[42919]: pgmap v37: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 MiB/s wr, 33 op/s 2024-05-24T11:39:22.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:22 smithi053 ceph-mon[47448]: pgmap v37: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 MiB/s wr, 33 op/s 2024-05-24T11:39:22.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:22 smithi137 ceph-mon[47907]: pgmap v37: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 MiB/s wr, 33 op/s 2024-05-24T11:39:23.955 INFO:teuthology.orchestra.run.smithi039.stderr:done. 2024-05-24T11:39:23.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-24T11:39:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:starting vacuum...end. 2024-05-24T11:39:24.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:24 smithi039 ceph-mon[42919]: pgmap v38: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 29 op/s 2024-05-24T11:39:24.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:24 smithi053 ceph-mon[47448]: pgmap v38: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 29 op/s 2024-05-24T11:39:24.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:24 smithi137 ceph-mon[47907]: pgmap v38: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 29 op/s 2024-05-24T11:39:26.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:26 smithi039 ceph-mon[42919]: pgmap v39: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 54 MiB/s wr, 51 op/s 2024-05-24T11:39:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:26 smithi053 ceph-mon[47448]: pgmap v39: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 54 MiB/s wr, 51 op/s 2024-05-24T11:39:26.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:26 smithi137 ceph-mon[47907]: pgmap v39: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 54 MiB/s wr, 51 op/s 2024-05-24T11:39:28.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:28 smithi039 ceph-mon[42919]: pgmap v40: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 37 op/s 2024-05-24T11:39:28.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:28 smithi053 ceph-mon[47448]: pgmap v40: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 37 op/s 2024-05-24T11:39:28.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:28 smithi137 ceph-mon[47907]: pgmap v40: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 37 op/s 2024-05-24T11:39:29.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 5.0 s, 955.4 tps, lat 98.079 ms stddev 150.771 2024-05-24T11:39:30.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:30 smithi053 ceph-mon[47448]: pgmap v41: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 48 MiB/s wr, 160 op/s 2024-05-24T11:39:30.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:30 smithi137 ceph-mon[47907]: pgmap v41: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 48 MiB/s wr, 160 op/s 2024-05-24T11:39:30.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:30 smithi039 ceph-mon[42919]: pgmap v41: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 48 MiB/s wr, 160 op/s 2024-05-24T11:39:32.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:32 smithi053 ceph-mon[47448]: pgmap v42: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 29 MiB/s wr, 146 op/s 2024-05-24T11:39:32.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:32 smithi137 ceph-mon[47907]: pgmap v42: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 29 MiB/s wr, 146 op/s 2024-05-24T11:39:32.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:32 smithi039 ceph-mon[42919]: pgmap v42: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 29 MiB/s wr, 146 op/s 2024-05-24T11:39:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.106+0000 7fe2b0932700 1 -- 172.21.15.39:0/1243872532 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400ba10 con 0x7fe2ac0ffdd0 2024-05-24T11:39:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.106+0000 7fe2b3b98700 1 -- 172.21.15.39:0/1243872532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2ac0ffdd0 msgr2=0x7fe2ac100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.106+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/1243872532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2ac0ffdd0 0x7fe2ac100250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009990 tx=0x7fe2a4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.107+0000 7fe2b3b98700 1 -- 172.21.15.39:0/1243872532 shutdown_connections 2024-05-24T11:39:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.107+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/1243872532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2ac105e90 0x7fe2ac106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.107+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/1243872532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2ac100790 0x7fe2ac10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.107+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/1243872532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2ac0ffdd0 0x7fe2ac100250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.107+0000 7fe2b3b98700 1 -- 172.21.15.39:0/1243872532 >> 172.21.15.39:0/1243872532 conn(0x7fe2ac076fd0 msgr2=0x7fe2ac0773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.107+0000 7fe2b3b98700 1 -- 172.21.15.39:0/1243872532 shutdown_connections 2024-05-24T11:39:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.107+0000 7fe2b3b98700 1 -- 172.21.15.39:0/1243872532 wait complete. 2024-05-24T11:39:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.108+0000 7fe2b3b98700 1 Processor -- start 2024-05-24T11:39:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.108+0000 7fe2b3b98700 1 -- start start 2024-05-24T11:39:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.108+0000 7fe2b3b98700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2ac0ffdd0 0x7fe2ac10c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.108+0000 7fe2b3b98700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2ac100790 0x7fe2ac1088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.108+0000 7fe2b3b98700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2ac105e90 0x7fe2ac108e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.108+0000 7fe2b3b98700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac109400 con 0x7fe2ac100790 2024-05-24T11:39:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.108+0000 7fe2b3b98700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac109570 con 0x7fe2ac0ffdd0 2024-05-24T11:39:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.108+0000 7fe2b3b98700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac109730 con 0x7fe2ac105e90 2024-05-24T11:39:34.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 10.0 s, 1251.8 tps, lat 79.711 ms stddev 112.671 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2ac100790 0x7fe2ac1088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2ac100790 0x7fe2ac1088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56990/0 (socket says 172.21.15.39:56990) 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b1133700 1 -- 172.21.15.39:0/2243252438 learned_addr learned my addr 172.21.15.39:0/2243252438 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b2135700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2ac105e90 0x7fe2ac108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b1934700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2ac0ffdd0 0x7fe2ac10c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b2135700 1 -- 172.21.15.39:0/2243252438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2ac0ffdd0 msgr2=0x7fe2ac10c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b2135700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2ac0ffdd0 0x7fe2ac10c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b2135700 1 -- 172.21.15.39:0/2243252438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2ac100790 msgr2=0x7fe2ac1088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b2135700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2ac100790 0x7fe2ac1088f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.109+0000 7fe2b2135700 1 -- 172.21.15.39:0/2243252438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a40095f0 con 0x7fe2ac105e90 2024-05-24T11:39:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.110+0000 7fe2b2135700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2ac105e90 0x7fe2ac108e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c7e0 tx=0x7fe2a800caf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.110+0000 7fe2a2ffd700 1 -- 172.21.15.39:0/2243252438 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80117f0 con 0x7fe2ac105e90 2024-05-24T11:39:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.110+0000 7fe2a2ffd700 1 -- 172.21.15.39:0/2243252438 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2a800d040 con 0x7fe2ac105e90 2024-05-24T11:39:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.110+0000 7fe2a2ffd700 1 -- 172.21.15.39:0/2243252438 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8012e00 con 0x7fe2ac105e90 2024-05-24T11:39:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.110+0000 7fe2b3b98700 1 -- 172.21.15.39:0/2243252438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac0696f0 con 0x7fe2ac105e90 2024-05-24T11:39:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.110+0000 7fe2b3b98700 1 -- 172.21.15.39:0/2243252438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac069bb0 con 0x7fe2ac105e90 2024-05-24T11:39:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.113+0000 7fe2a2ffd700 1 -- 172.21.15.39:0/2243252438 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fe2a801c460 con 0x7fe2ac105e90 2024-05-24T11:39:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.114+0000 7fe2bfd53b80 1 Processor -- start 2024-05-24T11:39:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.114+0000 7fe2bfd53b80 1 -- start start 2024-05-24T11:39:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5765950 0x55bba5765d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5766270 0x55bba575ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba57608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba56b70f0 con 0x55bba5765950 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba56b4ab0 con 0x55bba5766270 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba56b03a0 con 0x55bba5760480 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba57608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5765950 0x55bba5765d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5765950 0x55bba5765d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57004/0 (socket says 172.21.15.39:57004) 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2b1133700 1 -- 172.21.15.39:0/221897474 learned_addr learned my addr 172.21.15.39:0/221897474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.115+0000 7fe2a2ffd700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe2940a1430 0x7fe2940a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe2a2ffd700 1 -- 172.21.15.39:0/2243252438 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe2a80ccbb0 con 0x7fe2ac105e90 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe2b1133700 1 -- 172.21.15.39:0/221897474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5766270 msgr2=0x55bba575ff40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe2b1133700 1 --2- 172.21.15.39:0/221897474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5766270 0x55bba575ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe2b1133700 1 -- 172.21.15.39:0/221897474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5765950 msgr2=0x55bba5765d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:34.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe2b1133700 1 --2- 172.21.15.39:0/221897474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5765950 0x55bba5765d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe2b1133700 1 -- 172.21.15.39:0/221897474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bba5531350 con 0x55bba5760480 2024-05-24T11:39:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe2b1133700 1 --2- 172.21.15.39:0/221897474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba57608a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400b900 tx=0x7fe2a4005940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=784762d5cc37d77b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe29bfff700 1 -- 172.21.15.39:0/221897474 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400bc40 con 0x55bba5760480 2024-05-24T11:39:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe29bfff700 1 -- 172.21.15.39:0/221897474 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2a400bda0 con 0x55bba5760480 2024-05-24T11:39:34.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.116+0000 7fe29bfff700 1 -- 172.21.15.39:0/221897474 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4004cb0 con 0x55bba5760480 2024-05-24T11:39:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.117+0000 7fe2b2135700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe2940a1430 0x7fe2940a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.117+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/221897474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 msgr2=0x55bba57608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.117+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/221897474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba57608a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400b900 tx=0x7fe2a4005940 comp rx=0 tx=0).stop 2024-05-24T11:39:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.117+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/221897474 shutdown_connections 2024-05-24T11:39:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.117+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/221897474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba57608a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.117+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/221897474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5765950 0x55bba5765d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.117+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/221897474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5766270 0x55bba575ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.117+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/221897474 >> 172.21.15.39:0/221897474 conn(0x55bba56be1b0 msgr2=0x55bba575a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:34.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.118+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/221897474 shutdown_connections 2024-05-24T11:39:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.118+0000 7fe2b2135700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe2940a1430 0x7fe2940a38f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80ca1c0 tx=0x7fe2a8009f90 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.118+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/221897474 wait complete. 2024-05-24T11:39:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.118+0000 7fe2bfd53b80 1 -- start start 2024-05-24T11:39:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2bfd53b80 1 Processor -- start 2024-05-24T11:39:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba576c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba5769500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba5769c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba564c500 con 0x55bba5766270 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba564c1a0 con 0x55bba5765950 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba5533640 con 0x55bba5760480 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2bfd53b80 10 client.0 _ll_register_callbacks cb 0x55bba5655ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba5769c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.119+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba576c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba576c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:41280/0 (socket says 172.21.15.39:41280) 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2b1133700 1 -- 172.21.15.39:0/1102013061 learned_addr learned my addr 172.21.15.39:0/1102013061 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2b1934700 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba5769500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2b1133700 1 -- 172.21.15.39:0/1102013061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 msgr2=0x55bba5769500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2b1133700 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba5769500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2b1133700 1 -- 172.21.15.39:0/1102013061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 msgr2=0x55bba5769c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2b1133700 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba5769c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2b1133700 1 -- 172.21.15.39:0/1102013061 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a4024040 con 0x55bba5760480 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2b1133700 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba576c210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005910 tx=0x7fe2a4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=760a987f0b1ad309 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2997fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2997fa700 1 -- 172.21.15.39:0/1102013061 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a402bb20 con 0x55bba5760480 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/1102013061 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bba5609210 con 0x55bba5760480 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.120+0000 7fe2997fa700 1 -- 172.21.15.39:0/1102013061 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe2a4004500 con 0x55bba5760480 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2997fa700 1 -- 172.21.15.39:0/1102013061 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2a402bc80 con 0x55bba5760480 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2bfd53b80 10 client.25147 fetch_fsmap learned FSMap version 27 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/1102013061 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bba54ddb00 con 0x55bba5760480 2024-05-24T11:39:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2997fa700 1 -- 172.21.15.39:0/1102013061 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bba5609210 con 0x55bba5760480 2024-05-24T11:39:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2997fa700 1 -- 172.21.15.39:0/1102013061 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2a4038060 con 0x55bba5760480 2024-05-24T11:39:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2bfd53b80 10 client.25147 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:39:34.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2bfd53b80 10 client.25147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2bfd53b80 20 client.25147 populate_metadata read hostname 'smithi039' 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bba57797e0 0x55bba577bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2bfd53b80 4 client.25147 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.121+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/1102013061 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bba572f7b0 con 0x55bba57797e0 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.122+0000 7fe2b1934700 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bba57797e0 0x55bba577bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.122+0000 7fe2b1934700 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bba57797e0 0x55bba577bc00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b99ba0904ad32c server_cookie=101c6f11b50e61bc in_seq=0 out_seq=0 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.122+0000 7fe2997fa700 10 client.25147 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.123+0000 7fe2997fa700 1 -- 172.21.15.39:0/1102013061 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bba572f7b0 con 0x55bba57797e0 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.123+0000 7fe2997fa700 10 client.25147 handle_command_reply: tid=0 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.123+0000 7fe2bfd53b80 1 client.25147 shutdown 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/1102013061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 msgr2=0x55bba576c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba576c210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4005910 tx=0x7fe2a4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/1102013061 shutdown_connections 2024-05-24T11:39:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba576c210 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bba57797e0 0x55bba577bc00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba5769c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/1102013061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba5769500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/1102013061 >> 172.21.15.39:0/1102013061 conn(0x55bba56be1b0 msgr2=0x55bba57644a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2997fa700 0 client.25147 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.124+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/1102013061 shutdown_connections 2024-05-24T11:39:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.125+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/1102013061 wait complete. 2024-05-24T11:39:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.125+0000 7fe2bfd53b80 20 client.25147 trim_cache size 0 max 16384 2024-05-24T11:39:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2bfd53b80 1 Processor -- start 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2bfd53b80 1 -- start start 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba57756c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba575a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba576bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba576ab00 con 0x55bba5766270 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba576a220 con 0x55bba5765950 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba5780db0 con 0x55bba5760480 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba576bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2b1934700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba575a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba576bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57024/0 (socket says 172.21.15.39:57024) 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2b1133700 1 -- 172.21.15.39:0/3240185953 learned_addr learned my addr 172.21.15.39:0/3240185953 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2b1934700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba575a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49506/0 (socket says 172.21.15.39:49506) 2024-05-24T11:39:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.149+0000 7fe2b1133700 1 --2- 172.21.15.39:0/3240185953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba57756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.150+0000 7fe2b1133700 1 -- 172.21.15.39:0/3240185953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 msgr2=0x55bba57756c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.150+0000 7fe2b1133700 1 --2- 172.21.15.39:0/3240185953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba57756c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.150+0000 7fe2b1133700 1 -- 172.21.15.39:0/3240185953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 msgr2=0x55bba575a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.150+0000 7fe2b1133700 1 --2- 172.21.15.39:0/3240185953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba575a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.150+0000 7fe2b1133700 1 -- 172.21.15.39:0/3240185953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bba5676500 con 0x55bba5766270 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.150+0000 7fe2b1133700 1 --2- 172.21.15.39:0/3240185953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba576bd40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4006010 tx=0x7fe2a400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a69ce3fa75d2fe server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.150+0000 7fe2a0da5700 1 -- 172.21.15.39:0/3240185953 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a40177a0 con 0x55bba5766270 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.150+0000 7fe2a0da5700 1 -- 172.21.15.39:0/3240185953 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2a4017e70 con 0x55bba5766270 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.150+0000 7fe2a0da5700 1 -- 172.21.15.39:0/3240185953 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a402bc60 con 0x55bba5766270 2024-05-24T11:39:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.151+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3240185953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 msgr2=0x55bba576bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.151+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3240185953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba576bd40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4006010 tx=0x7fe2a400b260 comp rx=0 tx=0).stop 2024-05-24T11:39:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.151+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3240185953 shutdown_connections 2024-05-24T11:39:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.151+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3240185953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5760480 0x55bba57756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.151+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3240185953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5766270 0x55bba576bd40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.151+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3240185953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba575a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.151+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3240185953 >> 172.21.15.39:0/3240185953 conn(0x55bba56be1b0 msgr2=0x55bba575e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.152+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3240185953 shutdown_connections 2024-05-24T11:39:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.152+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3240185953 wait complete. 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.152+0000 7fe2bfd53b80 1 -- start start 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.152+0000 7fe2bfd53b80 1 Processor -- start 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5760480 0x55bba5770e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba57633f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2bfd53b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5766270 0x55bba5763930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba5763e70 con 0x55bba5760480 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba5763fe0 con 0x55bba5765950 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2bfd53b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba57641a0 con 0x55bba5766270 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2bfd53b80 10 client.0 _ll_register_callbacks cb 0x55bba5655ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:39:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5760480 0x55bba5770e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2b1934700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba57633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2b1934700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba57633f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49520/0 (socket says 172.21.15.39:49520) 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2b1133700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5760480 0x55bba5770e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57036/0 (socket says 172.21.15.39:57036) 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2b1934700 1 -- 172.21.15.39:0/3697043051 learned_addr learned my addr 172.21.15.39:0/3697043051 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.153+0000 7fe2b1133700 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5766270 0x55bba5763930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2b1934700 1 -- 172.21.15.39:0/3697043051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5766270 msgr2=0x55bba5763930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2b1934700 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5766270 0x55bba5763930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2b1934700 1 -- 172.21.15.39:0/3697043051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5760480 msgr2=0x55bba5770e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2b1934700 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5760480 0x55bba5770e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2b1934700 1 -- 172.21.15.39:0/3697043051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bba54ddb00 con 0x55bba5765950 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2b1934700 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba57633f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x55bba577c4d0 tx=0x7fe29c009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eacf3731e933d7a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2997fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:39:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2997fa700 1 -- 172.21.15.39:0/3697043051 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c019040 con 0x55bba5765950 2024-05-24T11:39:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3697043051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2a400bda0 con 0x55bba5765950 2024-05-24T11:39:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2997fa700 1 -- 172.21.15.39:0/3697043051 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe29c025e40 con 0x55bba5765950 2024-05-24T11:39:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2997fa700 1 -- 172.21.15.39:0/3697043051 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe29c025340 con 0x55bba5765950 2024-05-24T11:39:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2bfd53b80 10 client.25244 fetch_fsmap learned FSMap version 27 2024-05-24T11:39:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3697043051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bba5535c70 con 0x55bba5765950 2024-05-24T11:39:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.154+0000 7fe2997fa700 1 -- 172.21.15.39:0/3697043051 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe29c03b020 con 0x55bba5765950 2024-05-24T11:39:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.155+0000 7fe2997fa700 1 -- 172.21.15.39:0/3697043051 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe29c003c20 con 0x55bba5765950 2024-05-24T11:39:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.155+0000 7fe2bfd53b80 10 client.25244 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:39:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.155+0000 7fe2bfd53b80 10 client.25244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:39:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.155+0000 7fe2bfd53b80 20 client.25244 populate_metadata read hostname 'smithi039' 2024-05-24T11:39:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.155+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bba57d0400 0x55bba57d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.156+0000 7fe2bfd53b80 4 client.25244 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:39:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.156+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3697043051 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bba56a8e60 con 0x55bba57d0400 2024-05-24T11:39:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.156+0000 7fe2b1133700 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bba57d0400 0x55bba57d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.156+0000 7fe2b1133700 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bba57d0400 0x55bba57d2820 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989feb00737d97be server_cookie=74c4aff466e6ba32 in_seq=0 out_seq=0 2024-05-24T11:39:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.156+0000 7fe2997fa700 10 client.25244 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:34.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:34 smithi053 ceph-mon[47448]: pgmap v43: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 29 MiB/s wr, 146 op/s 2024-05-24T11:39:34.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:34 smithi137 ceph-mon[47907]: pgmap v43: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 29 MiB/s wr, 146 op/s 2024-05-24T11:39:34.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:34 smithi039 ceph-mon[42919]: pgmap v43: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 29 MiB/s wr, 146 op/s 2024-05-24T11:39:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.596+0000 7fe2997fa700 1 -- 172.21.15.39:0/3697043051 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bba56a8e60 con 0x55bba57d0400 2024-05-24T11:39:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.596+0000 7fe2997fa700 10 client.25244 handle_command_reply: tid=0 2024-05-24T11:39:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.596+0000 7fe2bfd53b80 1 client.25244 shutdown 2024-05-24T11:39:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3697043051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 msgr2=0x55bba57633f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba57633f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x55bba577c4d0 tx=0x7fe29c009fd0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3697043051 shutdown_connections 2024-05-24T11:39:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bba5766270 0x55bba5763930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55bba57d0400 0x55bba57d2820 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bba5760480 0x55bba5770e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 --2- 172.21.15.39:0/3697043051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bba5765950 0x55bba57633f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2997fa700 0 client.25244 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3697043051 >> 172.21.15.39:0/3697043051 conn(0x55bba56be1b0 msgr2=0x55bba5768d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3697043051 shutdown_connections 2024-05-24T11:39:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 1 -- 172.21.15.39:0/3697043051 wait complete. 2024-05-24T11:39:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.597+0000 7fe2bfd53b80 20 client.25244 trim_cache size 0 max 16384 2024-05-24T11:39:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:39:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:39:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:39:34.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:39:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 -- 172.21.15.39:0/2243252438 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe2940a1430 msgr2=0x7fe2940a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe2940a1430 0x7fe2940a38f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80ca1c0 tx=0x7fe2a8009f90 comp rx=0 tx=0).stop 2024-05-24T11:39:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 -- 172.21.15.39:0/2243252438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2ac105e90 msgr2=0x7fe2ac108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2ac105e90 0x7fe2ac108e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c7e0 tx=0x7fe2a800caf0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 -- 172.21.15.39:0/2243252438 shutdown_connections 2024-05-24T11:39:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2ac105e90 0x7fe2ac108e30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe2940a1430 0x7fe2940a38f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2ac0ffdd0 0x7fe2ac10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 --2- 172.21.15.39:0/2243252438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2ac100790 0x7fe2ac1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 -- 172.21.15.39:0/2243252438 >> 172.21.15.39:0/2243252438 conn(0x7fe2ac076fd0 msgr2=0x7fe2ac0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 -- 172.21.15.39:0/2243252438 shutdown_connections 2024-05-24T11:39:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:34.598+0000 7fe2b3b98700 1 -- 172.21.15.39:0/2243252438 wait complete. 2024-05-24T11:39:36.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:36 smithi053 ceph-mon[47448]: pgmap v44: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 42 MiB/s wr, 289 op/s 2024-05-24T11:39:36.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:36 smithi137 ceph-mon[47907]: pgmap v44: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 42 MiB/s wr, 289 op/s 2024-05-24T11:39:36.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:36 smithi039 ceph-mon[42919]: pgmap v44: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 42 MiB/s wr, 289 op/s 2024-05-24T11:39:38.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:38 smithi053 ceph-mon[47448]: pgmap v45: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 26 MiB/s wr, 267 op/s 2024-05-24T11:39:38.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:38 smithi137 ceph-mon[47907]: pgmap v45: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 26 MiB/s wr, 267 op/s 2024-05-24T11:39:38.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:38 smithi039 ceph-mon[42919]: pgmap v45: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 26 MiB/s wr, 267 op/s 2024-05-24T11:39:39.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 15.0 s, 1037.8 tps, lat 90.477 ms stddev 107.455 2024-05-24T11:39:40.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:40 smithi053 ceph-mon[47448]: pgmap v46: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 38 MiB/s wr, 748 op/s 2024-05-24T11:39:40.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:40 smithi137 ceph-mon[47907]: pgmap v46: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 38 MiB/s wr, 748 op/s 2024-05-24T11:39:40.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:40 smithi039 ceph-mon[42919]: pgmap v46: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 38 MiB/s wr, 748 op/s 2024-05-24T11:39:41.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:41 smithi053 ceph-mon[47448]: pgmap v47: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 25 MiB/s wr, 625 op/s 2024-05-24T11:39:41.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:41 smithi137 ceph-mon[47907]: pgmap v47: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 25 MiB/s wr, 625 op/s 2024-05-24T11:39:41.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:41 smithi039 ceph-mon[42919]: pgmap v47: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 25 MiB/s wr, 625 op/s 2024-05-24T11:39:43.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:43 smithi039 ceph-mon[42919]: pgmap v48: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 25 MiB/s wr, 625 op/s 2024-05-24T11:39:43.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:43 smithi053 ceph-mon[47448]: pgmap v48: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 25 MiB/s wr, 625 op/s 2024-05-24T11:39:43.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:43 smithi137 ceph-mon[47907]: pgmap v48: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 25 MiB/s wr, 625 op/s 2024-05-24T11:39:44.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 20.0 s, 1199.3 tps, lat 88.706 ms stddev 122.901 2024-05-24T11:39:44.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:44 smithi039 ceph-mon[42919]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-05-24T11:39:44.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:44 smithi053 ceph-mon[47448]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-05-24T11:39:44.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:44 smithi137 ceph-mon[47907]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-05-24T11:39:45.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:45 smithi039 ceph-mon[42919]: pgmap v49: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 38 MiB/s wr, 1.26k op/s 2024-05-24T11:39:45.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:45 smithi053 ceph-mon[47448]: pgmap v49: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 38 MiB/s wr, 1.26k op/s 2024-05-24T11:39:45.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:45 smithi137 ceph-mon[47907]: pgmap v49: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 38 MiB/s wr, 1.26k op/s 2024-05-24T11:39:47.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:47 smithi039 ceph-mon[42919]: pgmap v50: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 25 MiB/s wr, 1.11k op/s 2024-05-24T11:39:47.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:47 smithi053 ceph-mon[47448]: pgmap v50: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 25 MiB/s wr, 1.11k op/s 2024-05-24T11:39:47.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:47 smithi137 ceph-mon[47907]: pgmap v50: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 25 MiB/s wr, 1.11k op/s 2024-05-24T11:39:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.767+0000 7f7b6325d700 1 -- 172.21.15.39:0/3893673280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c1140f0 msgr2=0x7f7b5c1164e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.767+0000 7f7b6325d700 1 --2- 172.21.15.39:0/3893673280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c1140f0 0x7f7b5c1164e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009f90 tx=0x7f7b5400b330 comp rx=0 tx=0).stop 2024-05-24T11:39:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.767+0000 7f7b6325d700 1 -- 172.21.15.39:0/3893673280 shutdown_connections 2024-05-24T11:39:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.767+0000 7f7b6325d700 1 --2- 172.21.15.39:0/3893673280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c107fa0 0x7f7b5c108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.767+0000 7f7b6325d700 1 --2- 172.21.15.39:0/3893673280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c1140f0 0x7f7b5c1164e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.767+0000 7f7b6325d700 1 --2- 172.21.15.39:0/3893673280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c1088c0 0x7f7b5c113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.767+0000 7f7b6325d700 1 -- 172.21.15.39:0/3893673280 >> 172.21.15.39:0/3893673280 conn(0x7f7b5c06bf80 msgr2=0x7f7b5c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.767+0000 7f7b6325d700 1 -- 172.21.15.39:0/3893673280 shutdown_connections 2024-05-24T11:39:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.767+0000 7f7b6325d700 1 -- 172.21.15.39:0/3893673280 wait complete. 2024-05-24T11:39:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.769+0000 7f7b6325d700 1 Processor -- start 2024-05-24T11:39:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.769+0000 7f7b6325d700 1 -- start start 2024-05-24T11:39:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b6325d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c107fa0 0x7f7b5c1a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b6325d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c1088c0 0x7f7b5c1a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b6325d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c1140f0 0x7f7b5c1a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b6325d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c1a6ce0 con 0x7f7b5c1088c0 2024-05-24T11:39:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b6325d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c1a6e50 con 0x7f7b5c107fa0 2024-05-24T11:39:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b6325d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c1a7010 con 0x7f7b5c1140f0 2024-05-24T11:39:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c1088c0 0x7f7b5c1a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b60ff9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c107fa0 0x7f7b5c1a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c1088c0 0x7f7b5c1a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52638/0 (socket says 172.21.15.39:52638) 2024-05-24T11:39:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b60ff9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c107fa0 0x7f7b5c1a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49100/0 (socket says 172.21.15.39:49100) 2024-05-24T11:39:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b5bfff700 1 -- 172.21.15.39:0/2125527474 learned_addr learned my addr 172.21.15.39:0/2125527474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b617fa700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c1140f0 0x7f7b5c1a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b60ff9700 1 -- 172.21.15.39:0/2125527474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c1140f0 msgr2=0x7f7b5c1a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c1140f0 0x7f7b5c1a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b60ff9700 1 -- 172.21.15.39:0/2125527474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c1088c0 msgr2=0x7f7b5c1a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c1088c0 0x7f7b5c1a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.770+0000 7f7b60ff9700 1 -- 172.21.15.39:0/2125527474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b54009c40 con 0x7f7b5c107fa0 2024-05-24T11:39:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.771+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c107fa0 0x7f7b5c1a1dc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e9c0 tx=0x7f7b5000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.771+0000 7f7b59ffb700 1 -- 172.21.15.39:0/2125527474 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000c9c0 con 0x7f7b5c107fa0 2024-05-24T11:39:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.771+0000 7f7b6325d700 1 -- 172.21.15.39:0/2125527474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5c1a72f0 con 0x7f7b5c107fa0 2024-05-24T11:39:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.771+0000 7f7b59ffb700 1 -- 172.21.15.39:0/2125527474 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b50010430 con 0x7f7b5c107fa0 2024-05-24T11:39:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.771+0000 7f7b59ffb700 1 -- 172.21.15.39:0/2125527474 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50018680 con 0x7f7b5c107fa0 2024-05-24T11:39:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.771+0000 7f7b6325d700 1 -- 172.21.15.39:0/2125527474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c1aeb30 con 0x7f7b5c107fa0 2024-05-24T11:39:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.774+0000 7f7b59ffb700 1 -- 172.21.15.39:0/2125527474 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7b500105a0 con 0x7f7b5c107fa0 2024-05-24T11:39:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.775+0000 7f7b59ffb700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b4c0a1290 0x7f7b4c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.775+0000 7f7b59ffb700 1 -- 172.21.15.39:0/2125527474 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7b50014070 con 0x7f7b5c107fa0 2024-05-24T11:39:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.775+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b4c0a1290 0x7f7b4c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.776+0000 7f7b6f418b80 1 Processor -- start 2024-05-24T11:39:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.776+0000 7f7b6f418b80 1 -- start start 2024-05-24T11:39:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.776+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b4c0a1290 0x7f7b4c0a3750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c1a3310 tx=0x7f7b48009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19e8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e9270 0x55f4f19e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e3480 0x55f4f19e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f193a0f0 con 0x55f4f19e3480 2024-05-24T11:39:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f1937ab0 con 0x55f4f19e8950 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f19333a0 con 0x55f4f19e9270 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b60ff9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19e8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b60ff9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19e8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49102/0 (socket says 172.21.15.39:49102) 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b60ff9700 1 -- 172.21.15.39:0/1827559632 learned_addr learned my addr 172.21.15.39:0/1827559632 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b617fa700 1 --2- 172.21.15.39:0/1827559632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e3480 0x55f4f19e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b60ff9700 1 -- 172.21.15.39:0/1827559632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e9270 msgr2=0x55f4f19e2f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/1827559632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e9270 0x55f4f19e2f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b60ff9700 1 -- 172.21.15.39:0/1827559632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e3480 msgr2=0x55f4f19e38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/1827559632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e3480 0x55f4f19e38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b60ff9700 1 -- 172.21.15.39:0/1827559632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4f17b4350 con 0x55f4f19e8950 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.777+0000 7f7b617fa700 1 --2- 172.21.15.39:0/1827559632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e3480 0x55f4f19e38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.778+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/1827559632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19e8d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b500a1d30 tx=0x7f7b50099a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddaa454234ac8096 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.778+0000 7f7b46ffd700 1 -- 172.21.15.39:0/1827559632 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50099de0 con 0x55f4f19e8950 2024-05-24T11:39:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.778+0000 7f7b46ffd700 1 -- 172.21.15.39:0/1827559632 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b5009dc10 con 0x55f4f19e8950 2024-05-24T11:39:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.778+0000 7f7b46ffd700 1 -- 172.21.15.39:0/1827559632 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500992d0 con 0x55f4f19e8950 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1827559632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 msgr2=0x55f4f19e8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/1827559632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19e8d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b500a1d30 tx=0x7f7b50099a40 comp rx=0 tx=0).stop 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1827559632 shutdown_connections 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/1827559632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e3480 0x55f4f19e38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/1827559632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19e8d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/1827559632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e9270 0x55f4f19e2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1827559632 >> 172.21.15.39:0/1827559632 conn(0x55f4f19411b0 msgr2=0x55f4f19dd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1827559632 shutdown_connections 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1827559632 wait complete. 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.779+0000 7f7b6f418b80 1 -- start start 2024-05-24T11:39:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.780+0000 7f7b6f418b80 1 Processor -- start 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.780+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f19ec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.780+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e8950 0x55f4f19ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.780+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e9270 0x55f4f19ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.780+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f18cf500 con 0x55f4f19e8950 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.780+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f18cf1a0 con 0x55f4f19e9270 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.780+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f17b6640 con 0x55f4f19e3480 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.780+0000 7f7b6f418b80 10 client.0 _ll_register_callbacks cb 0x55f4f18d8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b60ff9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f19ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b60ff9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f19ec660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:54518/0 (socket says 172.21.15.39:54518) 2024-05-24T11:39:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b60ff9700 1 -- 172.21.15.39:0/2269430042 learned_addr learned my addr 172.21.15.39:0/2269430042 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b60ff9700 1 -- 172.21.15.39:0/2269430042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e9270 msgr2=0x55f4f19ed0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:39:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e9270 0x55f4f19ed0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b60ff9700 1 -- 172.21.15.39:0/2269430042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e8950 msgr2=0x55f4f19ecba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e8950 0x55f4f19ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b60ff9700 1 -- 172.21.15.39:0/2269430042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c1aeb30 con 0x55f4f19e3480 2024-05-24T11:39:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f19ec660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7b500ca4d0 tx=0x7f7b5009a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd0caea4a316c0b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.781+0000 7f7b3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:39:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b3bfff700 1 -- 172.21.15.39:0/2269430042 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500992d0 con 0x55f4f19e3480 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b3bfff700 1 -- 172.21.15.39:0/2269430042 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7b50099430 con 0x55f4f19e3480 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b6f418b80 1 -- 172.21.15.39:0/2269430042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4f188c210 con 0x55f4f19e3480 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b3bfff700 1 -- 172.21.15.39:0/2269430042 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b500a1dd0 con 0x55f4f19e3480 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b6f418b80 10 client.25162 fetch_fsmap learned FSMap version 27 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b6f418b80 1 -- 172.21.15.39:0/2269430042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f4f1760b00 con 0x55f4f19e3480 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b3bfff700 1 -- 172.21.15.39:0/2269430042 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f4f188c210 con 0x55f4f19e3480 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b3bfff700 1 -- 172.21.15.39:0/2269430042 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b5000cd20 con 0x55f4f19e3480 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b6f418b80 10 client.25162 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b6f418b80 10 client.25162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b6f418b80 20 client.25162 populate_metadata read hostname 'smithi039' 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.782+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f4f1a05100 0x55f4f1a074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.783+0000 7f7b6f418b80 4 client.25162 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.783+0000 7f7b6f418b80 1 -- 172.21.15.39:0/2269430042 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4f19b27b0 con 0x55f4f1a05100 2024-05-24T11:39:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.783+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f4f1a05100 0x55f4f1a074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.784+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f4f1a05100 0x55f4f1a074f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e56a29eba43d6bd server_cookie=62bf2f37e06eba13 in_seq=0 out_seq=0 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.784+0000 7f7b3bfff700 10 client.25162 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.785+0000 7f7b3bfff700 1 -- 172.21.15.39:0/2269430042 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4f19b27b0 con 0x55f4f1a05100 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.785+0000 7f7b3bfff700 10 client.25162 handle_command_reply: tid=0 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.785+0000 7f7b6f418b80 1 client.25162 shutdown 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 -- 172.21.15.39:0/2269430042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 msgr2=0x55f4f19ec660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f19ec660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7b500ca4d0 tx=0x7f7b5009a650 comp rx=0 tx=0).stop 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 -- 172.21.15.39:0/2269430042 shutdown_connections 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f19ec660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f4f1a05100 0x55f4f1a074f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e9270 0x55f4f19ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/2269430042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e8950 0x55f4f19ecba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 -- 172.21.15.39:0/2269430042 >> 172.21.15.39:0/2269430042 conn(0x55f4f19411b0 msgr2=0x55f4f19e1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b3bfff700 0 client.25162 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 -- 172.21.15.39:0/2269430042 shutdown_connections 2024-05-24T11:39:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 1 -- 172.21.15.39:0/2269430042 wait complete. 2024-05-24T11:39:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.786+0000 7f7b6f418b80 20 client.25162 trim_cache size 0 max 16384 2024-05-24T11:39:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.811+0000 7f7b6f418b80 1 Processor -- start 2024-05-24T11:39:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.811+0000 7f7b6f418b80 1 -- start start 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.811+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f1a00f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.811+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19dd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.811+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 0x55f4f19df520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.811+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f19da9e0 con 0x55f4f19e9270 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.811+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f19d8360 con 0x55f4f19e8950 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.811+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f19edc60 con 0x55f4f19e3480 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19dd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19dd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49120/0 (socket says 172.21.15.39:49120) 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b5bfff700 1 -- 172.21.15.39:0/1182975427 learned_addr learned my addr 172.21.15.39:0/1182975427 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b617fa700 1 --2- 172.21.15.39:0/1182975427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 0x55f4f19df520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/1182975427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f1a00f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b5bfff700 1 -- 172.21.15.39:0/1182975427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 msgr2=0x55f4f1a00f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/1182975427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f1a00f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b5bfff700 1 -- 172.21.15.39:0/1182975427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 msgr2=0x55f4f19df520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/1182975427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 0x55f4f19df520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b5bfff700 1 -- 172.21.15.39:0/1182975427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c109710 con 0x55f4f19e8950 2024-05-24T11:39:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.812+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/1182975427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19dd910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55f4f19edc30 tx=0x7f7b48014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1b03ff46dcaff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.813+0000 7f7b477fe700 1 -- 172.21.15.39:0/1182975427 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48022040 con 0x55f4f19e8950 2024-05-24T11:39:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.813+0000 7f7b477fe700 1 -- 172.21.15.39:0/1182975427 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b4800e3b0 con 0x55f4f19e8950 2024-05-24T11:39:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.813+0000 7f7b477fe700 1 -- 172.21.15.39:0/1182975427 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4801d670 con 0x55f4f19e8950 2024-05-24T11:39:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.813+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1182975427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 msgr2=0x55f4f19dd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.813+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/1182975427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19dd910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55f4f19edc30 tx=0x7f7b48014000 comp rx=0 tx=0).stop 2024-05-24T11:39:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.814+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1182975427 shutdown_connections 2024-05-24T11:39:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.814+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/1182975427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19e3480 0x55f4f1a00f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.814+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/1182975427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 0x55f4f19df520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.814+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/1182975427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e8950 0x55f4f19dd910 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55f4f19edc30 tx=0x7f7b48014000 comp rx=0 tx=0).stop 2024-05-24T11:39:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.814+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1182975427 >> 172.21.15.39:0/1182975427 conn(0x55f4f19411b0 msgr2=0x55f4f19e6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.814+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1182975427 shutdown_connections 2024-05-24T11:39:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.814+0000 7f7b6f418b80 1 -- 172.21.15.39:0/1182975427 wait complete. 2024-05-24T11:39:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.814+0000 7f7b6f418b80 1 -- start start 2024-05-24T11:39:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.814+0000 7f7b6f418b80 1 Processor -- start 2024-05-24T11:39:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e3480 0x55f4f19fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 0x55f4f19fa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b6f418b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19faed0 0x55f4f1a5a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f19fccc0 con 0x55f4f19e9270 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f1a5aae0 con 0x55f4f19e3480 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b6f418b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f1a5ac20 con 0x55f4f19faed0 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b6f418b80 10 client.0 _ll_register_callbacks cb 0x55f4f18d8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b60ff9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e3480 0x55f4f19fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 0x55f4f19fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b60ff9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e3480 0x55f4f19fa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49134/0 (socket says 172.21.15.39:49134) 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.815+0000 7f7b60ff9700 1 -- 172.21.15.39:0/238561355 learned_addr learned my addr 172.21.15.39:0/238561355 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b60ff9700 1 -- 172.21.15.39:0/238561355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19faed0 msgr2=0x55f4f1a5a5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:39:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19faed0 0x55f4f1a5a5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b60ff9700 1 -- 172.21.15.39:0/238561355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 msgr2=0x55f4f19fa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 0x55f4f19fa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b60ff9700 1 -- 172.21.15.39:0/238561355 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b48017070 con 0x55f4f19e3480 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 0x55f4f19fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b60ff9700 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e3480 0x55f4f19fa450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c5b0 tx=0x7f7b50009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8aa8171218a5084 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b3bfff700 1 -- 172.21.15.39:0/238561355 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500a1fd0 con 0x55f4f19e3480 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b3bfff700 1 -- 172.21.15.39:0/238561355 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f7b500992d0 con 0x55f4f19e3480 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b6f418b80 1 -- 172.21.15.39:0/238561355 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b5009dc10 con 0x55f4f19e3480 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b3bfff700 1 -- 172.21.15.39:0/238561355 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b500b7ef0 con 0x55f4f19e3480 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b6f418b80 10 client.25259 fetch_fsmap learned FSMap version 27 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.816+0000 7f7b6f418b80 1 -- 172.21.15.39:0/238561355 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f4f18f9500 con 0x55f4f19e3480 2024-05-24T11:39:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.817+0000 7f7b3bfff700 1 -- 172.21.15.39:0/238561355 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b5009dc10 con 0x55f4f19e3480 2024-05-24T11:39:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.817+0000 7f7b3bfff700 1 -- 172.21.15.39:0/238561355 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b5000cb20 con 0x55f4f19e3480 2024-05-24T11:39:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.817+0000 7f7b6f418b80 10 client.25259 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:39:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.817+0000 7f7b6f418b80 10 client.25259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:39:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.817+0000 7f7b6f418b80 20 client.25259 populate_metadata read hostname 'smithi039' 2024-05-24T11:39:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.817+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f4f1a5bf10 0x55f4f1a5e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:39:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.817+0000 7f7b6f418b80 4 client.25259 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:39:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.817+0000 7f7b6f418b80 1 -- 172.21.15.39:0/238561355 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4f192be60 con 0x55f4f1a5bf10 2024-05-24T11:39:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.817+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f4f1a5bf10 0x55f4f1a5e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:39:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.818+0000 7f7b5bfff700 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f4f1a5bf10 0x55f4f1a5e330 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22d7f3849447a62 server_cookie=f4dd163c6a4d304b in_seq=0 out_seq=0 2024-05-24T11:39:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.818+0000 7f7b3bfff700 10 client.25259 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b3bfff700 1 -- 172.21.15.39:0/238561355 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4f192be60 con 0x55f4f1a5bf10 2024-05-24T11:39:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b3bfff700 10 client.25259 handle_command_reply: tid=0 2024-05-24T11:39:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 client.25259 shutdown 2024-05-24T11:39:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 -- 172.21.15.39:0/238561355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e3480 msgr2=0x55f4f19fa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e3480 0x55f4f19fa450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c5b0 tx=0x7f7b50009f90 comp rx=0 tx=0).stop 2024-05-24T11:39:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 -- 172.21.15.39:0/238561355 shutdown_connections 2024-05-24T11:39:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4f19faed0 0x55f4f1a5a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f4f1a5bf10 0x55f4f1a5e330 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4f19e3480 0x55f4f19fa450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 --2- 172.21.15.39:0/238561355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4f19e9270 0x55f4f19fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b3bfff700 0 client.25259 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 -- 172.21.15.39:0/238561355 >> 172.21.15.39:0/238561355 conn(0x55f4f19411b0 msgr2=0x55f4f19e76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 -- 172.21.15.39:0/238561355 shutdown_connections 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.903+0000 7f7b6f418b80 1 -- 172.21.15.39:0/238561355 wait complete. 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6f418b80 20 client.25259 trim_cache size 0 max 16384 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:39:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 -- 172.21.15.39:0/2125527474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b4c0a1290 msgr2=0x7f7b4c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b4c0a1290 0x7f7b4c0a3750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c1a3310 tx=0x7f7b48009260 comp rx=0 tx=0).stop 2024-05-24T11:39:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 -- 172.21.15.39:0/2125527474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c107fa0 msgr2=0x7f7b5c1a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c107fa0 0x7f7b5c1a1dc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000e9c0 tx=0x7f7b5000ecd0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 -- 172.21.15.39:0/2125527474 shutdown_connections 2024-05-24T11:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b5c1140f0 0x7f7b5c1a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b4c0a1290 0x7f7b4c0a3750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b5c107fa0 0x7f7b5c1a1dc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 --2- 172.21.15.39:0/2125527474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b5c1088c0 0x7f7b5c1a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.904+0000 7f7b6325d700 1 -- 172.21.15.39:0/2125527474 >> 172.21.15.39:0/2125527474 conn(0x7f7b5c06bf80 msgr2=0x7f7b5c1099e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.905+0000 7f7b6325d700 1 -- 172.21.15.39:0/2125527474 shutdown_connections 2024-05-24T11:39:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:39:47.905+0000 7f7b6325d700 1 -- 172.21.15.39:0/2125527474 wait complete. 2024-05-24T11:39:49.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 25.0 s, 1245.4 tps, lat 80.291 ms stddev 98.685 2024-05-24T11:39:49.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:49 smithi039 ceph-mon[42919]: pgmap v51: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 35 MiB/s wr, 1.68k op/s 2024-05-24T11:39:49.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:49 smithi053 ceph-mon[47448]: pgmap v51: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 35 MiB/s wr, 1.68k op/s 2024-05-24T11:39:49.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:49 smithi137 ceph-mon[47907]: pgmap v51: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 35 MiB/s wr, 1.68k op/s 2024-05-24T11:39:51.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:51 smithi039 ceph-mon[42919]: pgmap v52: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-05-24T11:39:51.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:51 smithi053 ceph-mon[47448]: pgmap v52: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-05-24T11:39:51.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:51 smithi137 ceph-mon[47907]: pgmap v52: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-05-24T11:39:53.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:53 smithi039 ceph-mon[42919]: pgmap v53: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-05-24T11:39:53.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:53 smithi053 ceph-mon[47448]: pgmap v53: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-05-24T11:39:53.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:53 smithi137 ceph-mon[47907]: pgmap v53: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-05-24T11:39:54.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 30.0 s, 1193.2 tps, lat 83.736 ms stddev 99.256 2024-05-24T11:39:55.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:55 smithi053 ceph-mon[47448]: pgmap v54: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 33 MiB/s wr, 1.83k op/s 2024-05-24T11:39:55.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:55 smithi137 ceph-mon[47907]: pgmap v54: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 33 MiB/s wr, 1.83k op/s 2024-05-24T11:39:55.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:55 smithi039 ceph-mon[42919]: pgmap v54: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 33 MiB/s wr, 1.83k op/s 2024-05-24T11:39:57.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:57 smithi053 ceph-mon[47448]: pgmap v55: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 21 MiB/s wr, 1.20k op/s 2024-05-24T11:39:57.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:57 smithi137 ceph-mon[47907]: pgmap v55: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 21 MiB/s wr, 1.20k op/s 2024-05-24T11:39:57.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:57 smithi039 ceph-mon[42919]: pgmap v55: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 21 MiB/s wr, 1.20k op/s 2024-05-24T11:39:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 35.0 s, 1234.6 tps, lat 81.133 ms stddev 101.331 2024-05-24T11:39:59.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:39:59 smithi053 ceph-mon[47448]: pgmap v56: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 31 MiB/s wr, 1.88k op/s 2024-05-24T11:39:59.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:39:59 smithi137 ceph-mon[47907]: pgmap v56: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 31 MiB/s wr, 1.88k op/s 2024-05-24T11:39:59.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:39:59 smithi039 ceph-mon[42919]: pgmap v56: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 31 MiB/s wr, 1.88k op/s 2024-05-24T11:40:00.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:00 smithi053 ceph-mon[47448]: overall HEALTH_OK 2024-05-24T11:40:00.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:00 smithi137 ceph-mon[47907]: overall HEALTH_OK 2024-05-24T11:40:00.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:00 smithi039 ceph-mon[42919]: overall HEALTH_OK 2024-05-24T11:40:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.047+0000 7f223133f700 1 -- 172.21.15.39:0/2348815840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100a40 msgr2=0x7f222c10cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.047+0000 7f223133f700 1 --2- 172.21.15.39:0/2348815840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100a40 0x7f222c10cbe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f222400b0d0 tx=0x7f222400b3e0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.048+0000 7f223133f700 1 -- 172.21.15.39:0/2348815840 shutdown_connections 2024-05-24T11:40:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.048+0000 7f223133f700 1 --2- 172.21.15.39:0/2348815840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100a40 0x7f222c10cbe0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.048+0000 7f223133f700 1 --2- 172.21.15.39:0/2348815840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c104de0 0x7f222c1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.048+0000 7f223133f700 1 --2- 172.21.15.39:0/2348815840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c100080 0x7f222c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.048+0000 7f223133f700 1 -- 172.21.15.39:0/2348815840 >> 172.21.15.39:0/2348815840 conn(0x7f222c076c10 msgr2=0x7f222c077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.048+0000 7f223133f700 1 -- 172.21.15.39:0/2348815840 shutdown_connections 2024-05-24T11:40:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.048+0000 7f223133f700 1 -- 172.21.15.39:0/2348815840 wait complete. 2024-05-24T11:40:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.048+0000 7f223133f700 1 Processor -- start 2024-05-24T11:40:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.048+0000 7f223133f700 1 -- start start 2024-05-24T11:40:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f223133f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100080 0x7f222c10d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f223133f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c100a40 0x7f222c10db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f223133f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c104de0 0x7f222c10e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f223133f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c1a5700 con 0x7f222c100080 2024-05-24T11:40:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f223133f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c1a5870 con 0x7f222c104de0 2024-05-24T11:40:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f223133f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c1a5a30 con 0x7f222c100a40 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100080 0x7f222c10d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100080 0x7f222c10d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44346/0 (socket says 172.21.15.39:44346) 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222affd700 1 -- 172.21.15.39:0/485154749 learned_addr learned my addr 172.21.15.39:0/485154749 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222b7fe700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c104de0 0x7f222c10e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222a7fc700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c100a40 0x7f222c10db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222affd700 1 -- 172.21.15.39:0/485154749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c100a40 msgr2=0x7f222c10db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222affd700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c100a40 0x7f222c10db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222affd700 1 -- 172.21.15.39:0/485154749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c104de0 msgr2=0x7f222c10e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222affd700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c104de0 0x7f222c10e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222affd700 1 -- 172.21.15.39:0/485154749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224009d00 con 0x7f222c100080 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.049+0000 7f222b7fe700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c104de0 0x7f222c10e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.050+0000 7f222affd700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100080 0x7f222c10d630 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f221c00e9a0 tx=0x7f221c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:01.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.050+0000 7f2213fff700 1 -- 172.21.15.39:0/485154749 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c009790 con 0x7f222c100080 2024-05-24T11:40:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.050+0000 7f223133f700 1 -- 172.21.15.39:0/485154749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c1a5d10 con 0x7f222c100080 2024-05-24T11:40:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.050+0000 7f2213fff700 1 -- 172.21.15.39:0/485154749 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221c010430 con 0x7f222c100080 2024-05-24T11:40:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.050+0000 7f2213fff700 1 -- 172.21.15.39:0/485154749 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c00cea0 con 0x7f222c100080 2024-05-24T11:40:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.050+0000 7f223133f700 1 -- 172.21.15.39:0/485154749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c1a61c0 con 0x7f222c100080 2024-05-24T11:40:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.052+0000 7f2213fff700 1 -- 172.21.15.39:0/485154749 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f221c0105a0 con 0x7f222c100080 2024-05-24T11:40:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.053+0000 7f2213fff700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f22140a1290 0x7f22140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.053+0000 7f2213fff700 1 -- 172.21.15.39:0/485154749 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f221c014070 con 0x7f222c100080 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.053+0000 7f222a7fc700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f22140a1290 0x7f22140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.053+0000 7f223d4fab80 1 Processor -- start 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.053+0000 7f223d4fab80 1 -- start start 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 0x55a5be15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f22140a1290 0x7f22140a3750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f222c10ed60 tx=0x7f22200073d0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be155f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f223d4fab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be0ad0f0 con 0x55a5be15b950 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f223d4fab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be0aaab0 con 0x55a5be156480 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f223d4fab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be0a63a0 con 0x55a5be15c270 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:54114/0 (socket says 172.21.15.39:54114) 2024-05-24T11:40:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 -- 172.21.15.39:0/1460858856 learned_addr learned my addr 172.21.15.39:0/1460858856 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222b7fe700 1 --2- 172.21.15.39:0/1460858856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 -- 172.21.15.39:0/1460858856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 msgr2=0x55a5be1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 --2- 172.21.15.39:0/1460858856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be1568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 -- 172.21.15.39:0/1460858856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 msgr2=0x55a5be15bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 --2- 172.21.15.39:0/1460858856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 0x55a5be15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 -- 172.21.15.39:0/1460858856 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5bdf27350 con 0x55a5be15c270 2024-05-24T11:40:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222b7fe700 1 --2- 172.21.15.39:0/1460858856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:40:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.054+0000 7f222a7fc700 1 --2- 172.21.15.39:0/1460858856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be155f40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2220010e60 tx=0x7f2220013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c46eb0356c40e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f22117fa700 1 -- 172.21.15.39:0/1460858856 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220004220 con 0x55a5be15c270 2024-05-24T11:40:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f22117fa700 1 -- 172.21.15.39:0/1460858856 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2220015070 con 0x55a5be15c270 2024-05-24T11:40:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f22117fa700 1 -- 172.21.15.39:0/1460858856 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222001e680 con 0x55a5be15c270 2024-05-24T11:40:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f223d4fab80 1 -- 172.21.15.39:0/1460858856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 msgr2=0x55a5be155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f223d4fab80 1 --2- 172.21.15.39:0/1460858856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be155f40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2220010e60 tx=0x7f2220013000 comp rx=0 tx=0).stop 2024-05-24T11:40:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f223d4fab80 1 -- 172.21.15.39:0/1460858856 shutdown_connections 2024-05-24T11:40:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f223d4fab80 1 --2- 172.21.15.39:0/1460858856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f223d4fab80 1 --2- 172.21.15.39:0/1460858856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 0x55a5be15bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f223d4fab80 1 --2- 172.21.15.39:0/1460858856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be155f40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f223d4fab80 1 -- 172.21.15.39:0/1460858856 >> 172.21.15.39:0/1460858856 conn(0x55a5be0b41b0 msgr2=0x55a5be150840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.055+0000 7f223d4fab80 1 -- 172.21.15.39:0/1460858856 shutdown_connections 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 1 -- 172.21.15.39:0/1460858856 wait complete. 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 1 -- start start 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 1 Processor -- start 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15b950 0x55a5be15f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15c270 0x55a5be15fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be042500 con 0x55a5be15c270 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be0421a0 con 0x55a5be156480 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bdf29640 con 0x55a5be15b950 2024-05-24T11:40:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f223d4fab80 10 client.0 _ll_register_callbacks cb 0x55a5be04bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f222b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15c270 0x55a5be15fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f222b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15c270 0x55a5be15fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44350/0 (socket says 172.21.15.39:44350) 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f222b7fe700 1 -- 172.21.15.39:0/2797165491 learned_addr learned my addr 172.21.15.39:0/2797165491 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.056+0000 7f222a7fc700 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15b950 0x55a5be15f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f222b7fe700 1 -- 172.21.15.39:0/2797165491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15b950 msgr2=0x55a5be15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f222b7fe700 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15b950 0x55a5be15f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f222b7fe700 1 -- 172.21.15.39:0/2797165491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 msgr2=0x55a5be162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f222affd700 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f222b7fe700 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f222b7fe700 1 -- 172.21.15.39:0/2797165491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2220010b40 con 0x55a5be15c270 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f222a7fc700 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15b950 0x55a5be15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f222affd700 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:40:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f222b7fe700 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15c270 0x55a5be15fcc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2224009610 tx=0x7f222400bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6cf4d678cbfc18 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f220eff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f220eff5700 1 -- 172.21.15.39:0/2797165491 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222400e030 con 0x55a5be15c270 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f220eff5700 1 -- 172.21.15.39:0/2797165491 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2224003a00 con 0x55a5be15c270 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f223d4fab80 1 -- 172.21.15.39:0/2797165491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5bdfff210 con 0x55a5be15c270 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.057+0000 7f220eff5700 1 -- 172.21.15.39:0/2797165491 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2224007ea0 con 0x55a5be15c270 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f223d4fab80 10 client.15579 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f223d4fab80 1 -- 172.21.15.39:0/2797165491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a5bded3b00 con 0x55a5be15c270 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f220eff5700 1 -- 172.21.15.39:0/2797165491 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a5bdfff210 con 0x55a5be15c270 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f220eff5700 1 -- 172.21.15.39:0/2797165491 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f222400b100 con 0x55a5be15c270 2024-05-24T11:40:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f223d4fab80 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f223d4fab80 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f223d4fab80 20 client.15579 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f223d4fab80 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a5be16fbc0 0x55a5be171fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f223d4fab80 4 client.15579 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:40:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f223d4fab80 1 -- 172.21.15.39:0/2797165491 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5be1257b0 con 0x55a5be16fbc0 2024-05-24T11:40:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.058+0000 7f222affd700 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a5be16fbc0 0x55a5be171fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.059+0000 7f222affd700 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a5be16fbc0 0x55a5be171fe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a682888aeda91d0 server_cookie=f0450affc899f698 in_seq=0 out_seq=0 2024-05-24T11:40:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.059+0000 7f220eff5700 10 client.15579 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.059+0000 7f220eff5700 1 -- 172.21.15.39:0/2797165491 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5be1257b0 con 0x55a5be16fbc0 2024-05-24T11:40:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.059+0000 7f220eff5700 10 client.15579 handle_command_reply: tid=0 2024-05-24T11:40:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 client.15579 shutdown 2024-05-24T11:40:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 -- 172.21.15.39:0/2797165491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15c270 msgr2=0x55a5be15fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15c270 0x55a5be15fcc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2224009610 tx=0x7f222400bcd0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 -- 172.21.15.39:0/2797165491 shutdown_connections 2024-05-24T11:40:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a5be16fbc0 0x55a5be171fe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15c270 0x55a5be15fcc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be156480 0x55a5be162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 --2- 172.21.15.39:0/2797165491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15b950 0x55a5be15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f220eff5700 0 client.15579 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 -- 172.21.15.39:0/2797165491 >> 172.21.15.39:0/2797165491 conn(0x55a5be0b41b0 msgr2=0x55a5be159460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 -- 172.21.15.39:0/2797165491 shutdown_connections 2024-05-24T11:40:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 1 -- 172.21.15.39:0/2797165491 wait complete. 2024-05-24T11:40:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.060+0000 7f223d4fab80 20 client.15579 trim_cache size 0 max 16384 2024-05-24T11:40:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.078+0000 7f223d4fab80 1 Processor -- start 2024-05-24T11:40:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.078+0000 7f223d4fab80 1 -- start start 2024-05-24T11:40:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.078+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be156480 0x55a5be150910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.078+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15b950 0x55a5be159b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.078+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.078+0000 7f223d4fab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be15a040 con 0x55a5be156480 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.078+0000 7f223d4fab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be1551c0 con 0x55a5be15b950 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.078+0000 7f223d4fab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be151510 con 0x55a5be15c270 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be156480 0x55a5be150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be156480 0x55a5be150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44362/0 (socket says 172.21.15.39:44362) 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222affd700 1 -- 172.21.15.39:0/1703099164 learned_addr learned my addr 172.21.15.39:0/1703099164 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222a7fc700 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15b950 0x55a5be159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222b7fe700 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222affd700 1 -- 172.21.15.39:0/1703099164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 msgr2=0x55a5be161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222affd700 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be161dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222affd700 1 -- 172.21.15.39:0/1703099164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15b950 msgr2=0x55a5be159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222affd700 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15b950 0x55a5be159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222affd700 1 -- 172.21.15.39:0/1703099164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5be06c500 con 0x55a5be156480 2024-05-24T11:40:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222a7fc700 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15b950 0x55a5be159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:40:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f222affd700 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be156480 0x55a5be150910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c2d0 tx=0x7f221c00c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebeb195d7ff3f1a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f2211ffb700 1 -- 172.21.15.39:0/1703099164 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0b1070 con 0x55a5be156480 2024-05-24T11:40:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f2211ffb700 1 -- 172.21.15.39:0/1703099164 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221c09a690 con 0x55a5be156480 2024-05-24T11:40:01.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.079+0000 7f2211ffb700 1 -- 172.21.15.39:0/1703099164 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0982d0 con 0x55a5be156480 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.080+0000 7f223d4fab80 1 -- 172.21.15.39:0/1703099164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be156480 msgr2=0x55a5be150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.080+0000 7f223d4fab80 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be156480 0x55a5be150910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c2d0 tx=0x7f221c00c480 comp rx=0 tx=0).stop 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.080+0000 7f223d4fab80 1 -- 172.21.15.39:0/1703099164 shutdown_connections 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.080+0000 7f223d4fab80 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be15c270 0x55a5be161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.080+0000 7f223d4fab80 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be156480 0x55a5be150910 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.080+0000 7f223d4fab80 1 --2- 172.21.15.39:0/1703099164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15b950 0x55a5be159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.080+0000 7f223d4fab80 1 -- 172.21.15.39:0/1703099164 >> 172.21.15.39:0/1703099164 conn(0x55a5be0b41b0 msgr2=0x55a5be1593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.080+0000 7f223d4fab80 1 -- 172.21.15.39:0/1703099164 shutdown_connections 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.080+0000 7f223d4fab80 1 -- 172.21.15.39:0/1703099164 wait complete. 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f223d4fab80 1 -- start start 2024-05-24T11:40:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f223d4fab80 1 Processor -- start 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be156480 0x55a5be166c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 0x55a5be167180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f223d4fab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15c270 0x55a5be15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f223d4fab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be15e400 con 0x55a5be15b950 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f223d4fab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be15e570 con 0x55a5be15c270 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f223d4fab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5be15e730 con 0x55a5be156480 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f223d4fab80 10 client.0 _ll_register_callbacks cb 0x55a5be04bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 0x55a5be167180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 0x55a5be167180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44376/0 (socket says 172.21.15.39:44376) 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222a7fc700 1 -- 172.21.15.39:0/595917633 learned_addr learned my addr 172.21.15.39:0/595917633 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222affd700 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be156480 0x55a5be166c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222b7fe700 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15c270 0x55a5be15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222a7fc700 1 -- 172.21.15.39:0/595917633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be156480 msgr2=0x55a5be166c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222a7fc700 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be156480 0x55a5be166c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222a7fc700 1 -- 172.21.15.39:0/595917633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15c270 msgr2=0x55a5be15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222a7fc700 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15c270 0x55a5be15dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222a7fc700 1 -- 172.21.15.39:0/595917633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c1a61c0 con 0x55a5be15b950 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222b7fe700 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15c270 0x55a5be15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f222a7fc700 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 0x55a5be167180 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2220010e30 tx=0x7f2220004fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e06c552c5227f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.081+0000 7f220eff5700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f220eff5700 1 -- 172.21.15.39:0/595917633 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220010e90 con 0x55a5be15b950 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f220eff5700 1 -- 172.21.15.39:0/595917633 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2220017070 con 0x55a5be15b950 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f223d4fab80 1 -- 172.21.15.39:0/595917633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5be09ee60 con 0x55a5be15b950 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f220eff5700 1 -- 172.21.15.39:0/595917633 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2220005410 con 0x55a5be15b950 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f223d4fab80 10 client.15585 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f223d4fab80 1 -- 172.21.15.39:0/595917633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a5bdf2bc70 con 0x55a5be15b950 2024-05-24T11:40:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f220eff5700 1 -- 172.21.15.39:0/595917633 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2220042020 con 0x55a5be15b950 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f220eff5700 1 -- 172.21.15.39:0/595917633 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2220024070 con 0x55a5be15b950 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f223d4fab80 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f223d4fab80 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f223d4fab80 20 client.15585 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f223d4fab80 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a5be1c6f70 0x55a5be1c9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f223d4fab80 4 client.15585 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.082+0000 7f223d4fab80 1 -- 172.21.15.39:0/595917633 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a5bdef4c90 con 0x55a5be1c6f70 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.083+0000 7f222affd700 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a5be1c6f70 0x55a5be1c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.083+0000 7f220eff5700 10 client.15585 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.083+0000 7f222affd700 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a5be1c6f70 0x55a5be1c9390 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2344332717e465bf server_cookie=346919176e39005a in_seq=0 out_seq=0 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.120+0000 7f220eff5700 1 -- 172.21.15.39:0/595917633 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a5bdef4c90 con 0x55a5be1c6f70 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.120+0000 7f220eff5700 10 client.15585 handle_command_reply: tid=0 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.120+0000 7f223d4fab80 1 client.15585 shutdown 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 -- 172.21.15.39:0/595917633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 msgr2=0x55a5be167180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 0x55a5be167180 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2220010e30 tx=0x7f2220004fe0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 -- 172.21.15.39:0/595917633 shutdown_connections 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5be156480 0x55a5be166c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a5be1c6f70 0x55a5be1c9390 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a5be15c270 0x55a5be15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f220eff5700 0 client.15585 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 --2- 172.21.15.39:0/595917633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5be15b950 0x55a5be167180 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 -- 172.21.15.39:0/595917633 >> 172.21.15.39:0/595917633 conn(0x55a5be0b41b0 msgr2=0x55a5be1555d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 -- 172.21.15.39:0/595917633 shutdown_connections 2024-05-24T11:40:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 1 -- 172.21.15.39:0/595917633 wait complete. 2024-05-24T11:40:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.121+0000 7f223d4fab80 20 client.15585 trim_cache size 0 max 16384 2024-05-24T11:40:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:40:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:40:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:40:01.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:40:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 -- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f22140a1290 msgr2=0x7f22140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f22140a1290 0x7f22140a3750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f222c10ed60 tx=0x7f22200073d0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 -- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100080 msgr2=0x7f222c10d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100080 0x7f222c10d630 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f221c00e9a0 tx=0x7f221c00ecb0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 -- 172.21.15.39:0/485154749 shutdown_connections 2024-05-24T11:40:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f22140a1290 0x7f22140a3750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f222c104de0 0x7f222c10e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f222c100080 0x7f222c10d630 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 --2- 172.21.15.39:0/485154749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c100a40 0x7f222c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.122+0000 7f223133f700 1 -- 172.21.15.39:0/485154749 >> 172.21.15.39:0/485154749 conn(0x7f222c076c10 msgr2=0x7f222c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.123+0000 7f223133f700 1 -- 172.21.15.39:0/485154749 shutdown_connections 2024-05-24T11:40:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:01.123+0000 7f223133f700 1 -- 172.21.15.39:0/485154749 wait complete. 2024-05-24T11:40:01.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:01 smithi053 ceph-mon[47448]: pgmap v57: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-05-24T11:40:01.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:01 smithi137 ceph-mon[47907]: pgmap v57: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-05-24T11:40:01.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:01 smithi039 ceph-mon[42919]: pgmap v57: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-05-24T11:40:03.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:03 smithi053 ceph-mon[47448]: pgmap v58: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-05-24T11:40:03.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:03 smithi137 ceph-mon[47907]: pgmap v58: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-05-24T11:40:03.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:03 smithi039 ceph-mon[42919]: pgmap v58: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-05-24T11:40:04.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 40.0 s, 1207.6 tps, lat 81.942 ms stddev 80.188 2024-05-24T11:40:05.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:05 smithi053 ceph-mon[47448]: pgmap v59: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 29 MiB/s wr, 1.93k op/s 2024-05-24T11:40:05.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:05 smithi137 ceph-mon[47907]: pgmap v59: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 29 MiB/s wr, 1.93k op/s 2024-05-24T11:40:05.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:05 smithi039 ceph-mon[42919]: pgmap v59: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 29 MiB/s wr, 1.93k op/s 2024-05-24T11:40:07.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:07 smithi053 ceph-mon[47448]: pgmap v60: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 19 MiB/s wr, 1.30k op/s 2024-05-24T11:40:07.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:07 smithi137 ceph-mon[47907]: pgmap v60: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 19 MiB/s wr, 1.30k op/s 2024-05-24T11:40:07.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:07 smithi039 ceph-mon[42919]: pgmap v60: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 19 MiB/s wr, 1.30k op/s 2024-05-24T11:40:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 45.0 s, 1099.4 tps, lat 91.887 ms stddev 380.142 2024-05-24T11:40:09.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:09 smithi053 ceph-mon[47448]: pgmap v61: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 28 MiB/s wr, 2.01k op/s 2024-05-24T11:40:09.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:09 smithi137 ceph-mon[47907]: pgmap v61: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 28 MiB/s wr, 2.01k op/s 2024-05-24T11:40:09.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:09 smithi039 ceph-mon[42919]: pgmap v61: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 28 MiB/s wr, 2.01k op/s 2024-05-24T11:40:11.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:11 smithi053 ceph-mon[47448]: pgmap v62: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-24T11:40:11.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:11 smithi137 ceph-mon[47907]: pgmap v62: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-24T11:40:11.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:11 smithi039 ceph-mon[42919]: pgmap v62: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-24T11:40:13.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:13 smithi053 ceph-mon[47448]: pgmap v63: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-24T11:40:13.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:13 smithi137 ceph-mon[47907]: pgmap v63: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-24T11:40:13.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:13 smithi039 ceph-mon[42919]: pgmap v63: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-24T11:40:14.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 50.0 s, 1313.6 tps, lat 76.076 ms stddev 71.631 2024-05-24T11:40:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.300+0000 7f919e512700 1 -- 172.21.15.39:0/3909910721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9198105e10 msgr2=0x7f91981061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.300+0000 7f919e512700 1 --2- 172.21.15.39:0/3909910721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9198105e10 0x7f91981061f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9188009990 tx=0x7f9188009ca0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.300+0000 7f919e512700 1 -- 172.21.15.39:0/3909910721 shutdown_connections 2024-05-24T11:40:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.300+0000 7f919e512700 1 --2- 172.21.15.39:0/3909910721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198069c30 0x7f91981039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.300+0000 7f919e512700 1 --2- 172.21.15.39:0/3909910721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9198105e10 0x7f91981061f0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9188009990 tx=0x7f9188009ca0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.300+0000 7f919e512700 1 --2- 172.21.15.39:0/3909910721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9198069190 0x7f9198069610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.300+0000 7f919e512700 1 -- 172.21.15.39:0/3909910721 >> 172.21.15.39:0/3909910721 conn(0x7f9198076c10 msgr2=0x7f9198077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.300+0000 7f919e512700 1 -- 172.21.15.39:0/3909910721 shutdown_connections 2024-05-24T11:40:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.300+0000 7f919e512700 1 -- 172.21.15.39:0/3909910721 wait complete. 2024-05-24T11:40:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.301+0000 7f919e512700 1 Processor -- start 2024-05-24T11:40:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.301+0000 7f919e512700 1 -- start start 2024-05-24T11:40:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f919e512700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198069190 0x7f91981a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f919e512700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9198069c30 0x7f91981a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f919e512700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f919819da80 0x7f919819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f919e512700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919819e500 con 0x7f9198069c30 2024-05-24T11:40:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f919e512700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919819e640 con 0x7f919819da80 2024-05-24T11:40:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f919e512700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919819e800 con 0x7f9198069190 2024-05-24T11:40:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f91977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9198069c30 0x7f91981a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f91977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9198069c30 0x7f91981a3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52736/0 (socket says 172.21.15.39:52736) 2024-05-24T11:40:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f91977fe700 1 -- 172.21.15.39:0/1730404564 learned_addr learned my addr 172.21.15.39:0/1730404564 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.302+0000 7f9197fff700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198069190 0x7f91981a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.303+0000 7f91977fe700 1 -- 172.21.15.39:0/1730404564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198069190 msgr2=0x7f91981a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.303+0000 7f91977fe700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198069190 0x7f91981a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.303+0000 7f91977fe700 1 -- 172.21.15.39:0/1730404564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f919819da80 msgr2=0x7f919819df30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-24T11:40:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.303+0000 7f91977fe700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f919819da80 0x7f919819df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.303+0000 7f91977fe700 1 -- 172.21.15.39:0/1730404564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880095f0 con 0x7f9198069c30 2024-05-24T11:40:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.303+0000 7f91977fe700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9198069c30 0x7f91981a3e90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b550 tx=0x7f918c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.303+0000 7f91957fa700 1 -- 172.21.15.39:0/1730404564 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c011830 con 0x7f9198069c30 2024-05-24T11:40:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.304+0000 7f91957fa700 1 -- 172.21.15.39:0/1730404564 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f918c011e20 con 0x7f9198069c30 2024-05-24T11:40:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.304+0000 7f91957fa700 1 -- 172.21.15.39:0/1730404564 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c010e90 con 0x7f9198069c30 2024-05-24T11:40:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.304+0000 7f919e512700 1 -- 172.21.15.39:0/1730404564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919806aa20 con 0x7f9198069c30 2024-05-24T11:40:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.305+0000 7f919e512700 1 -- 172.21.15.39:0/1730404564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919806aee0 con 0x7f9198069c30 2024-05-24T11:40:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.309+0000 7f91aa6cdb80 1 Processor -- start 2024-05-24T11:40:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.309+0000 7f91aa6cdb80 1 -- start start 2024-05-24T11:40:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf940950 0x5619bf940d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf93af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf93b480 0x5619bf93b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf8920f0 con 0x5619bf941270 2024-05-24T11:40:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf88fab0 con 0x5619bf93b480 2024-05-24T11:40:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf88b3a0 con 0x5619bf940950 2024-05-24T11:40:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f919caaf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf940950 0x5619bf940d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f919caaf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf940950 0x5619bf940d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47940/0 (socket says 172.21.15.39:47940) 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f919caaf700 1 -- 172.21.15.39:0/3520880501 learned_addr learned my addr 172.21.15.39:0/3520880501 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f91957fa700 1 -- 172.21.15.39:0/1730404564 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f918c011990 con 0x7f9198069c30 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f919caaf700 1 -- 172.21.15.39:0/3520880501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf93b480 msgr2=0x5619bf93b8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f919caaf700 1 --2- 172.21.15.39:0/3520880501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf93b480 0x5619bf93b8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f919caaf700 1 -- 172.21.15.39:0/3520880501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 msgr2=0x5619bf93af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f919caaf700 1 --2- 172.21.15.39:0/3520880501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf93af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f919caaf700 1 -- 172.21.15.39:0/3520880501 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619bf70c350 con 0x5619bf940950 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f919caaf700 1 --2- 172.21.15.39:0/3520880501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf940950 0x5619bf940d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9190009d00 tx=0x7f919000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49d041396794996f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.310+0000 7f917e7fc700 1 -- 172.21.15.39:0/3520880501 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000e070 con 0x5619bf940950 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.311+0000 7f917e7fc700 1 -- 172.21.15.39:0/3520880501 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9190009e50 con 0x5619bf940950 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.311+0000 7f917e7fc700 1 -- 172.21.15.39:0/3520880501 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190011390 con 0x5619bf940950 2024-05-24T11:40:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.311+0000 7f91957fa700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f91800a1290 0x7f91800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.311+0000 7f91957fa700 1 -- 172.21.15.39:0/1730404564 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f918c0cc3e0 con 0x7f9198069c30 2024-05-24T11:40:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.311+0000 7f9197fff700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f91800a1290 0x7f91800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.311+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/3520880501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf940950 msgr2=0x5619bf940d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.311+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/3520880501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf940950 0x5619bf940d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9190009d00 tx=0x7f919000b330 comp rx=0 tx=0).stop 2024-05-24T11:40:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.312+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/3520880501 shutdown_connections 2024-05-24T11:40:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.312+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/3520880501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf940950 0x5619bf940d30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.312+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/3520880501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf93b480 0x5619bf93b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.312+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/3520880501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf93af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.312+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/3520880501 >> 172.21.15.39:0/3520880501 conn(0x5619bf8991b0 msgr2=0x5619bf935840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.312+0000 7f9197fff700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f91800a1290 0x7f91800a3750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9188003760 tx=0x7f9188005220 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.313+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/3520880501 shutdown_connections 2024-05-24T11:40:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.313+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/3520880501 wait complete. 2024-05-24T11:40:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.313+0000 7f91aa6cdb80 1 -- start start 2024-05-24T11:40:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.313+0000 7f91aa6cdb80 1 Processor -- start 2024-05-24T11:40:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 0x5619bf947160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf944450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf944a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf827500 con 0x5619bf941270 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf8271a0 con 0x5619bf940950 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf70e640 con 0x5619bf93b480 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f91aa6cdb80 10 client.0 _ll_register_callbacks cb 0x5619bf830ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f91977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf944450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f919caaf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf944a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f919caaf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf944a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52756/0 (socket says 172.21.15.39:52756) 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f91977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf944450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:41722/0 (socket says 172.21.15.39:41722) 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f919caaf700 1 -- 172.21.15.39:0/742801120 learned_addr learned my addr 172.21.15.39:0/742801120 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.314+0000 7f9197fff700 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 0x5619bf947160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.315+0000 7f919caaf700 1 -- 172.21.15.39:0/742801120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 msgr2=0x5619bf947160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.315+0000 7f919caaf700 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 0x5619bf947160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.315+0000 7f919caaf700 1 -- 172.21.15.39:0/742801120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 msgr2=0x5619bf944450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.315+0000 7f919caaf700 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf944450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.315+0000 7f919caaf700 1 -- 172.21.15.39:0/742801120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91900099b0 con 0x5619bf941270 2024-05-24T11:40:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.315+0000 7f919caaf700 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf944a80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f91900127b0 tx=0x7f91900127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec44bed27988a4d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.315+0000 7f91777fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:40:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.316+0000 7f91777fe700 1 -- 172.21.15.39:0/742801120 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000e040 con 0x5619bf941270 2024-05-24T11:40:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.316+0000 7f91777fe700 1 -- 172.21.15.39:0/742801120 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f919001b830 con 0x5619bf941270 2024-05-24T11:40:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.316+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/742801120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619bf7e4210 con 0x5619bf941270 2024-05-24T11:40:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.316+0000 7f91777fe700 1 -- 172.21.15.39:0/742801120 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f919001b9f0 con 0x5619bf941270 2024-05-24T11:40:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91aa6cdb80 10 client.15591 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/742801120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5619bf6b8b00 con 0x5619bf941270 2024-05-24T11:40:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91777fe700 1 -- 172.21.15.39:0/742801120 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5619bf7e4210 con 0x5619bf941270 2024-05-24T11:40:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91777fe700 1 -- 172.21.15.39:0/742801120 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9190027d20 con 0x5619bf941270 2024-05-24T11:40:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91aa6cdb80 10 client.15591 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91aa6cdb80 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91aa6cdb80 20 client.15591 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5619bf954af0 0x5619bf956ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91aa6cdb80 4 client.15591 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:40:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.317+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/742801120 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619bf90a7b0 con 0x5619bf954af0 2024-05-24T11:40:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.318+0000 7f9197fff700 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5619bf954af0 0x5619bf956ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.319+0000 7f91777fe700 10 client.15591 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.319+0000 7f9197fff700 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5619bf954af0 0x5619bf956ee0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf49a0f3f8ce27f server_cookie=d21043988b4dfb8b in_seq=0 out_seq=0 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.319+0000 7f91777fe700 1 -- 172.21.15.39:0/742801120 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619bf90a7b0 con 0x5619bf954af0 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.319+0000 7f91777fe700 10 client.15591 handle_command_reply: tid=0 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 client.15591 shutdown 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/742801120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 msgr2=0x5619bf944a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf944a80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f91900127b0 tx=0x7f91900127e0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/742801120 shutdown_connections 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 0x5619bf947160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5619bf954af0 0x5619bf956ee0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf944a80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/742801120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf944450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/742801120 >> 172.21.15.39:0/742801120 conn(0x5619bf8991b0 msgr2=0x5619bf93f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91777fe700 0 client.15591 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.320+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/742801120 shutdown_connections 2024-05-24T11:40:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.321+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/742801120 wait complete. 2024-05-24T11:40:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.321+0000 7f91aa6cdb80 20 client.15591 trim_cache size 0 max 16384 2024-05-24T11:40:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.341+0000 7f91aa6cdb80 1 Processor -- start 2024-05-24T11:40:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.341+0000 7f91aa6cdb80 1 -- start start 2024-05-24T11:40:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 0x5619bf942680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf9449c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf935910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf9405f0 con 0x5619bf941270 2024-05-24T11:40:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf937a80 con 0x5619bf940950 2024-05-24T11:40:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf9307c0 con 0x5619bf93b480 2024-05-24T11:40:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf9449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf9449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:41736/0 (socket says 172.21.15.39:41736) 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91977fe700 1 -- 172.21.15.39:0/1678392374 learned_addr learned my addr 172.21.15.39:0/1678392374 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f9197fff700 1 --2- 172.21.15.39:0/1678392374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 0x5619bf942680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f919caaf700 1 --2- 172.21.15.39:0/1678392374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf935910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91977fe700 1 -- 172.21.15.39:0/1678392374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 msgr2=0x5619bf942680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91977fe700 1 --2- 172.21.15.39:0/1678392374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 0x5619bf942680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91977fe700 1 -- 172.21.15.39:0/1678392374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 msgr2=0x5619bf935910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91977fe700 1 --2- 172.21.15.39:0/1678392374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf935910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.342+0000 7f91977fe700 1 -- 172.21.15.39:0/1678392374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619bf851500 con 0x5619bf940950 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.343+0000 7f91977fe700 1 --2- 172.21.15.39:0/1678392374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf9449c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f918c010900 tx=0x7f918c0b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afedc55886ba16bb server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.343+0000 7f917effd700 1 -- 172.21.15.39:0/1678392374 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c010380 con 0x5619bf940950 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.343+0000 7f917effd700 1 -- 172.21.15.39:0/1678392374 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f918c099d40 con 0x5619bf940950 2024-05-24T11:40:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.343+0000 7f917effd700 1 -- 172.21.15.39:0/1678392374 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c09d170 con 0x5619bf940950 2024-05-24T11:40:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.344+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/1678392374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 msgr2=0x5619bf9449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.344+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/1678392374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf9449c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f918c010900 tx=0x7f918c0b9040 comp rx=0 tx=0).stop 2024-05-24T11:40:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.346+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/1678392374 shutdown_connections 2024-05-24T11:40:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.346+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/1678392374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf93b480 0x5619bf942680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.346+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/1678392374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf941270 0x5619bf935910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.346+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/1678392374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf9449c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.346+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/1678392374 >> 172.21.15.39:0/1678392374 conn(0x5619bf8991b0 msgr2=0x5619bf942d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.346+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/1678392374 shutdown_connections 2024-05-24T11:40:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.346+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/1678392374 wait complete. 2024-05-24T11:40:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.347+0000 7f91aa6cdb80 1 -- start start 2024-05-24T11:40:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.347+0000 7f91aa6cdb80 1 Processor -- start 2024-05-24T11:40:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.347+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf93b480 0x5619bf94b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.347+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf94bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.347+0000 7f91aa6cdb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf941270 0x5619bf949d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.347+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf94a410 con 0x5619bf93b480 2024-05-24T11:40:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.347+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf94a580 con 0x5619bf940950 2024-05-24T11:40:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.347+0000 7f91aa6cdb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bf94a740 con 0x5619bf941270 2024-05-24T11:40:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.347+0000 7f91aa6cdb80 10 client.0 _ll_register_callbacks cb 0x5619bf830ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf94bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf94bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:41744/0 (socket says 172.21.15.39:41744) 2024-05-24T11:40:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91977fe700 1 -- 172.21.15.39:0/4278484888 learned_addr learned my addr 172.21.15.39:0/4278484888 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:14.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f919caaf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf941270 0x5619bf949d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91977fe700 1 -- 172.21.15.39:0/4278484888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf941270 msgr2=0x5619bf949d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91977fe700 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf941270 0x5619bf949d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91977fe700 1 -- 172.21.15.39:0/4278484888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf93b480 msgr2=0x5619bf94b860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:40:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91977fe700 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf93b480 0x5619bf94b860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91977fe700 1 -- 172.21.15.39:0/4278484888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919806aee0 con 0x5619bf940950 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f919caaf700 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf941270 0x5619bf949d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91977fe700 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf94bda0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f918c0ca170 tx=0x7f918c010850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10836449024caab1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91777fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.348+0000 7f91777fe700 1 -- 172.21.15.39:0/4278484888 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c0b9130 con 0x5619bf940950 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91777fe700 1 -- 172.21.15.39:0/4278484888 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f918c082e40 con 0x5619bf940950 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/4278484888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9190009e50 con 0x5619bf940950 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91777fe700 1 -- 172.21.15.39:0/4278484888 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f918c099640 con 0x5619bf940950 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91aa6cdb80 10 client.25283 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/4278484888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5619bf710c70 con 0x5619bf940950 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91777fe700 1 -- 172.21.15.39:0/4278484888 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9190009e50 con 0x5619bf940950 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91777fe700 1 -- 172.21.15.39:0/4278484888 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f918c0998a0 con 0x5619bf940950 2024-05-24T11:40:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91aa6cdb80 10 client.25283 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91aa6cdb80 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91aa6cdb80 20 client.25283 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.349+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5619bf9abe80 0x5619bf9ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.350+0000 7f91aa6cdb80 4 client.25283 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:40:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.350+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/4278484888 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619bf883e60 con 0x5619bf9abe80 2024-05-24T11:40:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.350+0000 7f9197fff700 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5619bf9abe80 0x5619bf9ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.351+0000 7f91777fe700 10 client.25283 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.351+0000 7f9197fff700 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5619bf9abe80 0x5619bf9ae2a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3479bf71782bf6a server_cookie=c187441d28f7de62 in_seq=0 out_seq=0 2024-05-24T11:40:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91777fe700 1 -- 172.21.15.39:0/4278484888 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619bf883e60 con 0x5619bf9abe80 2024-05-24T11:40:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91777fe700 10 client.25283 handle_command_reply: tid=0 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91aa6cdb80 1 client.25283 shutdown 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/4278484888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 msgr2=0x5619bf94bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf94bda0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f918c0ca170 tx=0x7f918c010850 comp rx=0 tx=0).stop 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/4278484888 shutdown_connections 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5619bf941270 0x5619bf949d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5619bf9abe80 0x5619bf9ae2a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619bf93b480 0x5619bf94b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91aa6cdb80 1 --2- 172.21.15.39:0/4278484888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5619bf940950 0x5619bf94bda0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91777fe700 0 client.25283 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.372+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/4278484888 >> 172.21.15.39:0/4278484888 conn(0x5619bf8991b0 msgr2=0x5619bf93ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.373+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/4278484888 shutdown_connections 2024-05-24T11:40:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.373+0000 7f91aa6cdb80 1 -- 172.21.15.39:0/4278484888 wait complete. 2024-05-24T11:40:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.373+0000 7f91aa6cdb80 20 client.25283 trim_cache size 0 max 16384 2024-05-24T11:40:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:40:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:40:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:40:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:40:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 -- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f91800a1290 msgr2=0x7f91800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f91800a1290 0x7f91800a3750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9188003760 tx=0x7f9188005220 comp rx=0 tx=0).stop 2024-05-24T11:40:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 -- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9198069c30 msgr2=0x7f91981a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9198069c30 0x7f91981a3e90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b550 tx=0x7f918c00b860 comp rx=0 tx=0).stop 2024-05-24T11:40:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 -- 172.21.15.39:0/1730404564 shutdown_connections 2024-05-24T11:40:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9198069190 0x7f91981a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f91800a1290 0x7f91800a3750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f919819da80 0x7f919819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 --2- 172.21.15.39:0/1730404564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9198069c30 0x7f91981a3e90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 -- 172.21.15.39:0/1730404564 >> 172.21.15.39:0/1730404564 conn(0x7f9198076c10 msgr2=0x7f91980fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 -- 172.21.15.39:0/1730404564 shutdown_connections 2024-05-24T11:40:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:14.374+0000 7f919e512700 1 -- 172.21.15.39:0/1730404564 wait complete. 2024-05-24T11:40:15.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:15 smithi053 ceph-mon[47448]: pgmap v64: 97 pgs: 97 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 27 MiB/s wr, 1.92k op/s 2024-05-24T11:40:15.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:15 smithi137 ceph-mon[47907]: pgmap v64: 97 pgs: 97 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 27 MiB/s wr, 1.92k op/s 2024-05-24T11:40:15.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:15 smithi039 ceph-mon[42919]: pgmap v64: 97 pgs: 97 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 27 MiB/s wr, 1.92k op/s 2024-05-24T11:40:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:17 smithi053 ceph-mon[47448]: pgmap v65: 97 pgs: 97 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 18 MiB/s wr, 1.29k op/s 2024-05-24T11:40:17.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:17 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:40:17.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:17 smithi137 ceph-mon[47907]: pgmap v65: 97 pgs: 97 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 18 MiB/s wr, 1.29k op/s 2024-05-24T11:40:17.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:17 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:40:17.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:17 smithi039 ceph-mon[42919]: pgmap v65: 97 pgs: 97 active+clean; 1.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 18 MiB/s wr, 1.29k op/s 2024-05-24T11:40:17.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:17 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:40:18.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:18 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:40:18.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:18 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:40:18.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:18 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:40:18.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:18 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:40:18.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:18 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:40:18.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:18 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:40:18.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:18 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:40:18.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:18 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:40:18.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:18 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:40:19.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 55.0 s, 1185.0 tps, lat 84.589 ms stddev 84.995 2024-05-24T11:40:19.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:19 smithi053 ceph-mon[47448]: pgmap v66: 97 pgs: 97 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 27 MiB/s wr, 1.93k op/s 2024-05-24T11:40:19.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:19 smithi137 ceph-mon[47907]: pgmap v66: 97 pgs: 97 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 27 MiB/s wr, 1.93k op/s 2024-05-24T11:40:19.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:19 smithi039 ceph-mon[42919]: pgmap v66: 97 pgs: 97 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 27 MiB/s wr, 1.93k op/s 2024-05-24T11:40:21.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:21 smithi053 ceph-mon[47448]: pgmap v67: 97 pgs: 97 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:40:21.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:21 smithi137 ceph-mon[47907]: pgmap v67: 97 pgs: 97 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:40:21.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:21 smithi039 ceph-mon[42919]: pgmap v67: 97 pgs: 97 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:40:23.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:23 smithi053 ceph-mon[47448]: pgmap v68: 97 pgs: 97 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:40:23.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:23 smithi137 ceph-mon[47907]: pgmap v68: 97 pgs: 97 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:40:23.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:23 smithi039 ceph-mon[42919]: pgmap v68: 97 pgs: 97 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:40:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 60.0 s, 1284.2 tps, lat 77.715 ms stddev 68.165 2024-05-24T11:40:25.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:25 smithi053 ceph-mon[47448]: pgmap v69: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 25 MiB/s wr, 1.84k op/s 2024-05-24T11:40:25.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:25 smithi137 ceph-mon[47907]: pgmap v69: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 25 MiB/s wr, 1.84k op/s 2024-05-24T11:40:25.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:25 smithi039 ceph-mon[42919]: pgmap v69: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 25 MiB/s wr, 1.84k op/s 2024-05-24T11:40:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.535+0000 7f658e25f700 1 -- 172.21.15.39:0/326269487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 msgr2=0x7f6588108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.535+0000 7f658e25f700 1 --2- 172.21.15.39:0/326269487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 0x7f6588108380 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f65780098f0 tx=0x7f6578009c00 comp rx=0 tx=0).stop 2024-05-24T11:40:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.535+0000 7f658e25f700 1 -- 172.21.15.39:0/326269487 shutdown_connections 2024-05-24T11:40:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.535+0000 7f658e25f700 1 --2- 172.21.15.39:0/326269487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65881140f0 0x7f65881164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.535+0000 7f658e25f700 1 --2- 172.21.15.39:0/326269487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 0x7f6588108380 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.535+0000 7f658e25f700 1 --2- 172.21.15.39:0/326269487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65881088c0 0x7f6588113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.535+0000 7f658e25f700 1 -- 172.21.15.39:0/326269487 >> 172.21.15.39:0/326269487 conn(0x7f658806bf80 msgr2=0x7f658806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.536+0000 7f658e25f700 1 -- 172.21.15.39:0/326269487 shutdown_connections 2024-05-24T11:40:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.536+0000 7f658e25f700 1 -- 172.21.15.39:0/326269487 wait complete. 2024-05-24T11:40:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.537+0000 7f658e25f700 1 Processor -- start 2024-05-24T11:40:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.537+0000 7f658e25f700 1 -- start start 2024-05-24T11:40:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.537+0000 7f658e25f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 0x7f65881a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.537+0000 7f658e25f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65881088c0 0x7f65881a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.537+0000 7f658e25f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65881140f0 0x7f658819f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f658e25f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658819ff50 con 0x7f6588107fa0 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f658e25f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65881a00c0 con 0x7f65881088c0 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f658e25f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65881a0280 con 0x7f65881140f0 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f65877fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 0x7f65881a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f65877fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 0x7f65881a57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56970/0 (socket says 172.21.15.39:56970) 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f65877fe700 1 -- 172.21.15.39:0/2295695656 learned_addr learned my addr 172.21.15.39:0/2295695656 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f6586ffd700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65881088c0 0x7f65881a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f6587fff700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65881140f0 0x7f658819f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f65877fe700 1 -- 172.21.15.39:0/2295695656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65881140f0 msgr2=0x7f658819f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f65877fe700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65881140f0 0x7f658819f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f65877fe700 1 -- 172.21.15.39:0/2295695656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65881088c0 msgr2=0x7f65881a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f65877fe700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65881088c0 0x7f65881a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f65877fe700 1 -- 172.21.15.39:0/2295695656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6578009520 con 0x7f6588107fa0 2024-05-24T11:40:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f6586ffd700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65881088c0 0x7f65881a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:40:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.538+0000 7f65877fe700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 0x7f65881a57c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6578005950 tx=0x7f657800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.539+0000 7f6584ff9700 1 -- 172.21.15.39:0/2295695656 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657801c070 con 0x7f6588107fa0 2024-05-24T11:40:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.539+0000 7f6584ff9700 1 -- 172.21.15.39:0/2295695656 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f657800b9f0 con 0x7f6588107fa0 2024-05-24T11:40:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.539+0000 7f658e25f700 1 -- 172.21.15.39:0/2295695656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65881a0500 con 0x7f6588107fa0 2024-05-24T11:40:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.539+0000 7f658e25f700 1 -- 172.21.15.39:0/2295695656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65881ac7b0 con 0x7f6588107fa0 2024-05-24T11:40:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.539+0000 7f6584ff9700 1 -- 172.21.15.39:0/2295695656 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65780038e0 con 0x7f6588107fa0 2024-05-24T11:40:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f659a41ab80 1 Processor -- start 2024-05-24T11:40:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f659a41ab80 1 -- start start 2024-05-24T11:40:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a8970 0x55af767a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767a4f40 0x55af767a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a5900 0x55af767b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f659a41ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af766fc0f0 con 0x55af767a8970 2024-05-24T11:40:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f659a41ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af766f9ab0 con 0x55af767a5900 2024-05-24T11:40:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f659a41ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af766f53a0 con 0x55af767a4f40 2024-05-24T11:40:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a5900 0x55af767b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a5900 0x55af767b9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:39374/0 (socket says 172.21.15.39:39374) 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.544+0000 7f6586ffd700 1 -- 172.21.15.39:0/2612935761 learned_addr learned my addr 172.21.15.39:0/2612935761 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f6586ffd700 1 -- 172.21.15.39:0/2612935761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767a4f40 msgr2=0x55af767a53c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f6586ffd700 1 --2- 172.21.15.39:0/2612935761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767a4f40 0x55af767a53c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f6586ffd700 1 -- 172.21.15.39:0/2612935761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a8970 msgr2=0x55af767a8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f6587fff700 1 --2- 172.21.15.39:0/2612935761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a8970 0x55af767a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f6586ffd700 1 --2- 172.21.15.39:0/2612935761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a8970 0x55af767a8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f6586ffd700 1 -- 172.21.15.39:0/2612935761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af76576350 con 0x55af767a5900 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f6587fff700 1 --2- 172.21.15.39:0/2612935761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a8970 0x55af767a8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f6586ffd700 1 --2- 172.21.15.39:0/2612935761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a5900 0x55af767b9bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f657c0115f0 tx=0x7f657c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57570f3acbf6f83e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f6584ff9700 1 -- 172.21.15.39:0/2295695656 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6578020030 con 0x7f6588107fa0 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.545+0000 7f656dffb700 1 -- 172.21.15.39:0/2612935761 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c004030 con 0x55af767a5900 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f656dffb700 1 -- 172.21.15.39:0/2612935761 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f657c003680 con 0x55af767a5900 2024-05-24T11:40:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f656dffb700 1 -- 172.21.15.39:0/2612935761 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c0196b0 con 0x55af767a5900 2024-05-24T11:40:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f6584ff9700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f65700a1310 0x7f65700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f6584ff9700 1 -- 172.21.15.39:0/2295695656 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f65780cd8e0 con 0x7f6588107fa0 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f659a41ab80 1 -- 172.21.15.39:0/2612935761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a5900 msgr2=0x55af767b9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2612935761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a5900 0x55af767b9bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f657c0115f0 tx=0x7f657c011900 comp rx=0 tx=0).stop 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f6587fff700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f65700a1310 0x7f65700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f659a41ab80 1 -- 172.21.15.39:0/2612935761 shutdown_connections 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2612935761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a5900 0x55af767b9bb0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f657c0115f0 tx=0x7f657c011900 comp rx=0 tx=0).stop 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2612935761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a8970 0x55af767a8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2612935761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767a4f40 0x55af767a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f659a41ab80 1 -- 172.21.15.39:0/2612935761 >> 172.21.15.39:0/2612935761 conn(0x55af767031b0 msgr2=0x55af7679f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.546+0000 7f659a41ab80 1 -- 172.21.15.39:0/2612935761 shutdown_connections 2024-05-24T11:40:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 1 -- 172.21.15.39:0/2612935761 wait complete. 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 1 -- start start 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f6587fff700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f65700a1310 0x7f65700a37d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f658806a570 tx=0x7f658000b040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 1 Processor -- start 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767a4f40 0x55af767b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767b9600 0x55af767b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af76691500 con 0x55af767b9600 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af766911a0 con 0x55af767a8970 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af76578640 con 0x55af767a4f40 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.547+0000 7f659a41ab80 10 client.0 _ll_register_callbacks cb 0x55af7669aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6587fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767b9600 0x55af767b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:39390/0 (socket says 172.21.15.39:39390) 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6586ffd700 1 -- 172.21.15.39:0/2263702659 learned_addr learned my addr 172.21.15.39:0/2263702659 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f65877fe700 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767a4f40 0x55af767b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6587fff700 1 -- 172.21.15.39:0/2263702659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767a4f40 msgr2=0x55af767b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6587fff700 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767a4f40 0x55af767b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6587fff700 1 -- 172.21.15.39:0/2263702659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 msgr2=0x55af767b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6587fff700 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6587fff700 1 -- 172.21.15.39:0/2263702659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c009d50 con 0x55af767b9600 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6587fff700 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767b9600 0x55af767b9a80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6580007c30 tx=0x7f6580004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8b2b79f77a5e32 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.548+0000 7f6566ffd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.549+0000 7f6566ffd700 1 -- 172.21.15.39:0/2263702659 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658000e040 con 0x55af767b9600 2024-05-24T11:40:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.549+0000 7f6566ffd700 1 -- 172.21.15.39:0/2263702659 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f6580016ab0 con 0x55af767b9600 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.549+0000 7f659a41ab80 1 -- 172.21.15.39:0/2263702659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af7664e210 con 0x55af767b9600 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.549+0000 7f6566ffd700 1 -- 172.21.15.39:0/2263702659 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6580016c70 con 0x55af767b9600 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.549+0000 7f659a41ab80 10 client.15603 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.549+0000 7f659a41ab80 1 -- 172.21.15.39:0/2263702659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af76522b00 con 0x55af767b9600 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.549+0000 7f6566ffd700 1 -- 172.21.15.39:0/2263702659 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af7664e210 con 0x55af767b9600 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.549+0000 7f6566ffd700 1 -- 172.21.15.39:0/2263702659 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6580023cf0 con 0x55af767b9600 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.550+0000 7f659a41ab80 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.550+0000 7f659a41ab80 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.550+0000 7f659a41ab80 20 client.15603 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.550+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55af767d0300 0x55af767d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.550+0000 7f659a41ab80 4 client.15603 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.550+0000 7f659a41ab80 1 -- 172.21.15.39:0/2263702659 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af767747b0 con 0x55af767d0300 2024-05-24T11:40:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.550+0000 7f65877fe700 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55af767d0300 0x55af767d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.551+0000 7f65877fe700 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55af767d0300 0x55af767d2720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6c45c11cfea0bb server_cookie=33f8c0a4507c95a9 in_seq=0 out_seq=0 2024-05-24T11:40:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.551+0000 7f6566ffd700 10 client.15603 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.551+0000 7f6566ffd700 1 -- 172.21.15.39:0/2263702659 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af767747b0 con 0x55af767d0300 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.551+0000 7f6566ffd700 10 client.15603 handle_command_reply: tid=0 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.551+0000 7f659a41ab80 1 client.15603 shutdown 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 -- 172.21.15.39:0/2263702659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767b9600 msgr2=0x55af767b9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767b9600 0x55af767b9a80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6580007c30 tx=0x7f6580004b40 comp rx=0 tx=0).stop 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 -- 172.21.15.39:0/2263702659 shutdown_connections 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767a4f40 0x55af767b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55af767d0300 0x55af767d2720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767b9600 0x55af767b9a80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 --2- 172.21.15.39:0/2263702659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 -- 172.21.15.39:0/2263702659 >> 172.21.15.39:0/2263702659 conn(0x55af767031b0 msgr2=0x55af767a1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f6566ffd700 0 client.15603 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 -- 172.21.15.39:0/2263702659 shutdown_connections 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 1 -- 172.21.15.39:0/2263702659 wait complete. 2024-05-24T11:40:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.552+0000 7f659a41ab80 20 client.15603 trim_cache size 0 max 16384 2024-05-24T11:40:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f659a41ab80 1 Processor -- start 2024-05-24T11:40:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f659a41ab80 1 -- start start 2024-05-24T11:40:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a4f40 0x55af767a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767b9600 0x55af767b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f659a41ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af767aece0 con 0x55af767a4f40 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f659a41ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af767a0960 con 0x55af767a8970 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f659a41ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af767af9f0 con 0x55af767b9600 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:39406/0 (socket says 172.21.15.39:39406) 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.575+0000 7f6586ffd700 1 -- 172.21.15.39:0/491002997 learned_addr learned my addr 172.21.15.39:0/491002997 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.576+0000 7f6586ffd700 1 -- 172.21.15.39:0/491002997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767b9600 msgr2=0x55af767b99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.576+0000 7f6586ffd700 1 --2- 172.21.15.39:0/491002997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767b9600 0x55af767b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.576+0000 7f6586ffd700 1 -- 172.21.15.39:0/491002997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a4f40 msgr2=0x55af767a8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.576+0000 7f6586ffd700 1 --2- 172.21.15.39:0/491002997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a4f40 0x55af767a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.576+0000 7f6586ffd700 1 -- 172.21.15.39:0/491002997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af766bb500 con 0x55af767a8970 2024-05-24T11:40:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.577+0000 7f6586ffd700 1 --2- 172.21.15.39:0/491002997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767aeed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f657c0097c0 tx=0x7f657c0037f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d09db883045118 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.578+0000 7f656e7fc700 1 -- 172.21.15.39:0/491002997 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c003a90 con 0x55af767a8970 2024-05-24T11:40:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.579+0000 7f656e7fc700 1 -- 172.21.15.39:0/491002997 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f657c003bf0 con 0x55af767a8970 2024-05-24T11:40:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.579+0000 7f656e7fc700 1 -- 172.21.15.39:0/491002997 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c019a10 con 0x55af767a8970 2024-05-24T11:40:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 -- 172.21.15.39:0/491002997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 msgr2=0x55af767aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 --2- 172.21.15.39:0/491002997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767aeed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f657c0097c0 tx=0x7f657c0037f0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 -- 172.21.15.39:0/491002997 shutdown_connections 2024-05-24T11:40:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 --2- 172.21.15.39:0/491002997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767b9600 0x55af767b99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 --2- 172.21.15.39:0/491002997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a4f40 0x55af767a8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 --2- 172.21.15.39:0/491002997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767aeed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 -- 172.21.15.39:0/491002997 >> 172.21.15.39:0/491002997 conn(0x55af767031b0 msgr2=0x55af7679be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 -- 172.21.15.39:0/491002997 shutdown_connections 2024-05-24T11:40:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 -- 172.21.15.39:0/491002997 wait complete. 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.580+0000 7f659a41ab80 1 -- start start 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f659a41ab80 1 Processor -- start 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a4f40 0x55af767b8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f659a41ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767b9600 0x55af767b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f659a41ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af767b63f0 con 0x55af767a4f40 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f659a41ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af767b6560 con 0x55af767a8970 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f659a41ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af767b6720 con 0x55af767b9600 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f659a41ab80 10 client.0 _ll_register_callbacks cb 0x55af7669aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f6586ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:39420/0 (socket says 172.21.15.39:39420) 2024-05-24T11:40:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.581+0000 7f6586ffd700 1 -- 172.21.15.39:0/1984366694 learned_addr learned my addr 172.21.15.39:0/1984366694 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.582+0000 7f6587fff700 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767b9600 0x55af767b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.582+0000 7f65877fe700 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a4f40 0x55af767b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.582+0000 7f6586ffd700 1 -- 172.21.15.39:0/1984366694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767b9600 msgr2=0x55af767b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.582+0000 7f6586ffd700 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767b9600 0x55af767b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.582+0000 7f6586ffd700 1 -- 172.21.15.39:0/1984366694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a4f40 msgr2=0x55af767b8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.582+0000 7f6586ffd700 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a4f40 0x55af767b8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.582+0000 7f6586ffd700 1 -- 172.21.15.39:0/1984366694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af76576350 con 0x55af767a8970 2024-05-24T11:40:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.582+0000 7f6586ffd700 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b8d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f657c001eb0 tx=0x7f657c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61d3859b8485e4b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.582+0000 7f6566ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:40:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.583+0000 7f6566ffd700 1 -- 172.21.15.39:0/1984366694 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c005ce0 con 0x55af767a8970 2024-05-24T11:40:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.583+0000 7f6566ffd700 1 -- 172.21.15.39:0/1984366694 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f657c0198a0 con 0x55af767a8970 2024-05-24T11:40:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.584+0000 7f659a41ab80 1 -- 172.21.15.39:0/1984366694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f657c003680 con 0x55af767a8970 2024-05-24T11:40:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.584+0000 7f6566ffd700 1 -- 172.21.15.39:0/1984366694 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f657c019a80 con 0x55af767a8970 2024-05-24T11:40:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.584+0000 7f659a41ab80 10 client.25301 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:27.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.584+0000 7f659a41ab80 1 -- 172.21.15.39:0/1984366694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af7657ac70 con 0x55af767a8970 2024-05-24T11:40:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.585+0000 7f6566ffd700 1 -- 172.21.15.39:0/1984366694 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f657c003680 con 0x55af767a8970 2024-05-24T11:40:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.586+0000 7f6566ffd700 1 -- 172.21.15.39:0/1984366694 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f657c03fb80 con 0x55af767a8970 2024-05-24T11:40:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.586+0000 7f659a41ab80 10 client.25301 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.586+0000 7f659a41ab80 10 client.25301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.586+0000 7f659a41ab80 20 client.25301 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.586+0000 7f659a41ab80 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55af76826670 0x55af76828a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.586+0000 7f659a41ab80 4 client.25301 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:40:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.586+0000 7f659a41ab80 1 -- 172.21.15.39:0/1984366694 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af766ede60 con 0x55af76826670 2024-05-24T11:40:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.587+0000 7f65877fe700 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55af76826670 0x55af76828a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.587+0000 7f6566ffd700 10 client.25301 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.587+0000 7f65877fe700 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55af76826670 0x55af76828a60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4370824c131802 server_cookie=dcde229103e9a3f5 in_seq=0 out_seq=0 2024-05-24T11:40:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.615+0000 7f6566ffd700 1 -- 172.21.15.39:0/1984366694 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af766ede60 con 0x55af76826670 2024-05-24T11:40:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.615+0000 7f6566ffd700 10 client.25301 handle_command_reply: tid=0 2024-05-24T11:40:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.615+0000 7f659a41ab80 1 client.25301 shutdown 2024-05-24T11:40:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f659a41ab80 1 -- 172.21.15.39:0/1984366694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 msgr2=0x55af767b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f659a41ab80 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b8d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f657c001eb0 tx=0x7f657c011900 comp rx=0 tx=0).stop 2024-05-24T11:40:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f659a41ab80 1 -- 172.21.15.39:0/1984366694 shutdown_connections 2024-05-24T11:40:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f659a41ab80 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af767b9600 0x55af767b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f659a41ab80 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55af76826670 0x55af76828a60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f659a41ab80 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55af767a4f40 0x55af767b8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f659a41ab80 1 --2- 172.21.15.39:0/1984366694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af767a8970 0x55af767b8d90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f659a41ab80 1 -- 172.21.15.39:0/1984366694 >> 172.21.15.39:0/1984366694 conn(0x55af767031b0 msgr2=0x55af767a9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f6566ffd700 0 client.25301 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.616+0000 7f659a41ab80 1 -- 172.21.15.39:0/1984366694 shutdown_connections 2024-05-24T11:40:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.617+0000 7f659a41ab80 1 -- 172.21.15.39:0/1984366694 wait complete. 2024-05-24T11:40:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.618+0000 7f659a41ab80 20 client.25301 trim_cache size 0 max 16384 2024-05-24T11:40:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:40:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:40:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:40:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:40:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 -- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f65700a1310 msgr2=0x7f65700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f65700a1310 0x7f65700a37d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f658806a570 tx=0x7f658000b040 comp rx=0 tx=0).stop 2024-05-24T11:40:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 -- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 msgr2=0x7f65881a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 0x7f65881a57c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6578005950 tx=0x7f657800fa00 comp rx=0 tx=0).stop 2024-05-24T11:40:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 -- 172.21.15.39:0/2295695656 shutdown_connections 2024-05-24T11:40:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65881140f0 0x7f658819f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f65700a1310 0x7f65700a37d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6588107fa0 0x7f65881a57c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 --2- 172.21.15.39:0/2295695656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65881088c0 0x7f65881a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 -- 172.21.15.39:0/2295695656 >> 172.21.15.39:0/2295695656 conn(0x7f658806bf80 msgr2=0x7f658806fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 -- 172.21.15.39:0/2295695656 shutdown_connections 2024-05-24T11:40:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:27.619+0000 7f658e25f700 1 -- 172.21.15.39:0/2295695656 wait complete. 2024-05-24T11:40:27.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:27 smithi053 ceph-mon[47448]: pgmap v70: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2024-05-24T11:40:27.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:27 smithi137 ceph-mon[47907]: pgmap v70: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2024-05-24T11:40:27.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:27 smithi039 ceph-mon[42919]: pgmap v70: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2024-05-24T11:40:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 65.0 s, 1063.6 tps, lat 87.614 ms stddev 123.606 2024-05-24T11:40:29.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:29 smithi137 ceph-mon[47907]: pgmap v71: 97 pgs: 97 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 27 MiB/s wr, 1.95k op/s 2024-05-24T11:40:29.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:29 smithi039 ceph-mon[42919]: pgmap v71: 97 pgs: 97 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 27 MiB/s wr, 1.95k op/s 2024-05-24T11:40:29.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:29 smithi053 ceph-mon[47448]: pgmap v71: 97 pgs: 97 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 27 MiB/s wr, 1.95k op/s 2024-05-24T11:40:31.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:31 smithi137 ceph-mon[47907]: pgmap v72: 97 pgs: 97 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2024-05-24T11:40:31.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:31 smithi039 ceph-mon[42919]: pgmap v72: 97 pgs: 97 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2024-05-24T11:40:31.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:31 smithi053 ceph-mon[47448]: pgmap v72: 97 pgs: 97 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2024-05-24T11:40:33.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:33 smithi039 ceph-mon[42919]: pgmap v73: 97 pgs: 97 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2024-05-24T11:40:33.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:33 smithi053 ceph-mon[47448]: pgmap v73: 97 pgs: 97 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2024-05-24T11:40:33.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:33 smithi137 ceph-mon[47907]: pgmap v73: 97 pgs: 97 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2024-05-24T11:40:34.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 70.0 s, 1095.8 tps, lat 89.691 ms stddev 145.207 2024-05-24T11:40:35.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:35 smithi039 ceph-mon[42919]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2024-05-24T11:40:35.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:35 smithi053 ceph-mon[47448]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2024-05-24T11:40:35.998 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:35 smithi137 ceph-mon[47907]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 32 MiB/s wr, 1.88k op/s 2024-05-24T11:40:37.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:37 smithi039 ceph-mon[42919]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 24 MiB/s wr, 1.26k op/s 2024-05-24T11:40:37.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:37 smithi053 ceph-mon[47448]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 24 MiB/s wr, 1.26k op/s 2024-05-24T11:40:37.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:37 smithi137 ceph-mon[47907]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 24 MiB/s wr, 1.26k op/s 2024-05-24T11:40:39.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 75.0 s, 1091.3 tps, lat 99.183 ms stddev 138.548 2024-05-24T11:40:39.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:39 smithi039 ceph-mon[42919]: pgmap v76: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 35 MiB/s wr, 1.76k op/s 2024-05-24T11:40:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:39 smithi053 ceph-mon[47448]: pgmap v76: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 35 MiB/s wr, 1.76k op/s 2024-05-24T11:40:39.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:39 smithi137 ceph-mon[47907]: pgmap v76: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 35 MiB/s wr, 1.76k op/s 2024-05-24T11:40:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.790+0000 7f16a3b2b700 1 -- 172.21.15.39:0/102859686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c0ff520 msgr2=0x7f169c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.790+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/102859686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c0ff520 0x7f169c0ff9a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f169800aad0 tx=0x7f169800ade0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.790+0000 7f16a3b2b700 1 -- 172.21.15.39:0/102859686 shutdown_connections 2024-05-24T11:40:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.790+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/102859686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169c1055e0 0x7f169c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.790+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/102859686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169c0ffee0 0x7f169c10c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.790+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/102859686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c0ff520 0x7f169c0ff9a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.790+0000 7f16a3b2b700 1 -- 172.21.15.39:0/102859686 >> 172.21.15.39:0/102859686 conn(0x7f169c0763a0 msgr2=0x7f169c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.791+0000 7f16a3b2b700 1 -- 172.21.15.39:0/102859686 shutdown_connections 2024-05-24T11:40:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.791+0000 7f16a3b2b700 1 -- 172.21.15.39:0/102859686 wait complete. 2024-05-24T11:40:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.791+0000 7f16a3b2b700 1 Processor -- start 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.791+0000 7f16a3b2b700 1 -- start start 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a3b2b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c0ff520 0x7f169c1063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a3b2b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169c0ffee0 0x7f169c1068f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a3b2b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169c1055e0 0x7f169c106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a3b2b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c1a9ae0 con 0x7f169c0ff520 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a3b2b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c1a9c20 con 0x7f169c0ffee0 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a3b2b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c1a9d90 con 0x7f169c1055e0 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a18c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c0ff520 0x7f169c1063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a18c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c0ff520 0x7f169c1063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47062/0 (socket says 172.21.15.39:47062) 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a18c7700 1 -- 172.21.15.39:0/3441203913 learned_addr learned my addr 172.21.15.39:0/3441203913 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a20c8700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169c1055e0 0x7f169c106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.792+0000 7f16a10c6700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169c0ffee0 0x7f169c1068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.793+0000 7f16a20c8700 1 -- 172.21.15.39:0/3441203913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169c0ffee0 msgr2=0x7f169c1068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.793+0000 7f16a20c8700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169c0ffee0 0x7f169c1068f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.793+0000 7f16a20c8700 1 -- 172.21.15.39:0/3441203913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c0ff520 msgr2=0x7f169c1063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.793+0000 7f16a20c8700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c0ff520 0x7f169c1063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.793+0000 7f16a20c8700 1 -- 172.21.15.39:0/3441203913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169800a700 con 0x7f169c1055e0 2024-05-24T11:40:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.793+0000 7f16a20c8700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169c1055e0 0x7f169c106e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e970 tx=0x7f168c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.804+0000 7f1692ffd700 1 -- 172.21.15.39:0/3441203913 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c004220 con 0x7f169c1055e0 2024-05-24T11:40:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.804+0000 7f1692ffd700 1 -- 172.21.15.39:0/3441203913 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f168c004cc0 con 0x7f169c1055e0 2024-05-24T11:40:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.804+0000 7f16a3b2b700 1 -- 172.21.15.39:0/3441203913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c1aa070 con 0x7f169c1055e0 2024-05-24T11:40:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.804+0000 7f1692ffd700 1 -- 172.21.15.39:0/3441203913 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c005590 con 0x7f169c1055e0 2024-05-24T11:40:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.804+0000 7f16a3b2b700 1 -- 172.21.15.39:0/3441203913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c1aa550 con 0x7f169c1055e0 2024-05-24T11:40:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.807+0000 7f16afce6b80 1 Processor -- start 2024-05-24T11:40:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.808+0000 7f1692ffd700 1 -- 172.21.15.39:0/3441203913 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f168c010700 con 0x7f169c1055e0 2024-05-24T11:40:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.808+0000 7f16afce6b80 1 -- start start 2024-05-24T11:40:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16afce6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c8030f0 con 0x557f1c8ac480 2024-05-24T11:40:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16afce6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c800ab0 con 0x557f1c8b2270 2024-05-24T11:40:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16afce6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c7fc3a0 con 0x557f1c8b1950 2024-05-24T11:40:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16a10c6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16a10c6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:37144/0 (socket says 172.21.15.39:37144) 2024-05-24T11:40:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16a10c6700 1 -- 172.21.15.39:0/548005460 learned_addr learned my addr 172.21.15.39:0/548005460 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16a18c7700 1 --2- 172.21.15.39:0/548005460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.809+0000 7f16a10c6700 1 --2- 172.21.15.39:0/548005460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f1692ffd700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f16880a1440 0x7f16880a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f16a10c6700 1 -- 172.21.15.39:0/548005460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 msgr2=0x557f1c8abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f16a10c6700 1 --2- 172.21.15.39:0/548005460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f16a10c6700 1 -- 172.21.15.39:0/548005460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 msgr2=0x557f1c8ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f16a10c6700 1 --2- 172.21.15.39:0/548005460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f16a10c6700 1 -- 172.21.15.39:0/548005460 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f1c67d350 con 0x557f1c8b1950 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f1692ffd700 1 -- 172.21.15.39:0/3441203913 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f168c014070 con 0x7f169c1055e0 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f16a10c6700 1 --2- 172.21.15.39:0/548005460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8b1d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f169800a9f0 tx=0x7f16980058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b380a1e7b60b4a61 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f1687fff700 1 -- 172.21.15.39:0/548005460 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698003680 con 0x557f1c8b1950 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f1687fff700 1 -- 172.21.15.39:0/548005460 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16980037e0 con 0x557f1c8b1950 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f1687fff700 1 -- 172.21.15.39:0/548005460 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800e450 con 0x557f1c8b1950 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.810+0000 7f16a20c8700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f16880a1440 0x7f16880a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.811+0000 7f16afce6b80 1 -- 172.21.15.39:0/548005460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 msgr2=0x557f1c8b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.811+0000 7f16afce6b80 1 --2- 172.21.15.39:0/548005460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8b1d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f169800a9f0 tx=0x7f16980058e0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.811+0000 7f16afce6b80 1 -- 172.21.15.39:0/548005460 shutdown_connections 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.811+0000 7f16afce6b80 1 --2- 172.21.15.39:0/548005460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8b1d30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.811+0000 7f16afce6b80 1 --2- 172.21.15.39:0/548005460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.811+0000 7f16afce6b80 1 --2- 172.21.15.39:0/548005460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.811+0000 7f16afce6b80 1 -- 172.21.15.39:0/548005460 >> 172.21.15.39:0/548005460 conn(0x557f1c80a1b0 msgr2=0x557f1c8a6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.811+0000 7f16a20c8700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f16880a1440 0x7f16880a3900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f168c0a1a00 tx=0x7f168c013000 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.812+0000 7f16afce6b80 1 -- 172.21.15.39:0/548005460 shutdown_connections 2024-05-24T11:40:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.812+0000 7f16afce6b80 1 -- 172.21.15.39:0/548005460 wait complete. 2024-05-24T11:40:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.812+0000 7f16afce6b80 1 -- start start 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.812+0000 7f16afce6b80 1 Processor -- start 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.813+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8ac480 0x557f1c8b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.813+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.813+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8b2270 0x557f1c8b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.813+0000 7f16afce6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c798500 con 0x557f1c8b2270 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.813+0000 7f16afce6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c7981a0 con 0x557f1c8b1950 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.813+0000 7f16afce6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c67f640 con 0x557f1c8ac480 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.813+0000 7f16afce6b80 10 client.0 _ll_register_callbacks cb 0x557f1c7a1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a18c7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a18c7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:38918/0 (socket says 172.21.15.39:38918) 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a18c7700 1 -- 172.21.15.39:0/3614886356 learned_addr learned my addr 172.21.15.39:0/3614886356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a10c6700 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8b2270 0x557f1c8b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a10c6700 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8ac480 0x557f1c8b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a18c7700 1 -- 172.21.15.39:0/3614886356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8ac480 msgr2=0x557f1c8b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a18c7700 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8ac480 0x557f1c8b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a18c7700 1 -- 172.21.15.39:0/3614886356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8b2270 msgr2=0x557f1c8b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a18c7700 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8b2270 0x557f1c8b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a18c7700 1 -- 172.21.15.39:0/3614886356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169801a040 con 0x557f1c8b1950 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16a18c7700 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8b5ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1694014e10 tx=0x7f169400e610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c38fa9dd0ce3b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.814+0000 7f16857fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.815+0000 7f16857fa700 1 -- 172.21.15.39:0/3614886356 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169400e040 con 0x557f1c8b1950 2024-05-24T11:40:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.815+0000 7f16857fa700 1 -- 172.21.15.39:0/3614886356 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f169400eda0 con 0x557f1c8b1950 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.815+0000 7f16afce6b80 1 -- 172.21.15.39:0/3614886356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f1c755210 con 0x557f1c8b1950 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.815+0000 7f16857fa700 1 -- 172.21.15.39:0/3614886356 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1694003a00 con 0x557f1c8b1950 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.815+0000 7f16afce6b80 10 client.25313 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.815+0000 7f16afce6b80 1 -- 172.21.15.39:0/3614886356 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f1c629b00 con 0x557f1c8b1950 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.815+0000 7f16857fa700 1 -- 172.21.15.39:0/3614886356 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557f1c755210 con 0x557f1c8b1950 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16857fa700 1 -- 172.21.15.39:0/3614886356 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f169401c030 con 0x557f1c8b1950 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16afce6b80 10 client.25313 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16afce6b80 10 client.25313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16afce6b80 20 client.25313 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557f1c8d6ce0 0x557f1c8d9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16afce6b80 4 client.25313 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16afce6b80 1 -- 172.21.15.39:0/3614886356 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f1c87b7b0 con 0x557f1c8d6ce0 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16a10c6700 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557f1c8d6ce0 0x557f1c8d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16a10c6700 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557f1c8d6ce0 0x557f1c8d9100 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c937389c6bf1d1a0 server_cookie=d7a5a2e79840fb1b in_seq=0 out_seq=0 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.816+0000 7f16857fa700 10 client.25313 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16857fa700 1 -- 172.21.15.39:0/3614886356 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f1c87b7b0 con 0x557f1c8d6ce0 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16857fa700 10 client.25313 handle_command_reply: tid=0 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 client.25313 shutdown 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 -- 172.21.15.39:0/3614886356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 msgr2=0x557f1c8b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8b5ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1694014e10 tx=0x7f169400e610 comp rx=0 tx=0).stop 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 -- 172.21.15.39:0/3614886356 shutdown_connections 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8ac480 0x557f1c8b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557f1c8d6ce0 0x557f1c8d9100 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8b2270 0x557f1c8b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3614886356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8b5ad0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 -- 172.21.15.39:0/3614886356 >> 172.21.15.39:0/3614886356 conn(0x557f1c80a1b0 msgr2=0x557f1c8aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16857fa700 0 client.25313 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 -- 172.21.15.39:0/3614886356 shutdown_connections 2024-05-24T11:40:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.817+0000 7f16afce6b80 1 -- 172.21.15.39:0/3614886356 wait complete. 2024-05-24T11:40:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.818+0000 7f16afce6b80 20 client.25313 trim_cache size 0 max 16384 2024-05-24T11:40:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.844+0000 7f16afce6b80 1 Processor -- start 2024-05-24T11:40:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.844+0000 7f16afce6b80 1 -- start start 2024-05-24T11:40:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8a6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8bcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b2270 0x557f1c8bd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16afce6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c8a17c0 con 0x557f1c8ac480 2024-05-24T11:40:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16afce6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c8d4340 con 0x557f1c8b1950 2024-05-24T11:40:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16afce6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c8a39b0 con 0x557f1c8b2270 2024-05-24T11:40:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16a10c6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16a10c6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47092/0 (socket says 172.21.15.39:47092) 2024-05-24T11:40:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16a10c6700 1 -- 172.21.15.39:0/111360808 learned_addr learned my addr 172.21.15.39:0/111360808 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16a18c7700 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8bcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.845+0000 7f16a10c6700 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b2270 0x557f1c8bd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f16a18c7700 1 -- 172.21.15.39:0/111360808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b2270 msgr2=0x557f1c8bd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f16a18c7700 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b2270 0x557f1c8bd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f16a18c7700 1 -- 172.21.15.39:0/111360808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 msgr2=0x557f1c8a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f16a18c7700 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f16a18c7700 1 -- 172.21.15.39:0/111360808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f1c7c2500 con 0x557f1c8b1950 2024-05-24T11:40:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f16a10c6700 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:40:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f16a18c7700 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8bcbd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f169400f010 tx=0x7f169401e600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff2a73b97b8137fe server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f1690da5700 1 -- 172.21.15.39:0/111360808 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169400eda0 con 0x557f1c8b1950 2024-05-24T11:40:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f1690da5700 1 -- 172.21.15.39:0/111360808 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f169400e030 con 0x557f1c8b1950 2024-05-24T11:40:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.846+0000 7f1690da5700 1 -- 172.21.15.39:0/111360808 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169401ec30 con 0x557f1c8b1950 2024-05-24T11:40:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.847+0000 7f16afce6b80 1 -- 172.21.15.39:0/111360808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 msgr2=0x557f1c8bcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.847+0000 7f16afce6b80 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8bcbd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f169400f010 tx=0x7f169401e600 comp rx=0 tx=0).stop 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.847+0000 7f16afce6b80 1 -- 172.21.15.39:0/111360808 shutdown_connections 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.847+0000 7f16afce6b80 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b2270 0x557f1c8bd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.847+0000 7f16afce6b80 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.847+0000 7f16afce6b80 1 --2- 172.21.15.39:0/111360808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b1950 0x557f1c8bcbd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.847+0000 7f16afce6b80 1 -- 172.21.15.39:0/111360808 >> 172.21.15.39:0/111360808 conn(0x557f1c80a1b0 msgr2=0x557f1c8b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.847+0000 7f16afce6b80 1 -- 172.21.15.39:0/111360808 shutdown_connections 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.847+0000 7f16afce6b80 1 -- 172.21.15.39:0/111360808 wait complete. 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.848+0000 7f16afce6b80 1 -- start start 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.848+0000 7f16afce6b80 1 Processor -- start 2024-05-24T11:40:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.848+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.848+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8babc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.848+0000 7f16afce6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.848+0000 7f16afce6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c8bb8e0 con 0x557f1c8ac480 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.848+0000 7f16afce6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c8bba50 con 0x557f1c8b2270 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.848+0000 7f16afce6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1c92c230 con 0x557f1c8b1950 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.848+0000 7f16afce6b80 10 client.0 _ll_register_callbacks cb 0x557f1c7a1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a10c6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a10c6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8bb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:38940/0 (socket says 172.21.15.39:38940) 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a10c6700 1 -- 172.21.15.39:0/3072431965 learned_addr learned my addr 172.21.15.39:0/3072431965 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a10c6700 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a18c7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8babc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a10c6700 1 -- 172.21.15.39:0/3072431965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 msgr2=0x557f1c8babc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a10c6700 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8babc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a10c6700 1 -- 172.21.15.39:0/3072431965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 msgr2=0x557f1c8bcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a10c6700 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8bcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.849+0000 7f16a10c6700 1 -- 172.21.15.39:0/3072431965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f1c629b00 con 0x557f1c8b2270 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.850+0000 7f16a10c6700 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8bb180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1698005ac0 tx=0x7f1698027040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62924f36dcd5769b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.850+0000 7f16857fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.850+0000 7f16857fa700 1 -- 172.21.15.39:0/3072431965 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16980363e0 con 0x557f1c8b2270 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.850+0000 7f16857fa700 1 -- 172.21.15.39:0/3072431965 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1698000e40 con 0x557f1c8b2270 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.850+0000 7f16afce6b80 1 -- 172.21.15.39:0/3072431965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16980037e0 con 0x557f1c8b2270 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.850+0000 7f16857fa700 1 -- 172.21.15.39:0/3072431965 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1698036540 con 0x557f1c8b2270 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16afce6b80 10 client.25319 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16afce6b80 1 -- 172.21.15.39:0/3072431965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f1c681c70 con 0x557f1c8b2270 2024-05-24T11:40:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16857fa700 1 -- 172.21.15.39:0/3072431965 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16980037e0 con 0x557f1c8b2270 2024-05-24T11:40:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16857fa700 1 -- 172.21.15.39:0/3072431965 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f169801e070 con 0x557f1c8b2270 2024-05-24T11:40:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16afce6b80 10 client.25319 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16afce6b80 10 client.25319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16afce6b80 20 client.25319 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557f1c92d610 0x557f1c92fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16afce6b80 4 client.25319 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:40:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.851+0000 7f16afce6b80 1 -- 172.21.15.39:0/3072431965 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f1c7f4e60 con 0x557f1c92d610 2024-05-24T11:40:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.852+0000 7f16a18c7700 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557f1c92d610 0x557f1c92fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.852+0000 7f16a18c7700 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557f1c92d610 0x557f1c92fa30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef176ef39a907445 server_cookie=9edf2ca920601401 in_seq=0 out_seq=0 2024-05-24T11:40:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.852+0000 7f16857fa700 10 client.25319 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.886+0000 7f16857fa700 1 -- 172.21.15.39:0/3072431965 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f1c7f4e60 con 0x557f1c92d610 2024-05-24T11:40:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.886+0000 7f16857fa700 10 client.25319 handle_command_reply: tid=0 2024-05-24T11:40:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.886+0000 7f16afce6b80 1 client.25319 shutdown 2024-05-24T11:40:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.887+0000 7f16afce6b80 1 -- 172.21.15.39:0/3072431965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 msgr2=0x557f1c8bb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.887+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8bb180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1698005ac0 tx=0x7f1698027040 comp rx=0 tx=0).stop 2024-05-24T11:40:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.888+0000 7f16afce6b80 1 -- 172.21.15.39:0/3072431965 shutdown_connections 2024-05-24T11:40:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.888+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557f1c92d610 0x557f1c92fa30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.888+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557f1c8b2270 0x557f1c8bb180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.888+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f1c8ac480 0x557f1c8bcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.888+0000 7f16afce6b80 1 --2- 172.21.15.39:0/3072431965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f1c8b1950 0x557f1c8babc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.888+0000 7f16afce6b80 1 -- 172.21.15.39:0/3072431965 >> 172.21.15.39:0/3072431965 conn(0x557f1c80a1b0 msgr2=0x557f1c8af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.888+0000 7f16857fa700 0 client.25319 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.888+0000 7f16afce6b80 1 -- 172.21.15.39:0/3072431965 shutdown_connections 2024-05-24T11:40:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.888+0000 7f16afce6b80 1 -- 172.21.15.39:0/3072431965 wait complete. 2024-05-24T11:40:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.889+0000 7f16afce6b80 20 client.25319 trim_cache size 0 max 16384 2024-05-24T11:40:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:40:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:40:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:40:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:40:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 -- 172.21.15.39:0/3441203913 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f16880a1440 msgr2=0x7f16880a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f16880a1440 0x7f16880a3900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f168c0a1a00 tx=0x7f168c013000 comp rx=0 tx=0).stop 2024-05-24T11:40:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 -- 172.21.15.39:0/3441203913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169c1055e0 msgr2=0x7f169c106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169c1055e0 0x7f169c106e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e970 tx=0x7f168c00ec80 comp rx=0 tx=0).stop 2024-05-24T11:40:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 -- 172.21.15.39:0/3441203913 shutdown_connections 2024-05-24T11:40:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169c1055e0 0x7f169c106e30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f16880a1440 0x7f16880a3900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c0ff520 0x7f169c1063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 --2- 172.21.15.39:0/3441203913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169c0ffee0 0x7f169c1068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 -- 172.21.15.39:0/3441203913 >> 172.21.15.39:0/3441203913 conn(0x7f169c0763a0 msgr2=0x7f169c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 -- 172.21.15.39:0/3441203913 shutdown_connections 2024-05-24T11:40:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:40.890+0000 7f16a3b2b700 1 -- 172.21.15.39:0/3441203913 wait complete. 2024-05-24T11:40:41.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:41 smithi039 ceph-mon[42919]: pgmap v77: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 24 MiB/s wr, 1.07k op/s 2024-05-24T11:40:41.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:41 smithi053 ceph-mon[47448]: pgmap v77: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 24 MiB/s wr, 1.07k op/s 2024-05-24T11:40:41.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:41 smithi137 ceph-mon[47907]: pgmap v77: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 24 MiB/s wr, 1.07k op/s 2024-05-24T11:40:43.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:43 smithi039 ceph-mon[42919]: pgmap v78: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 24 MiB/s wr, 1.07k op/s 2024-05-24T11:40:43.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:43 smithi053 ceph-mon[47448]: pgmap v78: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 24 MiB/s wr, 1.07k op/s 2024-05-24T11:40:43.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:43 smithi137 ceph-mon[47907]: pgmap v78: 97 pgs: 97 active+clean; 2.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 24 MiB/s wr, 1.07k op/s 2024-05-24T11:40:44.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 80.0 s, 992.0 tps, lat 101.016 ms stddev 124.247 2024-05-24T11:40:45.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:45 smithi053 ceph-mon[47448]: pgmap v79: 97 pgs: 97 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 34 MiB/s wr, 1.56k op/s 2024-05-24T11:40:45.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:45 smithi137 ceph-mon[47907]: pgmap v79: 97 pgs: 97 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 34 MiB/s wr, 1.56k op/s 2024-05-24T11:40:46.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:45 smithi039 ceph-mon[42919]: pgmap v79: 97 pgs: 97 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 34 MiB/s wr, 1.56k op/s 2024-05-24T11:40:47.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:47 smithi053 ceph-mon[47448]: pgmap v80: 97 pgs: 97 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 21 MiB/s wr, 987 op/s 2024-05-24T11:40:47.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:47 smithi137 ceph-mon[47907]: pgmap v80: 97 pgs: 97 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 21 MiB/s wr, 987 op/s 2024-05-24T11:40:48.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:47 smithi039 ceph-mon[42919]: pgmap v80: 97 pgs: 97 active+clean; 2.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 21 MiB/s wr, 987 op/s 2024-05-24T11:40:49.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 85.0 s, 1108.2 tps, lat 88.333 ms stddev 108.161 2024-05-24T11:40:49.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:49 smithi053 ceph-mon[47448]: pgmap v81: 97 pgs: 97 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 32 MiB/s wr, 1.57k op/s 2024-05-24T11:40:49.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:49 smithi137 ceph-mon[47907]: pgmap v81: 97 pgs: 97 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 32 MiB/s wr, 1.57k op/s 2024-05-24T11:40:50.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:49 smithi039 ceph-mon[42919]: pgmap v81: 97 pgs: 97 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 32 MiB/s wr, 1.57k op/s 2024-05-24T11:40:51.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:51 smithi053 ceph-mon[47448]: pgmap v82: 97 pgs: 97 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 21 MiB/s wr, 1.07k op/s 2024-05-24T11:40:51.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:51 smithi137 ceph-mon[47907]: pgmap v82: 97 pgs: 97 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 21 MiB/s wr, 1.07k op/s 2024-05-24T11:40:52.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:51 smithi039 ceph-mon[42919]: pgmap v82: 97 pgs: 97 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 21 MiB/s wr, 1.07k op/s 2024-05-24T11:40:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:53 smithi053 ceph-mon[47448]: pgmap v83: 97 pgs: 97 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 21 MiB/s wr, 1.07k op/s 2024-05-24T11:40:53.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:53 smithi137 ceph-mon[47907]: pgmap v83: 97 pgs: 97 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 21 MiB/s wr, 1.07k op/s 2024-05-24T11:40:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.065+0000 7f1395e43700 1 -- 172.21.15.39:0/628166618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13901010e0 msgr2=0x7f13901014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.065+0000 7f1395e43700 1 --2- 172.21.15.39:0/628166618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13901010e0 0x7f13901014c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1380009990 tx=0x7f1380009ca0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.066+0000 7f1395e43700 1 -- 172.21.15.39:0/628166618 shutdown_connections 2024-05-24T11:40:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.066+0000 7f1395e43700 1 --2- 172.21.15.39:0/628166618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1390106200 0x7f139010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.066+0000 7f1395e43700 1 --2- 172.21.15.39:0/628166618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13901010e0 0x7f13901014c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.066+0000 7f1395e43700 1 --2- 172.21.15.39:0/628166618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1390101a90 0x7f1390105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.066+0000 7f1395e43700 1 -- 172.21.15.39:0/628166618 >> 172.21.15.39:0/628166618 conn(0x7f13900fc940 msgr2=0x7f13900fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.066+0000 7f1395e43700 1 -- 172.21.15.39:0/628166618 shutdown_connections 2024-05-24T11:40:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.066+0000 7f1395e43700 1 -- 172.21.15.39:0/628166618 wait complete. 2024-05-24T11:40:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.067+0000 7f1395e43700 1 Processor -- start 2024-05-24T11:40:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.067+0000 7f1395e43700 1 -- start start 2024-05-24T11:40:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.067+0000 7f1395e43700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13901010e0 0x7f139010db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.067+0000 7f1395e43700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1390106200 0x7f139010ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.067+0000 7f1395e43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f139010b1b0 0x7f139010b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.067+0000 7f1395e43700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1390071ad0 con 0x7f139010b1b0 2024-05-24T11:40:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.067+0000 7f1395e43700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1390071c40 con 0x7f13901010e0 2024-05-24T11:40:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.067+0000 7f1395e43700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1390071db0 con 0x7f1390106200 2024-05-24T11:40:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13901010e0 0x7f139010db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13901010e0 0x7f139010db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:46966/0 (socket says 172.21.15.39:46966) 2024-05-24T11:40:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138f7fe700 1 -- 172.21.15.39:0/542073032 learned_addr learned my addr 172.21.15.39:0/542073032 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138ffff700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f139010b1b0 0x7f139010b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138effd700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1390106200 0x7f139010ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138effd700 1 -- 172.21.15.39:0/542073032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13901010e0 msgr2=0x7f139010db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138effd700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13901010e0 0x7f139010db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138effd700 1 -- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f139010b1b0 msgr2=0x7f139010b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138effd700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f139010b1b0 0x7f139010b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138effd700 1 -- 172.21.15.39:0/542073032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800095f0 con 0x7f1390106200 2024-05-24T11:40:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138ffff700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f139010b1b0 0x7f139010b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:40:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138effd700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1390106200 0x7f139010ac70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f139006a6f0 tx=0x7f138400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138cff9700 1 -- 172.21.15.39:0/542073032 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400f890 con 0x7f1390106200 2024-05-24T11:40:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138cff9700 1 -- 172.21.15.39:0/542073032 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f138400fe80 con 0x7f1390106200 2024-05-24T11:40:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f138cff9700 1 -- 172.21.15.39:0/542073032 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400e3b0 con 0x7f1390106200 2024-05-24T11:40:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.068+0000 7f1395e43700 1 -- 172.21.15.39:0/542073032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1390072090 con 0x7f1390106200 2024-05-24T11:40:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.069+0000 7f1395e43700 1 -- 172.21.15.39:0/542073032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1390072550 con 0x7f1390106200 2024-05-24T11:40:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.071+0000 7f138cff9700 1 -- 172.21.15.39:0/542073032 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f138400f9f0 con 0x7f1390106200 2024-05-24T11:40:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.072+0000 7f138cff9700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f13780a1310 0x7f13780a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.072+0000 7f138cff9700 1 -- 172.21.15.39:0/542073032 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f13840cc990 con 0x7f1390106200 2024-05-24T11:40:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.073+0000 7f138f7fe700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f13780a1310 0x7f13780a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:53 smithi039 ceph-mon[42919]: pgmap v83: 97 pgs: 97 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 21 MiB/s wr, 1.07k op/s 2024-05-24T11:40:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.073+0000 7f13a1ffeb80 1 Processor -- start 2024-05-24T11:40:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.073+0000 7f138f7fe700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f13780a1310 0x7f13780a37d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f13800098e0 tx=0x7f138000b580 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.073+0000 7f13a1ffeb80 1 -- start start 2024-05-24T11:40:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 0x5617516a8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 0x5617516ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516acea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617515ee0f0 con 0x5617516a8140 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617515ebab0 con 0x5617516aca40 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617515e73a0 con 0x5617516a8a60 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 0x5617516ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516acea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:46980/0 (socket says 172.21.15.39:46980) 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138ffff700 1 -- 172.21.15.39:0/886038889 learned_addr learned my addr 172.21.15.39:0/886038889 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138f7fe700 1 --2- 172.21.15.39:0/886038889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 0x5617516a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138effd700 1 -- 172.21.15.39:0/886038889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 msgr2=0x5617516acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138effd700 1 --2- 172.21.15.39:0/886038889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138effd700 1 -- 172.21.15.39:0/886038889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 msgr2=0x5617516a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138effd700 1 --2- 172.21.15.39:0/886038889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 0x5617516a8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138effd700 1 -- 172.21.15.39:0/886038889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561751468350 con 0x5617516a8a60 2024-05-24T11:40:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138ffff700 1 --2- 172.21.15.39:0/886038889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:40:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.074+0000 7f138effd700 1 --2- 172.21.15.39:0/886038889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 0x5617516ac500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13840ca0c0 tx=0x7f13840993e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29607a8954983457 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.075+0000 7f1375ffb700 1 -- 172.21.15.39:0/886038889 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138409a180 con 0x5617516a8a60 2024-05-24T11:40:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.075+0000 7f1375ffb700 1 -- 172.21.15.39:0/886038889 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1384099a80 con 0x5617516a8a60 2024-05-24T11:40:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.075+0000 7f1375ffb700 1 -- 172.21.15.39:0/886038889 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384099f20 con 0x5617516a8a60 2024-05-24T11:40:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.075+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/886038889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 msgr2=0x5617516ac500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.075+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/886038889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 0x5617516ac500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13840ca0c0 tx=0x7f13840993e0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.076+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/886038889 shutdown_connections 2024-05-24T11:40:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.076+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/886038889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.076+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/886038889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 0x5617516a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.076+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/886038889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 0x5617516ac500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.076+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/886038889 >> 172.21.15.39:0/886038889 conn(0x5617515f51b0 msgr2=0x561751691840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.076+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/886038889 shutdown_connections 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.076+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/886038889 wait complete. 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.076+0000 7f13a1ffeb80 1 -- start start 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.076+0000 7f13a1ffeb80 1 Processor -- start 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 0x5617516b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516a8a60 0x5617516b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516aca40 0x5617516b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561751583500 con 0x5617516a8140 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617515831a0 con 0x5617516a8a60 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56175146a640 con 0x5617516aca40 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f13a1ffeb80 10 client.0 _ll_register_callbacks cb 0x56175158cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516aca40 0x5617516b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516aca40 0x5617516b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38610/0 (socket says 172.21.15.39:38610) 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516a8a60 0x5617516b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138ffff700 1 -- 172.21.15.39:0/3005889520 learned_addr learned my addr 172.21.15.39:0/3005889520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138f7fe700 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 0x5617516b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138ffff700 1 -- 172.21.15.39:0/3005889520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516a8a60 msgr2=0x5617516b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138ffff700 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516a8a60 0x5617516b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138ffff700 1 -- 172.21.15.39:0/3005889520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 msgr2=0x5617516b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138ffff700 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 0x5617516b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138ffff700 1 -- 172.21.15.39:0/3005889520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138800a440 con 0x5617516aca40 2024-05-24T11:40:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.077+0000 7f138f7fe700 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 0x5617516b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:40:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.078+0000 7f138ffff700 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516aca40 0x5617516b1cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1388033470 tx=0x7f1388033780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=475790bf6e6325a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.078+0000 7f136effd700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.078+0000 7f136effd700 1 -- 172.21.15.39:0/3005889520 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388038660 con 0x5617516aca40 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.078+0000 7f136effd700 1 -- 172.21.15.39:0/3005889520 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1388006e40 con 0x5617516aca40 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.078+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3005889520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561751540210 con 0x5617516aca40 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.078+0000 7f136effd700 1 -- 172.21.15.39:0/3005889520 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13880387c0 con 0x5617516aca40 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.078+0000 7f13a1ffeb80 10 client.25240 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3005889520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f1390072550 con 0x5617516aca40 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f136effd700 1 -- 172.21.15.39:0/3005889520 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561751540210 con 0x5617516aca40 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f136effd700 1 -- 172.21.15.39:0/3005889520 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1388041930 con 0x5617516aca40 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f13a1ffeb80 10 client.25240 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f13a1ffeb80 10 client.25240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f13a1ffeb80 20 client.25240 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5617516a4270 0x5617516a6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f13a1ffeb80 4 client.25240 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:40:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3005889520 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617516667b0 con 0x5617516a4270 2024-05-24T11:40:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.079+0000 7f138f7fe700 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5617516a4270 0x5617516a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.080+0000 7f138f7fe700 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5617516a4270 0x5617516a6690 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ecd5bc97d5505fe server_cookie=a4e775bd9c2ab79e in_seq=0 out_seq=0 2024-05-24T11:40:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.080+0000 7f136effd700 10 client.25240 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.080+0000 7f136effd700 1 -- 172.21.15.39:0/3005889520 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5617516667b0 con 0x5617516a4270 2024-05-24T11:40:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.080+0000 7f136effd700 10 client.25240 handle_command_reply: tid=0 2024-05-24T11:40:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.080+0000 7f13a1ffeb80 1 client.25240 shutdown 2024-05-24T11:40:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3005889520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516aca40 msgr2=0x5617516b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516aca40 0x5617516b1cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1388033470 tx=0x7f1388033780 comp rx=0 tx=0).stop 2024-05-24T11:40:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3005889520 shutdown_connections 2024-05-24T11:40:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516aca40 0x5617516b1cf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5617516a4270 0x5617516a6690 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8140 0x5617516b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3005889520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516a8a60 0x5617516b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3005889520 >> 172.21.15.39:0/3005889520 conn(0x5617515f51b0 msgr2=0x561751695f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f136effd700 0 client.25240 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3005889520 shutdown_connections 2024-05-24T11:40:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3005889520 wait complete. 2024-05-24T11:40:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.082+0000 7f13a1ffeb80 20 client.25240 trim_cache size 0 max 16384 2024-05-24T11:40:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.110+0000 7f13a1ffeb80 1 Processor -- start 2024-05-24T11:40:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 90.0 s, 1248.4 tps, lat 81.624 ms stddev 75.217 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.110+0000 7f13a1ffeb80 1 -- start start 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.110+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516a8140 0x5617516aa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.110+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 0x5617516a8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.110+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516aca40 0x561751691910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.110+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561751689b90 con 0x5617516aca40 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.110+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617516afb60 con 0x5617516a8140 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.110+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617516ab7b0 con 0x5617516a8a60 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516aca40 0x561751691910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516aca40 0x561751691910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48300/0 (socket says 172.21.15.39:48300) 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138ffff700 1 -- 172.21.15.39:0/472129625 learned_addr learned my addr 172.21.15.39:0/472129625 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138effd700 1 --2- 172.21.15.39:0/472129625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 0x5617516a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138ffff700 1 -- 172.21.15.39:0/472129625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 msgr2=0x5617516a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138ffff700 1 --2- 172.21.15.39:0/472129625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 0x5617516a8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138ffff700 1 -- 172.21.15.39:0/472129625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516a8140 msgr2=0x5617516aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138ffff700 1 --2- 172.21.15.39:0/472129625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516a8140 0x5617516aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138ffff700 1 -- 172.21.15.39:0/472129625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561751414b00 con 0x5617516aca40 2024-05-24T11:40:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.111+0000 7f138ffff700 1 --2- 172.21.15.39:0/472129625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516aca40 0x561751691910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f80 tx=0x7f1388009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd5d7de9948f8ac server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.112+0000 7f13767fc700 1 -- 172.21.15.39:0/472129625 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388009ed0 con 0x5617516aca40 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.112+0000 7f13767fc700 1 -- 172.21.15.39:0/472129625 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1388032040 con 0x5617516aca40 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.112+0000 7f13767fc700 1 -- 172.21.15.39:0/472129625 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13880379a0 con 0x5617516aca40 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.112+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/472129625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516aca40 msgr2=0x561751691910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.112+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/472129625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516aca40 0x561751691910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f80 tx=0x7f1388009d90 comp rx=0 tx=0).stop 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.113+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/472129625 shutdown_connections 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.113+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/472129625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516aca40 0x561751691910 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.113+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/472129625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516a8140 0x5617516aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.113+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/472129625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8a60 0x5617516a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.113+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/472129625 >> 172.21.15.39:0/472129625 conn(0x5617515f51b0 msgr2=0x561751696820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.113+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/472129625 shutdown_connections 2024-05-24T11:40:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.113+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/472129625 wait complete. 2024-05-24T11:40:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.113+0000 7f13a1ffeb80 1 -- start start 2024-05-24T11:40:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f13a1ffeb80 1 Processor -- start 2024-05-24T11:40:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8140 0x5617516b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8a60 0x5617516b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f13a1ffeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617516b9260 con 0x5617516a8a60 2024-05-24T11:40:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617516b2e30 con 0x5617516aca40 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f13a1ffeb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617516b2ff0 con 0x5617516a8140 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f13a1ffeb80 10 client.0 _ll_register_callbacks cb 0x56175158cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f138effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8a60 0x5617516b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f138effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8a60 0x5617516b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48312/0 (socket says 172.21.15.39:48312) 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f138effd700 1 -- 172.21.15.39:0/3410660830 learned_addr learned my addr 172.21.15.39:0/3410660830 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.114+0000 7f138ffff700 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.115+0000 7f138ffff700 1 -- 172.21.15.39:0/3410660830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8140 msgr2=0x5617516b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.115+0000 7f138ffff700 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8140 0x5617516b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.115+0000 7f138ffff700 1 -- 172.21.15.39:0/3410660830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8a60 msgr2=0x5617516b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:40:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.115+0000 7f138ffff700 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8a60 0x5617516b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.115+0000 7f138ffff700 1 -- 172.21.15.39:0/3410660830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1390072550 con 0x5617516aca40 2024-05-24T11:40:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.116+0000 7f138ffff700 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516b1430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1388033470 tx=0x7f13880092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbc8cf514a2613fe server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:40:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.116+0000 7f136effd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:40:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.116+0000 7f136effd700 1 -- 172.21.15.39:0/3410660830 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388041910 con 0x5617516aca40 2024-05-24T11:40:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.117+0000 7f136effd700 1 -- 172.21.15.39:0/3410660830 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1388051030 con 0x5617516aca40 2024-05-24T11:40:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.117+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3410660830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1384099a80 con 0x5617516aca40 2024-05-24T11:40:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.117+0000 7f136effd700 1 -- 172.21.15.39:0/3410660830 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1388037600 con 0x5617516aca40 2024-05-24T11:40:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.117+0000 7f13a1ffeb80 10 client.25331 fetch_fsmap learned FSMap version 27 2024-05-24T11:40:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.117+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3410660830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5617515ad500 con 0x5617516aca40 2024-05-24T11:40:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.118+0000 7f136effd700 1 -- 172.21.15.39:0/3410660830 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1384099a80 con 0x5617516aca40 2024-05-24T11:40:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.118+0000 7f136effd700 1 -- 172.21.15.39:0/3410660830 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f138806a660 con 0x5617516aca40 2024-05-24T11:40:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.119+0000 7f13a1ffeb80 10 client.25331 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:40:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.119+0000 7f13a1ffeb80 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:40:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.119+0000 7f13a1ffeb80 20 client.25331 populate_metadata read hostname 'smithi039' 2024-05-24T11:40:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.119+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5617516b4410 0x5617517090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:40:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.119+0000 7f13a1ffeb80 4 client.25331 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:40:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.119+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3410660830 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617515dfe60 con 0x5617516b4410 2024-05-24T11:40:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.119+0000 7f138f7fe700 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5617516b4410 0x5617517090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:40:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.119+0000 7f138f7fe700 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5617516b4410 0x5617517090c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a6abe3ea2375317 server_cookie=53d8cf002e9d467 in_seq=0 out_seq=0 2024-05-24T11:40:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:54.119+0000 7f136effd700 10 client.25331 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:55.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:55 smithi053 ceph-mon[47448]: pgmap v84: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 30 MiB/s wr, 1.65k op/s 2024-05-24T11:40:55.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:55 smithi137 ceph-mon[47907]: pgmap v84: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 30 MiB/s wr, 1.65k op/s 2024-05-24T11:40:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.034+0000 7f136effd700 1 -- 172.21.15.39:0/3410660830 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617515dfe60 con 0x5617516b4410 2024-05-24T11:40:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.034+0000 7f136effd700 10 client.25331 handle_command_reply: tid=0 2024-05-24T11:40:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.034+0000 7f13a1ffeb80 1 client.25331 shutdown 2024-05-24T11:40:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.034+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3410660830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 msgr2=0x5617516b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.034+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516b1430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1388033470 tx=0x7f13880092f0 comp rx=0 tx=0).stop 2024-05-24T11:40:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.036+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3410660830 shutdown_connections 2024-05-24T11:40:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.036+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5617516a8140 0x5617516b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.036+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5617516b4410 0x5617517090c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.036+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617516aca40 0x5617516b1430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.036+0000 7f13a1ffeb80 1 --2- 172.21.15.39:0/3410660830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5617516a8a60 0x5617516b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.036+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3410660830 >> 172.21.15.39:0/3410660830 conn(0x5617515f51b0 msgr2=0x5617516a6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.036+0000 7f136effd700 0 client.25331 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:40:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.037+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3410660830 shutdown_connections 2024-05-24T11:40:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.037+0000 7f13a1ffeb80 1 -- 172.21.15.39:0/3410660830 wait complete. 2024-05-24T11:40:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.037+0000 7f13a1ffeb80 20 client.25331 trim_cache size 0 max 16384 2024-05-24T11:40:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:40:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:40:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:40:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:40:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 -- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f13780a1310 msgr2=0x7f13780a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f13780a1310 0x7f13780a37d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f13800098e0 tx=0x7f138000b580 comp rx=0 tx=0).stop 2024-05-24T11:40:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 -- 172.21.15.39:0/542073032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1390106200 msgr2=0x7f139010ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:40:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1390106200 0x7f139010ac70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f139006a6f0 tx=0x7f138400dc30 comp rx=0 tx=0).stop 2024-05-24T11:40:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 -- 172.21.15.39:0/542073032 shutdown_connections 2024-05-24T11:40:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f13780a1310 0x7f13780a37d0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f13800098e0 tx=0x7f138000b580 comp rx=0 tx=0).stop 2024-05-24T11:40:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f139010b1b0 0x7f139010b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13901010e0 0x7f139010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 --2- 172.21.15.39:0/542073032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1390106200 0x7f139010ac70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:40:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.038+0000 7f1395e43700 1 -- 172.21.15.39:0/542073032 >> 172.21.15.39:0/542073032 conn(0x7f13900fc940 msgr2=0x7f13900fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:40:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.040+0000 7f1395e43700 1 -- 172.21.15.39:0/542073032 shutdown_connections 2024-05-24T11:40:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:40:56.040+0000 7f1395e43700 1 -- 172.21.15.39:0/542073032 wait complete. 2024-05-24T11:40:56.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:55 smithi039 ceph-mon[42919]: pgmap v84: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 30 MiB/s wr, 1.65k op/s 2024-05-24T11:40:57.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:57 smithi053 ceph-mon[47448]: pgmap v85: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 20 MiB/s wr, 1.17k op/s 2024-05-24T11:40:57.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:57 smithi137 ceph-mon[47907]: pgmap v85: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 20 MiB/s wr, 1.17k op/s 2024-05-24T11:40:58.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:57 smithi039 ceph-mon[42919]: pgmap v85: 97 pgs: 97 active+clean; 2.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 20 MiB/s wr, 1.17k op/s 2024-05-24T11:40:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 95.0 s, 1356.6 tps, lat 73.868 ms stddev 19.620 2024-05-24T11:40:59.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:40:59 smithi053 ceph-mon[47448]: pgmap v86: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 28 MiB/s wr, 1.82k op/s 2024-05-24T11:40:59.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:40:59 smithi137 ceph-mon[47907]: pgmap v86: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 28 MiB/s wr, 1.82k op/s 2024-05-24T11:41:00.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:40:59 smithi039 ceph-mon[42919]: pgmap v86: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 28 MiB/s wr, 1.82k op/s 2024-05-24T11:41:01.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:01 smithi053 ceph-mon[47448]: pgmap v87: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-05-24T11:41:01.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:01 smithi137 ceph-mon[47907]: pgmap v87: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-05-24T11:41:02.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:01 smithi039 ceph-mon[42919]: pgmap v87: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-05-24T11:41:03.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:03 smithi053 ceph-mon[47448]: pgmap v88: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-05-24T11:41:03.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:03 smithi137 ceph-mon[47907]: pgmap v88: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-05-24T11:41:04.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:03 smithi039 ceph-mon[42919]: pgmap v88: 97 pgs: 97 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2024-05-24T11:41:04.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 100.0 s, 1267.2 tps, lat 78.967 ms stddev 24.520 2024-05-24T11:41:05.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:05 smithi053 ceph-mon[47448]: pgmap v89: 97 pgs: 97 active+clean; 2.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 25 MiB/s wr, 1.87k op/s 2024-05-24T11:41:05.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:05 smithi137 ceph-mon[47907]: pgmap v89: 97 pgs: 97 active+clean; 2.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 25 MiB/s wr, 1.87k op/s 2024-05-24T11:41:06.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:05 smithi039 ceph-mon[42919]: pgmap v89: 97 pgs: 97 active+clean; 2.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 25 MiB/s wr, 1.87k op/s 2024-05-24T11:41:07.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:07 smithi053 ceph-mon[47448]: pgmap v90: 97 pgs: 97 active+clean; 2.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:41:07.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:07 smithi137 ceph-mon[47907]: pgmap v90: 97 pgs: 97 active+clean; 2.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:41:08.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:07 smithi039 ceph-mon[42919]: pgmap v90: 97 pgs: 97 active+clean; 2.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:41:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 105.0 s, 1400.4 tps, lat 71.113 ms stddev 18.821 2024-05-24T11:41:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.209+0000 7fb039a0c700 1 -- 172.21.15.39:0/210526780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 msgr2=0x7fb03410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.209+0000 7fb039a0c700 1 --2- 172.21.15.39:0/210526780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 0x7fb03410ffb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009990 tx=0x7fb02c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.209+0000 7fb039a0c700 1 -- 172.21.15.39:0/210526780 shutdown_connections 2024-05-24T11:41:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.209+0000 7fb039a0c700 1 --2- 172.21.15.39:0/210526780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 0x7fb03410ffb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.209+0000 7fb039a0c700 1 --2- 172.21.15.39:0/210526780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb034103310 0x7fb0341036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.209+0000 7fb039a0c700 1 --2- 172.21.15.39:0/210526780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb034103cc0 0x7fb034107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.209+0000 7fb039a0c700 1 -- 172.21.15.39:0/210526780 >> 172.21.15.39:0/210526780 conn(0x7fb0340feb50 msgr2=0x7fb034100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.209+0000 7fb039a0c700 1 -- 172.21.15.39:0/210526780 shutdown_connections 2024-05-24T11:41:09.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.209+0000 7fb039a0c700 1 -- 172.21.15.39:0/210526780 wait complete. 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.210+0000 7fb039a0c700 1 Processor -- start 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.210+0000 7fb039a0c700 1 -- start start 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb039a0c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb034103310 0x7fb03410e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb039a0c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 0x7fb03410e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb039a0c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb03410ec90 0x7fb0341a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb039a0c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0341a9eb0 con 0x7fb034103310 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb039a0c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0341a9ff0 con 0x7fb034108430 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb039a0c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0341aa130 con 0x7fb03410ec90 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 0x7fb03410e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 0x7fb03410e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:55666/0 (socket says 172.21.15.39:55666) 2024-05-24T11:41:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb02a7fc700 1 -- 172.21.15.39:0/1404899159 learned_addr learned my addr 172.21.15.39:0/1404899159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb0337fe700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb03410ec90 0x7fb0341a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb032ffd700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb034103310 0x7fb03410e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.211+0000 7fb02a7fc700 1 -- 172.21.15.39:0/1404899159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb03410ec90 msgr2=0x7fb0341a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb02a7fc700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb03410ec90 0x7fb0341a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb02a7fc700 1 -- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb034103310 msgr2=0x7fb03410e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb02a7fc700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb034103310 0x7fb03410e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb02a7fc700 1 -- 172.21.15.39:0/1404899159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0095f0 con 0x7fb034108430 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb0337fe700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb03410ec90 0x7fb0341a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb02a7fc700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 0x7fb03410e750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00d950 tx=0x7fb01c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb038a0a700 1 -- 172.21.15.39:0/1404899159 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c004020 con 0x7fb034108430 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb038a0a700 1 -- 172.21.15.39:0/1404899159 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb01c007620 con 0x7fb034108430 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb038a0a700 1 -- 172.21.15.39:0/1404899159 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0053a0 con 0x7fb034108430 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb039a0c700 1 -- 172.21.15.39:0/1404899159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0341aa3c0 con 0x7fb034108430 2024-05-24T11:41:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.212+0000 7fb039a0c700 1 -- 172.21.15.39:0/1404899159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0341aa880 con 0x7fb034108430 2024-05-24T11:41:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.216+0000 7fb038a0a700 1 -- 172.21.15.39:0/1404899159 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb01c004750 con 0x7fb034108430 2024-05-24T11:41:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.218+0000 7fb038a0a700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb0200a1310 0x7fb0200a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.218+0000 7fb038a0a700 1 -- 172.21.15.39:0/1404899159 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb01c0cc4b0 con 0x7fb034108430 2024-05-24T11:41:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.218+0000 7fb032ffd700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb0200a1310 0x7fb0200a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.219+0000 7fb045bc7b80 1 Processor -- start 2024-05-24T11:41:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.219+0000 7fb045bc7b80 1 -- start start 2024-05-24T11:41:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.219+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba583950 0x5625ba583d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.219+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba57df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.219+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 0x5625ba57e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.219+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba4d50f0 con 0x5625ba57e480 2024-05-24T11:41:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.219+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba4d2ab0 con 0x5625ba584270 2024-05-24T11:41:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.219+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba4ce3a0 con 0x5625ba583950 2024-05-24T11:41:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba57df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba583950 0x5625ba583d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba583950 0x5625ba583d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:46644/0 (socket says 172.21.15.39:46644) 2024-05-24T11:41:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 -- 172.21.15.39:0/1710978223 learned_addr learned my addr 172.21.15.39:0/1710978223 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba57df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:55670/0 (socket says 172.21.15.39:55670) 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb0200a1310 0x7fb0200a37d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb02400f540 tx=0x7fb02400b040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb0337fe700 1 --2- 172.21.15.39:0/1710978223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 0x5625ba57e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 -- 172.21.15.39:0/1710978223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 msgr2=0x5625ba57df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 --2- 172.21.15.39:0/1710978223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba57df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 -- 172.21.15.39:0/1710978223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 msgr2=0x5625ba57e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 --2- 172.21.15.39:0/1710978223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 0x5625ba57e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 -- 172.21.15.39:0/1710978223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625ba34f350 con 0x5625ba583950 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb032ffd700 1 --2- 172.21.15.39:0/1710978223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba583950 0x5625ba583d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb024003d20 tx=0x7fb024014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=805145952e1bab3c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.220+0000 7fb029ffb700 1 -- 172.21.15.39:0/1710978223 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02400e040 con 0x5625ba583950 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.221+0000 7fb029ffb700 1 -- 172.21.15.39:0/1710978223 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02400f040 con 0x5625ba583950 2024-05-24T11:41:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.221+0000 7fb029ffb700 1 -- 172.21.15.39:0/1710978223 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024008de0 con 0x5625ba583950 2024-05-24T11:41:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.221+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1710978223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba583950 msgr2=0x5625ba583d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.221+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1710978223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba583950 0x5625ba583d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb024003d20 tx=0x7fb024014000 comp rx=0 tx=0).stop 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.221+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1710978223 shutdown_connections 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.221+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1710978223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba583950 0x5625ba583d30 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb024003d20 tx=0x7fb024014000 comp rx=0 tx=0).stop 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.221+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1710978223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 0x5625ba57e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.221+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1710978223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba57df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.221+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1710978223 >> 172.21.15.39:0/1710978223 conn(0x5625ba4dc1b0 msgr2=0x5625ba578840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.222+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1710978223 shutdown_connections 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.222+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1710978223 wait complete. 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.222+0000 7fb045bc7b80 1 -- start start 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb045bc7b80 1 Processor -- start 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 0x5625ba5877d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 0x5625ba587d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba588250 0x5625ba5a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba46a500 con 0x5625ba57e480 2024-05-24T11:41:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba46a1a0 con 0x5625ba588250 2024-05-24T11:41:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba351640 con 0x5625ba584270 2024-05-24T11:41:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb045bc7b80 10 client.0 _ll_register_callbacks cb 0x5625ba473ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb0337fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba588250 0x5625ba5a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb0337fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba588250 0x5625ba5a4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:55678/0 (socket says 172.21.15.39:55678) 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb0337fe700 1 -- 172.21.15.39:0/2463038905 learned_addr learned my addr 172.21.15.39:0/2463038905 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb032ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 0x5625ba5877d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.223+0000 7fb02a7fc700 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 0x5625ba587d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb0337fe700 1 -- 172.21.15.39:0/2463038905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 msgr2=0x5625ba587d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb0337fe700 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 0x5625ba587d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb0337fe700 1 -- 172.21.15.39:0/2463038905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 msgr2=0x5625ba5877d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb0337fe700 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 0x5625ba5877d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb0337fe700 1 -- 172.21.15.39:0/2463038905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb024003a00 con 0x5625ba588250 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb0337fe700 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba588250 0x5625ba5a4d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb02c005270 tx=0x7fb02c0052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad3c8a160e6769b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb012ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb012ffd700 1 -- 172.21.15.39:0/2463038905 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c028070 con 0x5625ba588250 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb045bc7b80 1 -- 172.21.15.39:0/2463038905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625ba427210 con 0x5625ba588250 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb012ffd700 1 -- 172.21.15.39:0/2463038905 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb02c004710 con 0x5625ba588250 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb012ffd700 1 -- 172.21.15.39:0/2463038905 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb02c01d030 con 0x5625ba588250 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb045bc7b80 10 client.25343 fetch_fsmap learned FSMap version 27 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.224+0000 7fb045bc7b80 1 -- 172.21.15.39:0/2463038905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625ba2fbb00 con 0x5625ba588250 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.225+0000 7fb012ffd700 1 -- 172.21.15.39:0/2463038905 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5625ba427210 con 0x5625ba588250 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.225+0000 7fb012ffd700 1 -- 172.21.15.39:0/2463038905 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb02c02b020 con 0x5625ba588250 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.225+0000 7fb045bc7b80 10 client.25343 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.225+0000 7fb045bc7b80 10 client.25343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:41:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.225+0000 7fb045bc7b80 20 client.25343 populate_metadata read hostname 'smithi039' 2024-05-24T11:41:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.225+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5625ba5a92f0 0x5625ba5ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.225+0000 7fb045bc7b80 4 client.25343 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:41:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.225+0000 7fb045bc7b80 1 -- 172.21.15.39:0/2463038905 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625ba54d7b0 con 0x5625ba5a92f0 2024-05-24T11:41:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.225+0000 7fb032ffd700 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5625ba5a92f0 0x5625ba5ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.226+0000 7fb012ffd700 10 client.25343 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.226+0000 7fb032ffd700 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5625ba5a92f0 0x5625ba5ab710 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f9c1ff9477fb5c3 server_cookie=87d4b98f9e7468da in_seq=0 out_seq=0 2024-05-24T11:41:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb012ffd700 1 -- 172.21.15.39:0/2463038905 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625ba54d7b0 con 0x5625ba5a92f0 2024-05-24T11:41:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb012ffd700 10 client.25343 handle_command_reply: tid=0 2024-05-24T11:41:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 client.25343 shutdown 2024-05-24T11:41:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 -- 172.21.15.39:0/2463038905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba588250 msgr2=0x5625ba5a4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba588250 0x5625ba5a4d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb02c005270 tx=0x7fb02c0052a0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 -- 172.21.15.39:0/2463038905 shutdown_connections 2024-05-24T11:41:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5625ba5a92f0 0x5625ba5ab710 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba588250 0x5625ba5a4d00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba57e480 0x5625ba5877d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb012ffd700 0 client.25343 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/2463038905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 0x5625ba587d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 -- 172.21.15.39:0/2463038905 >> 172.21.15.39:0/2463038905 conn(0x5625ba4dc1b0 msgr2=0x5625ba57cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 -- 172.21.15.39:0/2463038905 shutdown_connections 2024-05-24T11:41:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.227+0000 7fb045bc7b80 1 -- 172.21.15.39:0/2463038905 wait complete. 2024-05-24T11:41:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.228+0000 7fb045bc7b80 20 client.25343 trim_cache size 0 max 16384 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.249+0000 7fb045bc7b80 1 Processor -- start 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.249+0000 7fb045bc7b80 1 -- start start 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.249+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba57e480 0x5625ba5a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.249+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 0x5625ba578910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.249+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba5b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.249+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba588070 con 0x5625ba588250 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.249+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba582360 con 0x5625ba57e480 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.249+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba58a5a0 con 0x5625ba584270 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb0337fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba5b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb0337fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba5b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60404/0 (socket says 172.21.15.39:60404) 2024-05-24T11:41:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb0337fe700 1 -- 172.21.15.39:0/267599819 learned_addr learned my addr 172.21.15.39:0/267599819 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb02a7fc700 1 --2- 172.21.15.39:0/267599819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 0x5625ba578910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb032ffd700 1 --2- 172.21.15.39:0/267599819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba57e480 0x5625ba5a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb0337fe700 1 -- 172.21.15.39:0/267599819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 msgr2=0x5625ba578910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb0337fe700 1 --2- 172.21.15.39:0/267599819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 0x5625ba578910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb0337fe700 1 -- 172.21.15.39:0/267599819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba57e480 msgr2=0x5625ba5a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb0337fe700 1 --2- 172.21.15.39:0/267599819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba57e480 0x5625ba5a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.250+0000 7fb0337fe700 1 -- 172.21.15.39:0/267599819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625ba494500 con 0x5625ba588250 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.251+0000 7fb0337fe700 1 --2- 172.21.15.39:0/267599819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba5b0070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009ce0 tx=0x7fb02c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d28baff857894fb server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.251+0000 7fb02affd700 1 -- 172.21.15.39:0/267599819 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c028040 con 0x5625ba588250 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.251+0000 7fb02affd700 1 -- 172.21.15.39:0/267599819 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02c01e040 con 0x5625ba588250 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.251+0000 7fb02affd700 1 -- 172.21.15.39:0/267599819 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c01d040 con 0x5625ba588250 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.252+0000 7fb045bc7b80 1 -- 172.21.15.39:0/267599819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 msgr2=0x5625ba5b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.252+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/267599819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba5b0070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009ce0 tx=0x7fb02c009310 comp rx=0 tx=0).stop 2024-05-24T11:41:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.252+0000 7fb045bc7b80 1 -- 172.21.15.39:0/267599819 shutdown_connections 2024-05-24T11:41:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.252+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/267599819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba5b0070 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.252+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/267599819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba57e480 0x5625ba5a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.252+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/267599819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba584270 0x5625ba578910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.252+0000 7fb045bc7b80 1 -- 172.21.15.39:0/267599819 >> 172.21.15.39:0/267599819 conn(0x5625ba4dc1b0 msgr2=0x5625ba574e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.253+0000 7fb045bc7b80 1 -- 172.21.15.39:0/267599819 shutdown_connections 2024-05-24T11:41:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.253+0000 7fb045bc7b80 1 -- 172.21.15.39:0/267599819 wait complete. 2024-05-24T11:41:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.253+0000 7fb045bc7b80 1 -- start start 2024-05-24T11:41:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.253+0000 7fb045bc7b80 1 Processor -- start 2024-05-24T11:41:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba57e480 0x5625ba59ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba59f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb045bc7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba59f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba5fe7f0 con 0x5625ba588250 2024-05-24T11:41:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba5fe930 con 0x5625ba584270 2024-05-24T11:41:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb045bc7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625ba5fea70 con 0x5625ba57e480 2024-05-24T11:41:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb045bc7b80 10 client.0 _ll_register_callbacks cb 0x5625ba473ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:41:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba59f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb032ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba57e480 0x5625ba59ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb02a7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba59f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:55692/0 (socket says 172.21.15.39:55692) 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb02a7fc700 1 -- 172.21.15.39:0/1881953879 learned_addr learned my addr 172.21.15.39:0/1881953879 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb0337fe700 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba59f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb032ffd700 1 -- 172.21.15.39:0/1881953879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 msgr2=0x5625ba59f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb032ffd700 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba59f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb032ffd700 1 -- 172.21.15.39:0/1881953879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 msgr2=0x5625ba59f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb032ffd700 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba59f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.254+0000 7fb032ffd700 1 -- 172.21.15.39:0/1881953879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625ba2fbb00 con 0x5625ba57e480 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.255+0000 7fb032ffd700 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba57e480 0x5625ba59ee40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb024009000 tx=0x7fb0240085c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e448e8ce4c3871d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.255+0000 7fb012ffd700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.255+0000 7fb012ffd700 1 -- 172.21.15.39:0/1881953879 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02400e040 con 0x5625ba57e480 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.255+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1881953879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb02400f040 con 0x5625ba57e480 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.255+0000 7fb012ffd700 1 -- 172.21.15.39:0/1881953879 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb024008de0 con 0x5625ba57e480 2024-05-24T11:41:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.255+0000 7fb012ffd700 1 -- 172.21.15.39:0/1881953879 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0240087e0 con 0x5625ba57e480 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.255+0000 7fb045bc7b80 10 client.25258 fetch_fsmap learned FSMap version 27 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.255+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1881953879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625ba353c70 con 0x5625ba57e480 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.256+0000 7fb012ffd700 1 -- 172.21.15.39:0/1881953879 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02400f040 con 0x5625ba57e480 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.256+0000 7fb012ffd700 1 -- 172.21.15.39:0/1881953879 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb024004c40 con 0x5625ba57e480 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.256+0000 7fb045bc7b80 10 client.25258 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.256+0000 7fb045bc7b80 10 client.25258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.256+0000 7fb045bc7b80 20 client.25258 populate_metadata read hostname 'smithi039' 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.256+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5625ba600160 0x5625ba602580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.256+0000 7fb045bc7b80 4 client.25258 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:41:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.256+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1881953879 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5625ba4c6e60 con 0x5625ba600160 2024-05-24T11:41:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.257+0000 7fb02a7fc700 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5625ba600160 0x5625ba602580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.257+0000 7fb02a7fc700 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5625ba600160 0x5625ba602580 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=997743da6434d59f server_cookie=242f16668e13958b in_seq=0 out_seq=0 2024-05-24T11:41:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.257+0000 7fb012ffd700 10 client.25258 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.282+0000 7fb012ffd700 1 -- 172.21.15.39:0/1881953879 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5625ba4c6e60 con 0x5625ba600160 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.282+0000 7fb012ffd700 10 client.25258 handle_command_reply: tid=0 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.282+0000 7fb045bc7b80 1 client.25258 shutdown 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1881953879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba57e480 msgr2=0x5625ba59ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba57e480 0x5625ba59ee40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb024009000 tx=0x7fb0240085c0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1881953879 shutdown_connections 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625ba57e480 0x5625ba59ee40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5625ba600160 0x5625ba602580 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625ba588250 0x5625ba59f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 --2- 172.21.15.39:0/1881953879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5625ba584270 0x5625ba59f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1881953879 >> 172.21.15.39:0/1881953879 conn(0x5625ba4dc1b0 msgr2=0x5625ba5b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb012ffd700 0 client.25258 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1881953879 shutdown_connections 2024-05-24T11:41:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 1 -- 172.21.15.39:0/1881953879 wait complete. 2024-05-24T11:41:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb045bc7b80 20 client.25258 trim_cache size 0 max 16384 2024-05-24T11:41:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:41:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:41:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:41:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb039a0c700 1 -- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb0200a1310 msgr2=0x7fb0200a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb039a0c700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb0200a1310 0x7fb0200a37d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb02400f540 tx=0x7fb02400b040 comp rx=0 tx=0).stop 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb039a0c700 1 -- 172.21.15.39:0/1404899159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 msgr2=0x7fb03410e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.283+0000 7fb039a0c700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 0x7fb03410e750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00d950 tx=0x7fb01c00dc60 comp rx=0 tx=0).stop 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.284+0000 7fb039a0c700 1 -- 172.21.15.39:0/1404899159 shutdown_connections 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.284+0000 7fb039a0c700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb03410ec90 0x7fb0341a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.284+0000 7fb039a0c700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb0200a1310 0x7fb0200a37d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.284+0000 7fb039a0c700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb034103310 0x7fb03410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.284+0000 7fb039a0c700 1 --2- 172.21.15.39:0/1404899159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb034108430 0x7fb03410e750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.284+0000 7fb039a0c700 1 -- 172.21.15.39:0/1404899159 >> 172.21.15.39:0/1404899159 conn(0x7fb0340feb50 msgr2=0x7fb0340fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.285+0000 7fb039a0c700 1 -- 172.21.15.39:0/1404899159 shutdown_connections 2024-05-24T11:41:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:09.285+0000 7fb039a0c700 1 -- 172.21.15.39:0/1404899159 wait complete. 2024-05-24T11:41:09.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:09 smithi053 ceph-mon[47448]: pgmap v91: 97 pgs: 97 active+clean; 2.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 23 MiB/s wr, 1.93k op/s 2024-05-24T11:41:09.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:09 smithi137 ceph-mon[47907]: pgmap v91: 97 pgs: 97 active+clean; 2.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 23 MiB/s wr, 1.93k op/s 2024-05-24T11:41:10.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:09 smithi039 ceph-mon[42919]: pgmap v91: 97 pgs: 97 active+clean; 2.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 23 MiB/s wr, 1.93k op/s 2024-05-24T11:41:11.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:11 smithi053 ceph-mon[47448]: pgmap v92: 97 pgs: 97 active+clean; 2.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2024-05-24T11:41:11.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:11 smithi137 ceph-mon[47907]: pgmap v92: 97 pgs: 97 active+clean; 2.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2024-05-24T11:41:12.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:11 smithi039 ceph-mon[42919]: pgmap v92: 97 pgs: 97 active+clean; 2.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2024-05-24T11:41:13.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:13 smithi053 ceph-mon[47448]: pgmap v93: 97 pgs: 97 active+clean; 2.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2024-05-24T11:41:13.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:13 smithi137 ceph-mon[47907]: pgmap v93: 97 pgs: 97 active+clean; 2.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2024-05-24T11:41:14.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:13 smithi039 ceph-mon[42919]: pgmap v93: 97 pgs: 97 active+clean; 2.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2024-05-24T11:41:14.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 110.0 s, 1320.8 tps, lat 75.885 ms stddev 20.928 2024-05-24T11:41:15.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:15 smithi053 ceph-mon[47448]: pgmap v94: 97 pgs: 97 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 22 MiB/s wr, 1.90k op/s 2024-05-24T11:41:15.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:15 smithi137 ceph-mon[47907]: pgmap v94: 97 pgs: 97 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 22 MiB/s wr, 1.90k op/s 2024-05-24T11:41:16.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:15 smithi039 ceph-mon[42919]: pgmap v94: 97 pgs: 97 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 22 MiB/s wr, 1.90k op/s 2024-05-24T11:41:17.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:17 smithi053 ceph-mon[47448]: pgmap v95: 97 pgs: 97 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:41:17.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:17 smithi137 ceph-mon[47907]: pgmap v95: 97 pgs: 97 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:41:18.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:17 smithi039 ceph-mon[42919]: pgmap v95: 97 pgs: 97 active+clean; 2.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:41:18.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:18 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:41:18.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:18 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:41:18.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:18 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:41:18.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:18 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:41:18.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:18 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:41:18.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:18 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:41:18.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:18 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:41:18.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:18 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:41:19.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:18 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:41:19.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:18 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:41:19.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:18 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:41:19.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:18 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:41:19.118 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 115.0 s, 1244.9 tps, lat 80.479 ms stddev 23.790 2024-05-24T11:41:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:19 smithi053 ceph-mon[47448]: pgmap v96: 97 pgs: 97 active+clean; 2.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 21 MiB/s wr, 1.88k op/s 2024-05-24T11:41:19.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:19 smithi137 ceph-mon[47907]: pgmap v96: 97 pgs: 97 active+clean; 2.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 21 MiB/s wr, 1.88k op/s 2024-05-24T11:41:20.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:19 smithi039 ceph-mon[42919]: pgmap v96: 97 pgs: 97 active+clean; 2.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 21 MiB/s wr, 1.88k op/s 2024-05-24T11:41:21.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:21 smithi053 ceph-mon[47448]: pgmap v97: 97 pgs: 97 active+clean; 2.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 1.23k op/s 2024-05-24T11:41:21.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:21 smithi137 ceph-mon[47907]: pgmap v97: 97 pgs: 97 active+clean; 2.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 1.23k op/s 2024-05-24T11:41:22.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:21 smithi039 ceph-mon[42919]: pgmap v97: 97 pgs: 97 active+clean; 2.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 1.23k op/s 2024-05-24T11:41:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.458+0000 7fe0a64d2700 1 -- 172.21.15.39:0/207960241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106f10 msgr2=0x7fe0a01072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.458+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/207960241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106f10 0x7fe0a01072f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0900099c0 tx=0x7fe090009cd0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.458+0000 7fe0a64d2700 1 -- 172.21.15.39:0/207960241 shutdown_connections 2024-05-24T11:41:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.458+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/207960241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a0069ff0 0x7fe0a010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.458+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/207960241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106f10 0x7fe0a01072f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.458+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/207960241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a0069630 0x7fe0a0069ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.458+0000 7fe0a64d2700 1 -- 172.21.15.39:0/207960241 >> 172.21.15.39:0/207960241 conn(0x7fe0a0076c10 msgr2=0x7fe0a0077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.458+0000 7fe0a64d2700 1 -- 172.21.15.39:0/207960241 shutdown_connections 2024-05-24T11:41:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.458+0000 7fe0a64d2700 1 -- 172.21.15.39:0/207960241 wait complete. 2024-05-24T11:41:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.459+0000 7fe0a64d2700 1 Processor -- start 2024-05-24T11:41:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe0a64d2700 1 -- start start 2024-05-24T11:41:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe0a64d2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a0069630 0x7fe0a0100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe0a64d2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a0069ff0 0x7fe0a01005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe0a64d2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106f10 0x7fe0a0100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe0a64d2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00719b0 con 0x7fe0a0069ff0 2024-05-24T11:41:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe0a64d2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a0071af0 con 0x7fe0a0106f10 2024-05-24T11:41:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe0a64d2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a0071c60 con 0x7fe0a0069630 2024-05-24T11:41:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe09f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a0069ff0 0x7fe0a01005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe09f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a0069ff0 0x7fe0a01005c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41616/0 (socket says 172.21.15.39:41616) 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.460+0000 7fe09f7fe700 1 -- 172.21.15.39:0/223582811 learned_addr learned my addr 172.21.15.39:0/223582811 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.461+0000 7fe09ffff700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a0069630 0x7fe0a0100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.461+0000 7fe09f7fe700 1 -- 172.21.15.39:0/223582811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a0069630 msgr2=0x7fe0a0100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.461+0000 7fe09f7fe700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a0069630 0x7fe0a0100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.461+0000 7fe09f7fe700 1 -- 172.21.15.39:0/223582811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106f10 msgr2=0x7fe0a0100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.461+0000 7fe09f7fe700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106f10 0x7fe0a0100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.461+0000 7fe09f7fe700 1 -- 172.21.15.39:0/223582811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0900095f0 con 0x7fe0a0069ff0 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.461+0000 7fe09f7fe700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a0069ff0 0x7fe0a01005c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe09400e9a0 tx=0x7fe09400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.461+0000 7fe09d7fa700 1 -- 172.21.15.39:0/223582811 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094009790 con 0x7fe0a0069ff0 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.462+0000 7fe0a64d2700 1 -- 172.21.15.39:0/223582811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a0071f40 con 0x7fe0a0069ff0 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.462+0000 7fe09d7fa700 1 -- 172.21.15.39:0/223582811 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe09400cbd0 con 0x7fe0a0069ff0 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.462+0000 7fe09d7fa700 1 -- 172.21.15.39:0/223582811 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094018680 con 0x7fe0a0069ff0 2024-05-24T11:41:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.462+0000 7fe0a64d2700 1 -- 172.21.15.39:0/223582811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a00724a0 con 0x7fe0a0069ff0 2024-05-24T11:41:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.466+0000 7fe0b268db80 1 Processor -- start 2024-05-24T11:41:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0b268db80 1 -- start start 2024-05-24T11:41:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb798f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7998a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0b268db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb6f00f0 con 0x5609cb79e950 2024-05-24T11:41:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0b268db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb6edab0 con 0x5609cb799480 2024-05-24T11:41:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0b268db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb6e93a0 con 0x5609cb79f270 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0a4a6f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0a4a6f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41620/0 (socket says 172.21.15.39:41620) 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.467+0000 7fe0a4a6f700 1 -- 172.21.15.39:0/2974811512 learned_addr learned my addr 172.21.15.39:0/2974811512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe09ffff700 1 --2- 172.21.15.39:0/2974811512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb798f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe09d7fa700 1 -- 172.21.15.39:0/223582811 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe09400cd40 con 0x7fe0a0069ff0 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe09ffff700 1 -- 172.21.15.39:0/2974811512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 msgr2=0x5609cb7998a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe09ffff700 1 --2- 172.21.15.39:0/2974811512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7998a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe09ffff700 1 -- 172.21.15.39:0/2974811512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 msgr2=0x5609cb79ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe09ffff700 1 --2- 172.21.15.39:0/2974811512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe09ffff700 1 -- 172.21.15.39:0/2974811512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609cb56a350 con 0x5609cb79f270 2024-05-24T11:41:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe09ffff700 1 --2- 172.21.15.39:0/2974811512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb798f40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe090009990 tx=0x7fe0900056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fc1625d033d5e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe0867fc700 1 -- 172.21.15.39:0/2974811512 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09001c050 con 0x5609cb79f270 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe09d7fa700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe0880a1310 0x7fe0880a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe0867fc700 1 -- 172.21.15.39:0/2974811512 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe09001d040 con 0x5609cb79f270 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.468+0000 7fe0867fc700 1 -- 172.21.15.39:0/2974811512 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0900038e0 con 0x5609cb79f270 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.469+0000 7fe0a4a6f700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe0880a1310 0x7fe0880a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.469+0000 7fe09d7fa700 1 -- 172.21.15.39:0/223582811 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe094014070 con 0x7fe0a0069ff0 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.469+0000 7fe0b268db80 1 -- 172.21.15.39:0/2974811512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 msgr2=0x5609cb798f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.469+0000 7fe0b268db80 1 --2- 172.21.15.39:0/2974811512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb798f40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe090009990 tx=0x7fe0900056a0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.470+0000 7fe0b268db80 1 -- 172.21.15.39:0/2974811512 shutdown_connections 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.470+0000 7fe0b268db80 1 --2- 172.21.15.39:0/2974811512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.470+0000 7fe0b268db80 1 --2- 172.21.15.39:0/2974811512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.470+0000 7fe0b268db80 1 --2- 172.21.15.39:0/2974811512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb798f40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.470+0000 7fe0b268db80 1 -- 172.21.15.39:0/2974811512 >> 172.21.15.39:0/2974811512 conn(0x5609cb6f71b0 msgr2=0x5609cb793840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.470+0000 7fe0a4a6f700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe0880a1310 0x7fe0880a37d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe098009fc0 tx=0x7fe09800b040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.470+0000 7fe0b268db80 1 -- 172.21.15.39:0/2974811512 shutdown_connections 2024-05-24T11:41:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.470+0000 7fe0b268db80 1 -- 172.21.15.39:0/2974811512 wait complete. 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.471+0000 7fe0b268db80 1 -- start start 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.471+0000 7fe0b268db80 1 Processor -- start 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.471+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7a5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.471+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb7a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.471+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb7a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.471+0000 7fe0b268db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb685500 con 0x5609cb79e950 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.471+0000 7fe0b268db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb6851a0 con 0x5609cb799480 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.471+0000 7fe0b268db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb56c640 con 0x5609cb79f270 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.471+0000 7fe0b268db80 10 client.0 _ll_register_callbacks cb 0x5609cb68ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.472+0000 7fe09ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.472+0000 7fe09ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:36094/0 (socket says 172.21.15.39:36094) 2024-05-24T11:41:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.472+0000 7fe09ffff700 1 -- 172.21.15.39:0/3136661905 learned_addr learned my addr 172.21.15.39:0/3136661905 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.472+0000 7fe09ffff700 1 -- 172.21.15.39:0/3136661905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 msgr2=0x5609cb7a2ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:41:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.472+0000 7fe09ffff700 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb7a2ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.473+0000 7fe09ffff700 1 -- 172.21.15.39:0/3136661905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 msgr2=0x5609cb7a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.473+0000 7fe09ffff700 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb7a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.473+0000 7fe09ffff700 1 -- 172.21.15.39:0/3136661905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a010e620 con 0x5609cb799480 2024-05-24T11:41:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.475+0000 7fe09ffff700 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe090005cd0 tx=0x7fe090005570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1c0e7671859edf server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.475+0000 7fe07f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:41:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.475+0000 7fe07f7fe700 1 -- 172.21.15.39:0/3136661905 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090003880 con 0x5609cb799480 2024-05-24T11:41:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.475+0000 7fe07f7fe700 1 -- 172.21.15.39:0/3136661905 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe0900039e0 con 0x5609cb799480 2024-05-24T11:41:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.475+0000 7fe0b268db80 1 -- 172.21.15.39:0/3136661905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609cb642210 con 0x5609cb799480 2024-05-24T11:41:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe07f7fe700 1 -- 172.21.15.39:0/3136661905 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe090005320 con 0x5609cb799480 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe0b268db80 10 client.25355 fetch_fsmap learned FSMap version 27 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe0b268db80 1 -- 172.21.15.39:0/3136661905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5609cb516b00 con 0x5609cb799480 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe07f7fe700 1 -- 172.21.15.39:0/3136661905 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5609cb642210 con 0x5609cb799480 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe07f7fe700 1 -- 172.21.15.39:0/3136661905 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe09000fcd0 con 0x5609cb799480 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe0b268db80 10 client.25355 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe0b268db80 10 client.25355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe0b268db80 20 client.25355 populate_metadata read hostname 'smithi039' 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5609cb7b27a0 0x5609cb7b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe0b268db80 4 client.25355 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.476+0000 7fe0b268db80 1 -- 172.21.15.39:0/3136661905 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609cb7687b0 con 0x5609cb7b27a0 2024-05-24T11:41:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.477+0000 7fe09f7fe700 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5609cb7b27a0 0x5609cb7b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.477+0000 7fe07f7fe700 10 client.25355 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.477+0000 7fe09f7fe700 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5609cb7b27a0 0x5609cb7b4bc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2575e2b0274a1ad server_cookie=fe4b44b085ec037f in_seq=0 out_seq=0 2024-05-24T11:41:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.478+0000 7fe07f7fe700 1 -- 172.21.15.39:0/3136661905 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609cb7687b0 con 0x5609cb7b27a0 2024-05-24T11:41:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.478+0000 7fe07f7fe700 10 client.25355 handle_command_reply: tid=0 2024-05-24T11:41:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.478+0000 7fe0b268db80 1 client.25355 shutdown 2024-05-24T11:41:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.478+0000 7fe0b268db80 1 -- 172.21.15.39:0/3136661905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 msgr2=0x5609cb7a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.478+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe090005cd0 tx=0x7fe090005570 comp rx=0 tx=0).stop 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe0b268db80 1 -- 172.21.15.39:0/3136661905 shutdown_connections 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb7a2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5609cb7b27a0 0x5609cb7b4bc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7a5260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3136661905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb7a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe07f7fe700 0 client.25355 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe0b268db80 1 -- 172.21.15.39:0/3136661905 >> 172.21.15.39:0/3136661905 conn(0x5609cb6f71b0 msgr2=0x5609cb79d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe0b268db80 1 -- 172.21.15.39:0/3136661905 shutdown_connections 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe0b268db80 1 -- 172.21.15.39:0/3136661905 wait complete. 2024-05-24T11:41:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.479+0000 7fe0b268db80 20 client.25355 trim_cache size 0 max 16384 2024-05-24T11:41:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.504+0000 7fe0b268db80 1 Processor -- start 2024-05-24T11:41:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.504+0000 7fe0b268db80 1 -- start start 2024-05-24T11:41:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.504+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb799480 0x5609cb7b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.504+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb79e950 0x5609cb7a2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.504+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79f270 0x5609cb7a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.504+0000 7fe0b268db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb7b9d70 con 0x5609cb79f270 2024-05-24T11:41:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.504+0000 7fe0b268db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb7a0af0 con 0x5609cb79e950 2024-05-24T11:41:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.504+0000 7fe0b268db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb798200 con 0x5609cb799480 2024-05-24T11:41:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.505+0000 7fe0a4a6f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79f270 0x5609cb7a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.505+0000 7fe0a4a6f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79f270 0x5609cb7a2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41642/0 (socket says 172.21.15.39:41642) 2024-05-24T11:41:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.505+0000 7fe0a4a6f700 1 -- 172.21.15.39:0/3936769312 learned_addr learned my addr 172.21.15.39:0/3936769312 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.505+0000 7fe09f7fe700 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb79e950 0x5609cb7a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.505+0000 7fe0a4a6f700 1 -- 172.21.15.39:0/3936769312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb799480 msgr2=0x5609cb7b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe09ffff700 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb799480 0x5609cb7b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe0a4a6f700 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb799480 0x5609cb7b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe0a4a6f700 1 -- 172.21.15.39:0/3936769312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb79e950 msgr2=0x5609cb7a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe0a4a6f700 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb79e950 0x5609cb7a2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe0a4a6f700 1 -- 172.21.15.39:0/3936769312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609cb6af500 con 0x5609cb79f270 2024-05-24T11:41:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe09ffff700 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb799480 0x5609cb7b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:41:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe09f7fe700 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb79e950 0x5609cb7a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:41:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe0a4a6f700 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79f270 0x5609cb7a2c60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe0980098e0 tx=0x7fe098011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7721e2a2c6c6ae90 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe086ffd700 1 -- 172.21.15.39:0/3936769312 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098015ae0 con 0x5609cb79f270 2024-05-24T11:41:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe086ffd700 1 -- 172.21.15.39:0/3936769312 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe09800f040 con 0x5609cb79f270 2024-05-24T11:41:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.506+0000 7fe086ffd700 1 -- 172.21.15.39:0/3936769312 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980133c0 con 0x5609cb79f270 2024-05-24T11:41:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.508+0000 7fe0b268db80 1 -- 172.21.15.39:0/3936769312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79f270 msgr2=0x5609cb7a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.508+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79f270 0x5609cb7a2c60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe0980098e0 tx=0x7fe098011000 comp rx=0 tx=0).stop 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.509+0000 7fe0b268db80 1 -- 172.21.15.39:0/3936769312 shutdown_connections 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.509+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb799480 0x5609cb7b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.509+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79f270 0x5609cb7a2c60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.509+0000 7fe0b268db80 1 --2- 172.21.15.39:0/3936769312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb79e950 0x5609cb7a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.509+0000 7fe0b268db80 1 -- 172.21.15.39:0/3936769312 >> 172.21.15.39:0/3936769312 conn(0x5609cb6f71b0 msgr2=0x5609cb7976d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.509+0000 7fe0b268db80 1 -- 172.21.15.39:0/3936769312 shutdown_connections 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.509+0000 7fe0b268db80 1 -- 172.21.15.39:0/3936769312 wait complete. 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.509+0000 7fe0b268db80 1 -- start start 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe0b268db80 1 Processor -- start 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7aa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe0b268db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb79dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe0b268db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb79c530 con 0x5609cb79e950 2024-05-24T11:41:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe0b268db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb79c6a0 con 0x5609cb799480 2024-05-24T11:41:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe0b268db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609cb79c860 con 0x5609cb79f270 2024-05-24T11:41:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe0b268db80 10 client.0 _ll_register_callbacks cb 0x5609cb68ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:41:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe09f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe09f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41658/0 (socket says 172.21.15.39:41658) 2024-05-24T11:41:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe09f7fe700 1 -- 172.21.15.39:0/144309735 learned_addr learned my addr 172.21.15.39:0/144309735 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe09ffff700 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.510+0000 7fe0a4a6f700 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb79dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.511+0000 7fe0a4a6f700 1 -- 172.21.15.39:0/144309735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 msgr2=0x5609cb7aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.511+0000 7fe0a4a6f700 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.511+0000 7fe0a4a6f700 1 -- 172.21.15.39:0/144309735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 msgr2=0x5609cb79d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.511+0000 7fe0a4a6f700 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.511+0000 7fe0a4a6f700 1 -- 172.21.15.39:0/144309735 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a00724a0 con 0x5609cb79f270 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.511+0000 7fe09f7fe700 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.511+0000 7fe0a4a6f700 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb79dda0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe098004c90 tx=0x7fe098016d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7170cb1382876abe server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.511+0000 7fe07f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.511+0000 7fe07f7fe700 1 -- 172.21.15.39:0/144309735 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09801f600 con 0x5609cb79f270 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe0b268db80 1 -- 172.21.15.39:0/144309735 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe09001d040 con 0x5609cb79f270 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe07f7fe700 1 -- 172.21.15.39:0/144309735 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fe098005e40 con 0x5609cb79f270 2024-05-24T11:41:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe07f7fe700 1 -- 172.21.15.39:0/144309735 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe09801f760 con 0x5609cb79f270 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe0b268db80 10 client.25270 fetch_fsmap learned FSMap version 27 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe0b268db80 1 -- 172.21.15.39:0/144309735 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe09800eaf0 con 0x5609cb79f270 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe07f7fe700 1 -- 172.21.15.39:0/144309735 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe09001d040 con 0x5609cb79f270 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe07f7fe700 1 -- 172.21.15.39:0/144309735 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe098028390 con 0x5609cb79f270 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe0b268db80 10 client.25270 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe0b268db80 10 client.25270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe0b268db80 20 client.25270 populate_metadata read hostname 'smithi039' 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe0b268db80 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5609cb809740 0x5609cb80bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe0b268db80 4 client.25270 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.512+0000 7fe0b268db80 1 -- 172.21.15.39:0/144309735 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609cb6e1e60 con 0x5609cb809740 2024-05-24T11:41:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.513+0000 7fe09ffff700 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5609cb809740 0x5609cb80bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.513+0000 7fe09ffff700 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5609cb809740 0x5609cb80bb60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3816c8fe5f7111e6 server_cookie=be36e0ca90e7abfc in_seq=0 out_seq=0 2024-05-24T11:41:22.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.513+0000 7fe07f7fe700 10 client.25270 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe07f7fe700 1 -- 172.21.15.39:0/144309735 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609cb6e1e60 con 0x5609cb809740 2024-05-24T11:41:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe07f7fe700 10 client.25270 handle_command_reply: tid=0 2024-05-24T11:41:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 client.25270 shutdown 2024-05-24T11:41:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 -- 172.21.15.39:0/144309735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 msgr2=0x5609cb79dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb79dda0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe098004c90 tx=0x7fe098016d20 comp rx=0 tx=0).stop 2024-05-24T11:41:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 -- 172.21.15.39:0/144309735 shutdown_connections 2024-05-24T11:41:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609cb79f270 0x5609cb79dda0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5609cb809740 0x5609cb80bb60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609cb799480 0x5609cb7aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 --2- 172.21.15.39:0/144309735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5609cb79e950 0x5609cb79d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 -- 172.21.15.39:0/144309735 >> 172.21.15.39:0/144309735 conn(0x5609cb6f71b0 msgr2=0x5609cb7a0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe07f7fe700 0 client.25270 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 -- 172.21.15.39:0/144309735 shutdown_connections 2024-05-24T11:41:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 1 -- 172.21.15.39:0/144309735 wait complete. 2024-05-24T11:41:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.899+0000 7fe0b268db80 20 client.25270 trim_cache size 0 max 16384 2024-05-24T11:41:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:41:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:41:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:41:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:41:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 -- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe0880a1310 msgr2=0x7fe0880a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe0880a1310 0x7fe0880a37d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe098009fc0 tx=0x7fe09800b040 comp rx=0 tx=0).stop 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 -- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a0069ff0 msgr2=0x7fe0a01005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a0069ff0 0x7fe0a01005c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe09400e9a0 tx=0x7fe09400ecb0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 -- 172.21.15.39:0/223582811 shutdown_connections 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a0069630 0x7fe0a0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe0880a1310 0x7fe0880a37d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0a0106f10 0x7fe0a0100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 --2- 172.21.15.39:0/223582811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a0069ff0 0x7fe0a01005c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 -- 172.21.15.39:0/223582811 >> 172.21.15.39:0/223582811 conn(0x7fe0a0076c10 msgr2=0x7fe0a00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.900+0000 7fe0a64d2700 1 -- 172.21.15.39:0/223582811 shutdown_connections 2024-05-24T11:41:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:22.901+0000 7fe0a64d2700 1 -- 172.21.15.39:0/223582811 wait complete. 2024-05-24T11:41:23.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:23 smithi053 ceph-mon[47448]: pgmap v98: 97 pgs: 97 active+clean; 2.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-05-24T11:41:23.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:23 smithi137 ceph-mon[47907]: pgmap v98: 97 pgs: 97 active+clean; 2.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-05-24T11:41:24.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:23 smithi039 ceph-mon[42919]: pgmap v98: 97 pgs: 97 active+clean; 2.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-05-24T11:41:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 120.0 s, 1288.9 tps, lat 77.504 ms stddev 38.625 2024-05-24T11:41:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:25 smithi053 ceph-mon[47448]: pgmap v99: 97 pgs: 97 active+clean; 2.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 21 MiB/s wr, 1.89k op/s 2024-05-24T11:41:25.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:25 smithi137 ceph-mon[47907]: pgmap v99: 97 pgs: 97 active+clean; 2.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 21 MiB/s wr, 1.89k op/s 2024-05-24T11:41:26.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:25 smithi039 ceph-mon[42919]: pgmap v99: 97 pgs: 97 active+clean; 2.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 21 MiB/s wr, 1.89k op/s 2024-05-24T11:41:27.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:27 smithi053 ceph-mon[47448]: pgmap v100: 97 pgs: 97 active+clean; 2.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-05-24T11:41:27.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:27 smithi137 ceph-mon[47907]: pgmap v100: 97 pgs: 97 active+clean; 2.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-05-24T11:41:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:27 smithi039 ceph-mon[42919]: pgmap v100: 97 pgs: 97 active+clean; 2.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-05-24T11:41:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 125.0 s, 1387.0 tps, lat 72.212 ms stddev 19.250 2024-05-24T11:41:29.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:29 smithi053 ceph-mon[47448]: pgmap v101: 97 pgs: 97 active+clean; 2.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 20 MiB/s wr, 1.87k op/s 2024-05-24T11:41:29.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:29 smithi137 ceph-mon[47907]: pgmap v101: 97 pgs: 97 active+clean; 2.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 20 MiB/s wr, 1.87k op/s 2024-05-24T11:41:30.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:29 smithi039 ceph-mon[42919]: pgmap v101: 97 pgs: 97 active+clean; 2.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 20 MiB/s wr, 1.87k op/s 2024-05-24T11:41:31.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:31 smithi053 ceph-mon[47448]: pgmap v102: 97 pgs: 97 active+clean; 2.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:41:31.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:31 smithi137 ceph-mon[47907]: pgmap v102: 97 pgs: 97 active+clean; 2.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:41:32.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:31 smithi039 ceph-mon[42919]: pgmap v102: 97 pgs: 97 active+clean; 2.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:41:33.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:33 smithi053 ceph-mon[47448]: pgmap v103: 97 pgs: 97 active+clean; 2.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:41:33.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:33 smithi137 ceph-mon[47907]: pgmap v103: 97 pgs: 97 active+clean; 2.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:41:34.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:33 smithi039 ceph-mon[42919]: pgmap v103: 97 pgs: 97 active+clean; 2.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:41:34.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 130.0 s, 1314.2 tps, lat 76.043 ms stddev 21.461 2024-05-24T11:41:35.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:35 smithi053 ceph-mon[47448]: pgmap v104: 97 pgs: 97 active+clean; 2.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 21 MiB/s wr, 1.92k op/s 2024-05-24T11:41:35.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:35 smithi137 ceph-mon[47907]: pgmap v104: 97 pgs: 97 active+clean; 2.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 21 MiB/s wr, 1.92k op/s 2024-05-24T11:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.071+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3719908653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f01088c0 msgr2=0x7fd5f0113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.071+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3719908653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f01088c0 0x7fd5f0113bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40098f0 tx=0x7fd5e4009c00 comp rx=0 tx=0).stop 2024-05-24T11:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.072+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3719908653 shutdown_connections 2024-05-24T11:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.072+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3719908653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5f01140f0 0x7fd5f01164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.072+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3719908653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5f0107fa0 0x7fd5f0108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.072+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3719908653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f01088c0 0x7fd5f0113bb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.072+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3719908653 >> 172.21.15.39:0/3719908653 conn(0x7fd5f006bf80 msgr2=0x7fd5f006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.072+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3719908653 shutdown_connections 2024-05-24T11:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.072+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3719908653 wait complete. 2024-05-24T11:41:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5f4e5a700 1 Processor -- start 2024-05-24T11:41:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5f4e5a700 1 -- start start 2024-05-24T11:41:36.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:35 smithi039 ceph-mon[42919]: pgmap v104: 97 pgs: 97 active+clean; 2.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 21 MiB/s wr, 1.92k op/s 2024-05-24T11:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5f4e5a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f0107fa0 0x7fd5f01a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5f4e5a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5f01088c0 0x7fd5f01a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5f4e5a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5f01140f0 0x7fd5f019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5f4e5a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f019fe10 con 0x7fd5f01088c0 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5f4e5a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f019ff80 con 0x7fd5f0107fa0 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5f4e5a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f01a0140 con 0x7fd5f01140f0 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5ee59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f0107fa0 0x7fd5f01a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5ee59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f0107fa0 0x7fd5f01a57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:55316/0 (socket says 172.21.15.39:55316) 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5ee59c700 1 -- 172.21.15.39:0/3768968993 learned_addr learned my addr 172.21.15.39:0/3768968993 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.073+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5f01088c0 0x7fd5f01a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5eed9d700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5f01140f0 0x7fd5f019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5ee59c700 1 -- 172.21.15.39:0/3768968993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5f01140f0 msgr2=0x7fd5f019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5f01140f0 0x7fd5f019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5ee59c700 1 -- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5f01088c0 msgr2=0x7fd5f01a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5f01088c0 0x7fd5f01a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5ee59c700 1 -- 172.21.15.39:0/3768968993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4009520 con 0x7fd5f0107fa0 2024-05-24T11:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f0107fa0 0x7fd5f01a57a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9c0 tx=0x7fd5e000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5df7fe700 1 -- 172.21.15.39:0/3768968993 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000cbd0 con 0x7fd5f0107fa0 2024-05-24T11:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5df7fe700 1 -- 172.21.15.39:0/3768968993 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5e000eeb0 con 0x7fd5f0107fa0 2024-05-24T11:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5df7fe700 1 -- 172.21.15.39:0/3768968993 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0005410 con 0x7fd5f0107fa0 2024-05-24T11:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3768968993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f01a0340 con 0x7fd5f0107fa0 2024-05-24T11:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.074+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3768968993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f01aa5a0 con 0x7fd5f0107fa0 2024-05-24T11:41:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.078+0000 7fd5df7fe700 1 -- 172.21.15.39:0/3768968993 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd5e00055b0 con 0x7fd5f0107fa0 2024-05-24T11:41:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.080+0000 7fd601015b80 1 Processor -- start 2024-05-24T11:41:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.080+0000 7fd5df7fe700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5d80a1310 0x7fd5d80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.081+0000 7fd5df7fe700 1 -- 172.21.15.39:0/3768968993 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd5e0014070 con 0x7fd5f0107fa0 2024-05-24T11:41:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.082+0000 7fd5eed9d700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5d80a1310 0x7fd5d80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.082+0000 7fd601015b80 1 -- start start 2024-05-24T11:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.082+0000 7fd601015b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c31d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.082+0000 7fd601015b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c2bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.082+0000 7fd601015b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c2c480 0x55f728c2c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.082+0000 7fd601015b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728b830f0 con 0x55f728c32270 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.082+0000 7fd601015b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728b80ab0 con 0x55f728c31950 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.082+0000 7fd601015b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728b7c3a0 con 0x55f728c2c480 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5ee59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c31d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5edd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c2bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5edd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c2bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52298/0 (socket says 172.21.15.39:52298) 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5edd9b700 1 -- 172.21.15.39:0/192444297 learned_addr learned my addr 172.21.15.39:0/192444297 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5eed9d700 1 --2- 172.21.15.39:0/192444297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c2c480 0x55f728c2c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5edd9b700 1 -- 172.21.15.39:0/192444297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c2c480 msgr2=0x55f728c2c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/192444297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c2c480 0x55f728c2c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5edd9b700 1 -- 172.21.15.39:0/192444297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 msgr2=0x55f728c31d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/192444297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c31d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5edd9b700 1 -- 172.21.15.39:0/192444297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7289fd350 con 0x55f728c32270 2024-05-24T11:41:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.083+0000 7fd5eed9d700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5d80a1310 0x7fd5d80a37d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5f006a570 tx=0x7fd5e800b040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.084+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/192444297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c2bf40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009230 tx=0x7fd5e4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4179a9b02f0166a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.084+0000 7fd5dcff9700 1 -- 172.21.15.39:0/192444297 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400fd60 con 0x55f728c32270 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.084+0000 7fd5dcff9700 1 -- 172.21.15.39:0/192444297 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5e401d040 con 0x55f728c32270 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.084+0000 7fd5dcff9700 1 -- 172.21.15.39:0/192444297 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4005040 con 0x55f728c32270 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.085+0000 7fd601015b80 1 -- 172.21.15.39:0/192444297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 msgr2=0x55f728c2bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.085+0000 7fd601015b80 1 --2- 172.21.15.39:0/192444297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c2bf40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009230 tx=0x7fd5e4005dc0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.085+0000 7fd601015b80 1 -- 172.21.15.39:0/192444297 shutdown_connections 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.085+0000 7fd601015b80 1 --2- 172.21.15.39:0/192444297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c2c480 0x55f728c2c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.085+0000 7fd601015b80 1 --2- 172.21.15.39:0/192444297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c31d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.085+0000 7fd601015b80 1 --2- 172.21.15.39:0/192444297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c2bf40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.085+0000 7fd601015b80 1 -- 172.21.15.39:0/192444297 >> 172.21.15.39:0/192444297 conn(0x55f728b8a1b0 msgr2=0x55f728c26840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.086+0000 7fd601015b80 1 -- 172.21.15.39:0/192444297 shutdown_connections 2024-05-24T11:41:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.086+0000 7fd601015b80 1 -- 172.21.15.39:0/192444297 wait complete. 2024-05-24T11:41:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.086+0000 7fd601015b80 1 -- start start 2024-05-24T11:41:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.086+0000 7fd601015b80 1 Processor -- start 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd601015b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c38270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd601015b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c31950 0x55f728c35560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd601015b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c35ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd601015b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728b18500 con 0x55f728c32270 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd601015b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728b181a0 con 0x55f728c2c480 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd601015b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7289ff640 con 0x55f728c31950 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd601015b80 10 client.0 _ll_register_callbacks cb 0x55f728b21ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd5ee59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c38270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd5ee59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c38270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:55326/0 (socket says 172.21.15.39:55326) 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.087+0000 7fd5ee59c700 1 -- 172.21.15.39:0/2612225920 learned_addr learned my addr 172.21.15.39:0/2612225920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.088+0000 7fd5eed9d700 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c35ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.088+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c31950 0x55f728c35560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.088+0000 7fd5ee59c700 1 -- 172.21.15.39:0/2612225920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c31950 msgr2=0x55f728c35560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.088+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c31950 0x55f728c35560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.088+0000 7fd5ee59c700 1 -- 172.21.15.39:0/2612225920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 msgr2=0x55f728c35ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.088+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c35ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.088+0000 7fd5ee59c700 1 -- 172.21.15.39:0/2612225920 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f0116e60 con 0x55f728c2c480 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.088+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c38270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00ca6c0 tx=0x7fd5e009a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb9694d4e7edddc server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.088+0000 7fd5d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.089+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/2612225920 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00a24a0 con 0x55f728c2c480 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.089+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/2612225920 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd5e0099510 con 0x55f728c2c480 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.089+0000 7fd601015b80 1 -- 172.21.15.39:0/2612225920 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f728ad5210 con 0x55f728c2c480 2024-05-24T11:41:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.089+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/2612225920 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e00996d0 con 0x55f728c2c480 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.089+0000 7fd601015b80 10 client.25373 fetch_fsmap learned FSMap version 27 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.089+0000 7fd601015b80 1 -- 172.21.15.39:0/2612225920 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f7289a9b00 con 0x55f728c2c480 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.089+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/2612225920 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f728ad5210 con 0x55f728c2c480 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.089+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/2612225920 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5e00a2010 con 0x55f728c2c480 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.090+0000 7fd601015b80 10 client.25373 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.090+0000 7fd601015b80 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.090+0000 7fd601015b80 20 client.25373 populate_metadata read hostname 'smithi039' 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.090+0000 7fd601015b80 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f728c45b80 0x55f728c47fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.090+0000 7fd601015b80 4 client.25373 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.090+0000 7fd601015b80 1 -- 172.21.15.39:0/2612225920 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f728bfb7b0 con 0x55f728c45b80 2024-05-24T11:41:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.090+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f728c45b80 0x55f728c47fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.091+0000 7fd5d5ffb700 10 client.25373 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.091+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f728c45b80 0x55f728c47fa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979d46214e5d5b3 server_cookie=f5fe5594c153ca89 in_seq=0 out_seq=0 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.091+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/2612225920 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f728bfb7b0 con 0x55f728c45b80 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.091+0000 7fd5d5ffb700 10 client.25373 handle_command_reply: tid=0 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 client.25373 shutdown 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 -- 172.21.15.39:0/2612225920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 msgr2=0x55f728c38270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c38270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00ca6c0 tx=0x7fd5e009a7a0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 -- 172.21.15.39:0/2612225920 shutdown_connections 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f728c45b80 0x55f728c47fa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c32270 0x55f728c35ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c38270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 --2- 172.21.15.39:0/2612225920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c31950 0x55f728c35560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 -- 172.21.15.39:0/2612225920 >> 172.21.15.39:0/2612225920 conn(0x55f728b8a1b0 msgr2=0x55f728c2af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd5d5ffb700 0 client.25373 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.092+0000 7fd601015b80 1 -- 172.21.15.39:0/2612225920 shutdown_connections 2024-05-24T11:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.093+0000 7fd601015b80 1 -- 172.21.15.39:0/2612225920 wait complete. 2024-05-24T11:41:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.093+0000 7fd601015b80 20 client.25373 trim_cache size 0 max 16384 2024-05-24T11:41:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd601015b80 1 Processor -- start 2024-05-24T11:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd601015b80 1 -- start start 2024-05-24T11:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd601015b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c2c480 0x55f728c41d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd601015b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c41660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd601015b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 0x55f728c26910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd601015b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728c43300 con 0x55f728c2c480 2024-05-24T11:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd601015b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728c315e0 con 0x55f728c31950 2024-05-24T11:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd601015b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728c36510 con 0x55f728c32270 2024-05-24T11:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd5edd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c41660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd5edd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c41660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:55334/0 (socket says 172.21.15.39:55334) 2024-05-24T11:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd5edd9b700 1 -- 172.21.15.39:0/1799874633 learned_addr learned my addr 172.21.15.39:0/1799874633 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.119+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/1799874633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c2c480 0x55f728c41d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.120+0000 7fd5edd9b700 1 -- 172.21.15.39:0/1799874633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 msgr2=0x55f728c26910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.120+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/1799874633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 0x55f728c26910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.120+0000 7fd5edd9b700 1 -- 172.21.15.39:0/1799874633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c2c480 msgr2=0x55f728c41d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.120+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/1799874633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c2c480 0x55f728c41d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.120+0000 7fd5edd9b700 1 -- 172.21.15.39:0/1799874633 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f728b42500 con 0x55f728c31950 2024-05-24T11:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.120+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/1799874633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c41660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009230 tx=0x7fd5e4005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e37f9ffc120f05 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.120+0000 7fd5dd7fa700 1 -- 172.21.15.39:0/1799874633 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400f660 con 0x55f728c31950 2024-05-24T11:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.120+0000 7fd5dd7fa700 1 -- 172.21.15.39:0/1799874633 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5e400bb30 con 0x55f728c31950 2024-05-24T11:41:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.120+0000 7fd5dd7fa700 1 -- 172.21.15.39:0/1799874633 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40056b0 con 0x55f728c31950 2024-05-24T11:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.121+0000 7fd601015b80 1 -- 172.21.15.39:0/1799874633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 msgr2=0x55f728c41660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.121+0000 7fd601015b80 1 --2- 172.21.15.39:0/1799874633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c41660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009230 tx=0x7fd5e4005e30 comp rx=0 tx=0).stop 2024-05-24T11:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.122+0000 7fd601015b80 1 -- 172.21.15.39:0/1799874633 shutdown_connections 2024-05-24T11:41:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.122+0000 7fd601015b80 1 --2- 172.21.15.39:0/1799874633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 0x55f728c26910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.122+0000 7fd601015b80 1 --2- 172.21.15.39:0/1799874633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c2c480 0x55f728c41d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.122+0000 7fd601015b80 1 --2- 172.21.15.39:0/1799874633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c31950 0x55f728c41660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.122+0000 7fd601015b80 1 -- 172.21.15.39:0/1799874633 >> 172.21.15.39:0/1799874633 conn(0x55f728b8a1b0 msgr2=0x55f728c34aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.122+0000 7fd601015b80 1 -- 172.21.15.39:0/1799874633 shutdown_connections 2024-05-24T11:41:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.122+0000 7fd601015b80 1 -- 172.21.15.39:0/1799874633 wait complete. 2024-05-24T11:41:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.122+0000 7fd601015b80 1 -- start start 2024-05-24T11:41:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.122+0000 7fd601015b80 1 Processor -- start 2024-05-24T11:41:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd601015b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c37820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd601015b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c31950 0x55f728c37d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd601015b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 0x55f728c96010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd601015b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728c38440 con 0x55f728c31950 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd601015b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728c96550 con 0x55f728c2c480 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd601015b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f728c96710 con 0x55f728c32270 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd601015b80 10 client.0 _ll_register_callbacks cb 0x55f728b21ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd5edd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c31950 0x55f728c37d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd5edd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c31950 0x55f728c37d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52316/0 (socket says 172.21.15.39:52316) 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd5edd9b700 1 -- 172.21.15.39:0/3103114424 learned_addr learned my addr 172.21.15.39:0/3103114424 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd5eed9d700 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 0x55f728c96010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.123+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c37820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5eed9d700 1 -- 172.21.15.39:0/3103114424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 msgr2=0x55f728c37820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5eed9d700 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c37820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5eed9d700 1 -- 172.21.15.39:0/3103114424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c31950 msgr2=0x55f728c37d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5eed9d700 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c31950 0x55f728c37d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5eed9d700 1 -- 172.21.15.39:0/3103114424 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7289fd350 con 0x55f728c32270 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5edd9b700 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c31950 0x55f728c37d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5eed9d700 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 0x55f728c96010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009550 tx=0x7fd5e8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee8019e1fe49e81b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/3103114424 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800e050 con 0x55f728c32270 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd601015b80 1 -- 172.21.15.39:0/3103114424 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5e401d040 con 0x55f728c32270 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/3103114424 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd5e8024850 con 0x55f728c32270 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/3103114424 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e8007ed0 con 0x55f728c32270 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd601015b80 10 client.25285 fetch_fsmap learned FSMap version 27 2024-05-24T11:41:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.124+0000 7fd601015b80 1 -- 172.21.15.39:0/3103114424 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f728a01c70 con 0x55f728c32270 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/3103114424 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5e401d040 con 0x55f728c32270 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/3103114424 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5e802a020 con 0x55f728c32270 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd601015b80 10 client.25285 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd601015b80 10 client.25285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd601015b80 20 client.25285 populate_metadata read hostname 'smithi039' 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd601015b80 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f728cad040 0x55f728caf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd601015b80 4 client.25285 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd601015b80 1 -- 172.21.15.39:0/3103114424 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f728b74e60 con 0x55f728cad040 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f728cad040 0x55f728caf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd5ee59c700 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f728cad040 0x55f728caf430 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843a3ac184e8539c server_cookie=16b9bf8f9eb8270 in_seq=0 out_seq=0 2024-05-24T11:41:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.125+0000 7fd5d5ffb700 10 client.25285 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd5d5ffb700 1 -- 172.21.15.39:0/3103114424 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f728b74e60 con 0x55f728cad040 2024-05-24T11:41:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd5d5ffb700 10 client.25285 handle_command_reply: tid=0 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 client.25285 shutdown 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 -- 172.21.15.39:0/3103114424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 msgr2=0x55f728c96010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 0x55f728c96010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009550 tx=0x7fd5e8004ad0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 -- 172.21.15.39:0/3103114424 shutdown_connections 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f728c32270 0x55f728c96010 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55f728cad040 0x55f728caf430 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f728c2c480 0x55f728c37820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 --2- 172.21.15.39:0/3103114424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f728c31950 0x55f728c37d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd5d5ffb700 0 client.25285 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 -- 172.21.15.39:0/3103114424 >> 172.21.15.39:0/3103114424 conn(0x55f728b8a1b0 msgr2=0x55f728c2f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.157+0000 7fd601015b80 1 -- 172.21.15.39:0/3103114424 shutdown_connections 2024-05-24T11:41:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.158+0000 7fd601015b80 1 -- 172.21.15.39:0/3103114424 wait complete. 2024-05-24T11:41:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.159+0000 7fd601015b80 20 client.25285 trim_cache size 0 max 16384 2024-05-24T11:41:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:41:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:41:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:41:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:41:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5d80a1310 msgr2=0x7fd5d80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5d80a1310 0x7fd5d80a37d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5f006a570 tx=0x7fd5e800b040 comp rx=0 tx=0).stop 2024-05-24T11:41:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3768968993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f0107fa0 msgr2=0x7fd5f01a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f0107fa0 0x7fd5f01a57a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9c0 tx=0x7fd5e000ecd0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3768968993 shutdown_connections 2024-05-24T11:41:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5f01140f0 0x7fd5f019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5d80a1310 0x7fd5d80a37d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5f0107fa0 0x7fd5f01a57a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 --2- 172.21.15.39:0/3768968993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5f01088c0 0x7fd5f01a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3768968993 >> 172.21.15.39:0/3768968993 conn(0x7fd5f006bf80 msgr2=0x7fd5f0071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3768968993 shutdown_connections 2024-05-24T11:41:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:36.160+0000 7fd5f4e5a700 1 -- 172.21.15.39:0/3768968993 wait complete. 2024-05-24T11:41:37.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:37 smithi053 ceph-mon[47448]: pgmap v105: 97 pgs: 97 active+clean; 2.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:41:37.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:37 smithi137 ceph-mon[47907]: pgmap v105: 97 pgs: 97 active+clean; 2.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:41:38.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:37 smithi039 ceph-mon[42919]: pgmap v105: 97 pgs: 97 active+clean; 2.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:41:39.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 135.0 s, 1203.4 tps, lat 83.118 ms stddev 29.253 2024-05-24T11:41:39.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:39 smithi053 ceph-mon[47448]: pgmap v106: 97 pgs: 97 active+clean; 2.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 24 MiB/s wr, 1.88k op/s 2024-05-24T11:41:39.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:39 smithi137 ceph-mon[47907]: pgmap v106: 97 pgs: 97 active+clean; 2.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 24 MiB/s wr, 1.88k op/s 2024-05-24T11:41:40.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:39 smithi039 ceph-mon[42919]: pgmap v106: 97 pgs: 97 active+clean; 2.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 24 MiB/s wr, 1.88k op/s 2024-05-24T11:41:41.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:41 smithi053 ceph-mon[47448]: pgmap v107: 97 pgs: 97 active+clean; 2.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 17 MiB/s wr, 1.26k op/s 2024-05-24T11:41:41.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:41 smithi137 ceph-mon[47907]: pgmap v107: 97 pgs: 97 active+clean; 2.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 17 MiB/s wr, 1.26k op/s 2024-05-24T11:41:42.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:41 smithi039 ceph-mon[42919]: pgmap v107: 97 pgs: 97 active+clean; 2.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 17 MiB/s wr, 1.26k op/s 2024-05-24T11:41:43.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:43 smithi053 ceph-mon[47448]: pgmap v108: 97 pgs: 97 active+clean; 2.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 17 MiB/s wr, 1.26k op/s 2024-05-24T11:41:43.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:43 smithi137 ceph-mon[47907]: pgmap v108: 97 pgs: 97 active+clean; 2.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 17 MiB/s wr, 1.26k op/s 2024-05-24T11:41:44.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:43 smithi039 ceph-mon[42919]: pgmap v108: 97 pgs: 97 active+clean; 2.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 17 MiB/s wr, 1.26k op/s 2024-05-24T11:41:44.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 140.0 s, 1396.0 tps, lat 71.603 ms stddev 20.447 2024-05-24T11:41:45.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:45 smithi053 ceph-mon[47448]: pgmap v109: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 26 MiB/s wr, 1.85k op/s 2024-05-24T11:41:45.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:45 smithi137 ceph-mon[47907]: pgmap v109: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 26 MiB/s wr, 1.85k op/s 2024-05-24T11:41:46.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:45 smithi039 ceph-mon[42919]: pgmap v109: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 26 MiB/s wr, 1.85k op/s 2024-05-24T11:41:47.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:47 smithi053 ceph-mon[47448]: pgmap v110: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-05-24T11:41:47.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:47 smithi137 ceph-mon[47907]: pgmap v110: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-05-24T11:41:48.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:47 smithi039 ceph-mon[42919]: pgmap v110: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-05-24T11:41:49.118 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 145.0 s, 1345.5 tps, lat 74.170 ms stddev 21.436 2024-05-24T11:41:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.328+0000 7f82d9ac4700 1 -- 172.21.15.39:0/2092693849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82d4073340 msgr2=0x7f82d4073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.328+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/2092693849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82d4073340 0x7f82d4073720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82c40098f0 tx=0x7f82c4009c00 comp rx=0 tx=0).stop 2024-05-24T11:41:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.328+0000 7f82d9ac4700 1 -- 172.21.15.39:0/2092693849 shutdown_connections 2024-05-24T11:41:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.328+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/2092693849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82d4073340 0x7f82d4073720 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.328+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/2092693849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82d4111e90 0x7f82d4114280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.328+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/2092693849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d4073c60 0x7f82d4111950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.328+0000 7f82d9ac4700 1 -- 172.21.15.39:0/2092693849 >> 172.21.15.39:0/2092693849 conn(0x7f82d40fc9d0 msgr2=0x7f82d40fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.328+0000 7f82d9ac4700 1 -- 172.21.15.39:0/2092693849 shutdown_connections 2024-05-24T11:41:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.328+0000 7f82d9ac4700 1 -- 172.21.15.39:0/2092693849 wait complete. 2024-05-24T11:41:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.329+0000 7f82d9ac4700 1 Processor -- start 2024-05-24T11:41:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.329+0000 7f82d9ac4700 1 -- start start 2024-05-24T11:41:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d9ac4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82d4073340 0x7f82d41115a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d9ac4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82d4073c60 0x7f82d410c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d9ac4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d4111e90 0x7f82d410cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d9ac4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d410d160 con 0x7f82d4073c60 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d9ac4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d410d2d0 con 0x7f82d4111e90 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d9ac4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d410d490 con 0x7f82d4073340 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82d4073c60 0x7f82d410c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82d4073c60 0x7f82d410c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36830/0 (socket says 172.21.15.39:36830) 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d2ffd700 1 -- 172.21.15.39:0/3188080741 learned_addr learned my addr 172.21.15.39:0/3188080741 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d37fe700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82d4073340 0x7f82d41115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d3fff700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d4111e90 0x7f82d410cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d37fe700 1 -- 172.21.15.39:0/3188080741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d4111e90 msgr2=0x7f82d410cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d37fe700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d4111e90 0x7f82d410cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d37fe700 1 -- 172.21.15.39:0/3188080741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82d4073c60 msgr2=0x7f82d410c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d37fe700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82d4073c60 0x7f82d410c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d37fe700 1 -- 172.21.15.39:0/3188080741 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c4009520 con 0x7f82d4073340 2024-05-24T11:41:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.330+0000 7f82d3fff700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d4111e90 0x7f82d410cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:41:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.331+0000 7f82d37fe700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82d4073340 0x7f82d41115a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82c400b2e0 tx=0x7f82c400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.332+0000 7f82d0ff9700 1 -- 172.21.15.39:0/3188080741 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c401c070 con 0x7f82d4073340 2024-05-24T11:41:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.332+0000 7f82d0ff9700 1 -- 172.21.15.39:0/3188080741 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82c4005030 con 0x7f82d4073340 2024-05-24T11:41:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.332+0000 7f82d0ff9700 1 -- 172.21.15.39:0/3188080741 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4004b10 con 0x7f82d4073340 2024-05-24T11:41:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.332+0000 7f82d9ac4700 1 -- 172.21.15.39:0/3188080741 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d41b28d0 con 0x7f82d4073340 2024-05-24T11:41:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.332+0000 7f82d9ac4700 1 -- 172.21.15.39:0/3188080741 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d41b2d80 con 0x7f82d4073340 2024-05-24T11:41:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.335+0000 7f82d0ff9700 1 -- 172.21.15.39:0/3188080741 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f82c4004c70 con 0x7f82d4073340 2024-05-24T11:41:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.336+0000 7f82d0ff9700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f82bc0a13e0 0x7f82bc0a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.336+0000 7f82d0ff9700 1 -- 172.21.15.39:0/3188080741 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f82c40cda80 con 0x7f82d4073340 2024-05-24T11:41:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82d2ffd700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f82bc0a13e0 0x7f82bc0a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82e5c7fb80 1 Processor -- start 2024-05-24T11:41:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82e5c7fb80 1 -- start start 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82d2ffd700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f82bc0a13e0 0x7f82bc0a38a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82d410dd80 tx=0x7f82c800b500 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd2dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd27f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd288a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddc7f0f0 con 0x556cddd28480 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddc7cab0 con 0x556cddd2e270 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.337+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddc783a0 con 0x556cddd2d950 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd288a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd2dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd27f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd288a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36836/0 (socket says 172.21.15.39:36836) 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd2dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:42992/0 (socket says 172.21.15.39:42992) 2024-05-24T11:41:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd27f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:47496/0 (socket says 172.21.15.39:47496) 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d3fff700 1 -- 172.21.15.39:0/4039047744 learned_addr learned my addr 172.21.15.39:0/4039047744 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d3fff700 1 -- 172.21.15.39:0/4039047744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 msgr2=0x556cddd2dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d3fff700 1 --2- 172.21.15.39:0/4039047744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd2dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d3fff700 1 -- 172.21.15.39:0/4039047744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 msgr2=0x556cddd27f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d3fff700 1 --2- 172.21.15.39:0/4039047744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd27f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d3fff700 1 -- 172.21.15.39:0/4039047744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cddaf9350 con 0x556cddd28480 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.338+0000 7f82d3fff700 1 --2- 172.21.15.39:0/4039047744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd288a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009cc0 tx=0x7f82cc0075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e538e070dd4e2f47 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.339+0000 7f82b9ffb700 1 -- 172.21.15.39:0/4039047744 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc018070 con 0x556cddd28480 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.339+0000 7f82b9ffb700 1 -- 172.21.15.39:0/4039047744 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82cc00f050 con 0x556cddd28480 2024-05-24T11:41:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.339+0000 7f82b9ffb700 1 -- 172.21.15.39:0/4039047744 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc005830 con 0x556cddd28480 2024-05-24T11:41:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/4039047744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 msgr2=0x556cddd288a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/4039047744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd288a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009cc0 tx=0x7f82cc0075c0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/4039047744 shutdown_connections 2024-05-24T11:41:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/4039047744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd2dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/4039047744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd288a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/4039047744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd27f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/4039047744 >> 172.21.15.39:0/4039047744 conn(0x556cddc861b0 msgr2=0x556cddd22840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/4039047744 shutdown_connections 2024-05-24T11:41:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/4039047744 wait complete. 2024-05-24T11:41:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.340+0000 7f82e5c7fb80 1 -- start start 2024-05-24T11:41:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82e5c7fb80 1 Processor -- start 2024-05-24T11:41:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd34160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2d950 0x556cddd314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 0x556cddd31ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddc14500 con 0x556cddd28480 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddc141a0 con 0x556cddd2d950 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddafb640 con 0x556cddd2e270 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82e5c7fb80 10 client.0 _ll_register_callbacks cb 0x556cddc1dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2d950 0x556cddd314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82d37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd34160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2d950 0x556cddd314a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:47512/0 (socket says 172.21.15.39:47512) 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.341+0000 7f82d2ffd700 1 -- 172.21.15.39:0/1906950802 learned_addr learned my addr 172.21.15.39:0/1906950802 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.342+0000 7f82d3fff700 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 0x556cddd31ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.342+0000 7f82d2ffd700 1 -- 172.21.15.39:0/1906950802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 msgr2=0x556cddd31ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.342+0000 7f82d2ffd700 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 0x556cddd31ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.342+0000 7f82d2ffd700 1 -- 172.21.15.39:0/1906950802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 msgr2=0x556cddd34160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.342+0000 7f82d2ffd700 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd34160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.342+0000 7f82d2ffd700 1 -- 172.21.15.39:0/1906950802 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc009920 con 0x556cddd2d950 2024-05-24T11:41:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.342+0000 7f82d2ffd700 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2d950 0x556cddd314a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82c80116f0 tx=0x7f82c80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14efd8f2495924ba server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.342+0000 7f82b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:41:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.343+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1906950802 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8004030 con 0x556cddd2d950 2024-05-24T11:41:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.343+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1906950802 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f82c8017070 con 0x556cddd2d950 2024-05-24T11:41:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.343+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1906950802 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cddbd1210 con 0x556cddd2d950 2024-05-24T11:41:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.343+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1906950802 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c8004190 con 0x556cddd2d950 2024-05-24T11:41:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.343+0000 7f82e5c7fb80 10 client.25391 fetch_fsmap learned FSMap version 27 2024-05-24T11:41:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.343+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1906950802 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556cddaa5b00 con 0x556cddd2d950 2024-05-24T11:41:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.343+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1906950802 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556cddbd1210 con 0x556cddd2d950 2024-05-24T11:41:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.344+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1906950802 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f82c8003680 con 0x556cddd2d950 2024-05-24T11:41:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.344+0000 7f82e5c7fb80 10 client.25391 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:41:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.344+0000 7f82e5c7fb80 10 client.25391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:41:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.344+0000 7f82e5c7fb80 20 client.25391 populate_metadata read hostname 'smithi039' 2024-05-24T11:41:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.344+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556cddd41c50 0x556cddd44070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.344+0000 7f82e5c7fb80 4 client.25391 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.344+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1906950802 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cddcf77b0 con 0x556cddd41c50 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.345+0000 7f82d37fe700 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556cddd41c50 0x556cddd44070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.345+0000 7f82b2ffd700 10 client.25391 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.345+0000 7f82d37fe700 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556cddd41c50 0x556cddd44070 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4faa9b59550bc63 server_cookie=e11440c8ed050c77 in_seq=0 out_seq=0 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1906950802 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556cddcf77b0 con 0x556cddd41c50 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82b2ffd700 10 client.25391 handle_command_reply: tid=0 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 client.25391 shutdown 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1906950802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2d950 msgr2=0x556cddd314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2d950 0x556cddd314a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82c80116f0 tx=0x7f82c80047c0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1906950802 shutdown_connections 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 0x556cddd31ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556cddd41c50 0x556cddd44070 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd34160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1906950802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2d950 0x556cddd314a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1906950802 >> 172.21.15.39:0/1906950802 conn(0x556cddc861b0 msgr2=0x556cddd2c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82b2ffd700 0 client.25391 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1906950802 shutdown_connections 2024-05-24T11:41:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1906950802 wait complete. 2024-05-24T11:41:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.347+0000 7f82e5c7fb80 20 client.25391 trim_cache size 0 max 16384 2024-05-24T11:41:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.370+0000 7f82e5c7fb80 1 Processor -- start 2024-05-24T11:41:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.370+0000 7f82e5c7fb80 1 -- start start 2024-05-24T11:41:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.370+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd28480 0x556cddd31ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.370+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd2d950 0x556cddd24320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.370+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 0x556cddd22910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.370+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddd49280 con 0x556cddd2d950 2024-05-24T11:41:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.370+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddd2d390 con 0x556cddd28480 2024-05-24T11:41:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.370+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddd23a30 con 0x556cddd2e270 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d3fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 0x556cddd22910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd28480 0x556cddd31ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd28480 0x556cddd31ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:47518/0 (socket says 172.21.15.39:47518) 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d37fe700 1 -- 172.21.15.39:0/3530627127 learned_addr learned my addr 172.21.15.39:0/3530627127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d2ffd700 1 --2- 172.21.15.39:0/3530627127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd2d950 0x556cddd24320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d37fe700 1 -- 172.21.15.39:0/3530627127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 msgr2=0x556cddd22910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d37fe700 1 --2- 172.21.15.39:0/3530627127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 0x556cddd22910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d37fe700 1 -- 172.21.15.39:0/3530627127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd2d950 msgr2=0x556cddd24320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d37fe700 1 --2- 172.21.15.39:0/3530627127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd2d950 0x556cddd24320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d37fe700 1 -- 172.21.15.39:0/3530627127 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cddc3e500 con 0x556cddd28480 2024-05-24T11:41:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.371+0000 7f82d37fe700 1 --2- 172.21.15.39:0/3530627127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd28480 0x556cddd31ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82c409ab90 tx=0x7f82c4019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c63cbb18615770d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.372+0000 7f82ba7fc700 1 -- 172.21.15.39:0/3530627127 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40a2ff0 con 0x556cddd28480 2024-05-24T11:41:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.372+0000 7f82ba7fc700 1 -- 172.21.15.39:0/3530627127 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82c4098fa0 con 0x556cddd28480 2024-05-24T11:41:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.372+0000 7f82ba7fc700 1 -- 172.21.15.39:0/3530627127 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40cb630 con 0x556cddd28480 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/3530627127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd28480 msgr2=0x556cddd31ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/3530627127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd28480 0x556cddd31ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82c409ab90 tx=0x7f82c4019040 comp rx=0 tx=0).stop 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/3530627127 shutdown_connections 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/3530627127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2e270 0x556cddd22910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/3530627127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd28480 0x556cddd31ab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/3530627127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd2d950 0x556cddd24320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/3530627127 >> 172.21.15.39:0/3530627127 conn(0x556cddc861b0 msgr2=0x556cddd30d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/3530627127 shutdown_connections 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/3530627127 wait complete. 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.373+0000 7f82e5c7fb80 1 -- start start 2024-05-24T11:41:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82e5c7fb80 1 Processor -- start 2024-05-24T11:41:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd38c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd391b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82e5c7fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd36c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddd372a0 con 0x556cddd28480 2024-05-24T11:41:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddd37410 con 0x556cddd2e270 2024-05-24T11:41:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82e5c7fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cddd375d0 con 0x556cddd2d950 2024-05-24T11:41:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82e5c7fb80 10 client.0 _ll_register_callbacks cb 0x556cddc1dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd391b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:43018/0 (socket says 172.21.15.39:43018) 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82d2ffd700 1 -- 172.21.15.39:0/1150409045 learned_addr learned my addr 172.21.15.39:0/1150409045 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.374+0000 7f82d37fe700 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd38c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82d3fff700 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd36c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82d2ffd700 1 -- 172.21.15.39:0/1150409045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 msgr2=0x556cddd36c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82d2ffd700 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd36c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82d2ffd700 1 -- 172.21.15.39:0/1150409045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 msgr2=0x556cddd38c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82d2ffd700 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd38c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82d2ffd700 1 -- 172.21.15.39:0/1150409045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c40ba040 con 0x556cddd2d950 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82d3fff700 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd36c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:41:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82d2ffd700 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd391b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f82c8004790 tx=0x7f82c8010ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47229b81c595877f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:41:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:41:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1150409045 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c80043e0 con 0x556cddd2d950 2024-05-24T11:41:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.375+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1150409045 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f82c8017040 con 0x556cddd2d950 2024-05-24T11:41:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.376+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1150409045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82cc00f050 con 0x556cddd2d950 2024-05-24T11:41:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.376+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1150409045 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c8004540 con 0x556cddd2d950 2024-05-24T11:41:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.376+0000 7f82e5c7fb80 10 client.25303 fetch_fsmap learned FSMap version 27 2024-05-24T11:41:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.376+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1150409045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556cddafdc70 con 0x556cddd2d950 2024-05-24T11:41:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.376+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1150409045 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82cc00f050 con 0x556cddd2d950 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1150409045 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f82c801f8c0 con 0x556cddd2d950 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82e5c7fb80 10 client.25303 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82e5c7fb80 10 client.25303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82e5c7fb80 20 client.25303 populate_metadata read hostname 'smithi039' 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556cddd982a0 0x556cddd9a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82e5c7fb80 4 client.25303 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1150409045 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556cddc70e60 con 0x556cddd982a0 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82d37fe700 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556cddd982a0 0x556cddd9a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82b2ffd700 10 client.25303 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.377+0000 7f82d37fe700 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556cddd982a0 0x556cddd9a6c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d534e8ebb48943 server_cookie=a3098219fe559df1 in_seq=0 out_seq=0 2024-05-24T11:41:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82b2ffd700 1 -- 172.21.15.39:0/1150409045 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556cddc70e60 con 0x556cddd982a0 2024-05-24T11:41:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82b2ffd700 10 client.25303 handle_command_reply: tid=0 2024-05-24T11:41:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 client.25303 shutdown 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1150409045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 msgr2=0x556cddd391b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd391b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f82c8004790 tx=0x7f82c8010ad0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1150409045 shutdown_connections 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556cddd982a0 0x556cddd9a6c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556cddd2e270 0x556cddd36c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556cddd28480 0x556cddd38c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 --2- 172.21.15.39:0/1150409045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556cddd2d950 0x556cddd391b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1150409045 >> 172.21.15.39:0/1150409045 conn(0x556cddc861b0 msgr2=0x556cddd3d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82b2ffd700 0 client.25303 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:41:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.407+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1150409045 shutdown_connections 2024-05-24T11:41:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.408+0000 7f82e5c7fb80 1 -- 172.21.15.39:0/1150409045 wait complete. 2024-05-24T11:41:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.408+0000 7f82e5c7fb80 20 client.25303 trim_cache size 0 max 16384 2024-05-24T11:41:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:41:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:41:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:41:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 -- 172.21.15.39:0/3188080741 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f82bc0a13e0 msgr2=0x7f82bc0a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f82bc0a13e0 0x7f82bc0a38a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82d410dd80 tx=0x7f82c800b500 comp rx=0 tx=0).stop 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 -- 172.21.15.39:0/3188080741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82d4073340 msgr2=0x7f82d41115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82d4073340 0x7f82d41115a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82c400b2e0 tx=0x7f82c400fa00 comp rx=0 tx=0).stop 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 -- 172.21.15.39:0/3188080741 shutdown_connections 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82d4073340 0x7f82d41115a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f82bc0a13e0 0x7f82bc0a38a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d4111e90 0x7f82d410cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 --2- 172.21.15.39:0/3188080741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82d4073c60 0x7f82d410c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 -- 172.21.15.39:0/3188080741 >> 172.21.15.39:0/3188080741 conn(0x7f82d40fc9d0 msgr2=0x7f82d40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:41:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 -- 172.21.15.39:0/3188080741 shutdown_connections 2024-05-24T11:41:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:41:49.409+0000 7f82d9ac4700 1 -- 172.21.15.39:0/3188080741 wait complete. 2024-05-24T11:41:49.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:49 smithi053 ceph-mon[47448]: pgmap v111: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 27 MiB/s wr, 1.83k op/s 2024-05-24T11:41:49.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:49 smithi137 ceph-mon[47907]: pgmap v111: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 27 MiB/s wr, 1.83k op/s 2024-05-24T11:41:50.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:49 smithi039 ceph-mon[42919]: pgmap v111: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 27 MiB/s wr, 1.83k op/s 2024-05-24T11:41:51.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:51 smithi053 ceph-mon[47448]: pgmap v112: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:41:51.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:51 smithi137 ceph-mon[47907]: pgmap v112: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:41:52.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:51 smithi039 ceph-mon[42919]: pgmap v112: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:41:53.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:53 smithi053 ceph-mon[47448]: pgmap v113: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:41:53.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:53 smithi137 ceph-mon[47907]: pgmap v113: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:41:54.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:53 smithi039 ceph-mon[42919]: pgmap v113: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-05-24T11:41:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 150.0 s, 1309.7 tps, lat 76.499 ms stddev 21.482 2024-05-24T11:41:55.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:55 smithi053 ceph-mon[47448]: pgmap v114: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 26 MiB/s wr, 1.89k op/s 2024-05-24T11:41:55.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:55 smithi137 ceph-mon[47907]: pgmap v114: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 26 MiB/s wr, 1.89k op/s 2024-05-24T11:41:56.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:55 smithi039 ceph-mon[42919]: pgmap v114: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 26 MiB/s wr, 1.89k op/s 2024-05-24T11:41:57.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:57 smithi137 ceph-mon[47907]: pgmap v115: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 17 MiB/s wr, 1.29k op/s 2024-05-24T11:41:58.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:57 smithi039 ceph-mon[42919]: pgmap v115: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 17 MiB/s wr, 1.29k op/s 2024-05-24T11:41:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:57 smithi053 ceph-mon[47448]: pgmap v115: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 17 MiB/s wr, 1.29k op/s 2024-05-24T11:41:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 155.0 s, 1273.6 tps, lat 78.537 ms stddev 23.922 2024-05-24T11:42:00.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:41:59 smithi039 ceph-mon[42919]: pgmap v116: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 24 MiB/s wr, 1.86k op/s 2024-05-24T11:42:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:41:59 smithi053 ceph-mon[47448]: pgmap v116: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 24 MiB/s wr, 1.86k op/s 2024-05-24T11:42:00.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:41:59 smithi137 ceph-mon[47907]: pgmap v116: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 24 MiB/s wr, 1.86k op/s 2024-05-24T11:42:02.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:01 smithi039 ceph-mon[42919]: pgmap v117: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:01 smithi053 ceph-mon[47448]: pgmap v117: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:02.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:01 smithi137 ceph-mon[47907]: pgmap v117: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.584+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/2194679103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dd0107ec0 msgr2=0x7f4dd0114380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.584+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/2194679103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dd0107ec0 0x7f4dd0114380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800b0d0 tx=0x7f4dc800b3e0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.584+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/2194679103 shutdown_connections 2024-05-24T11:42:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.584+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/2194679103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dd010d5c0 0x7f4dd010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.584+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/2194679103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dd0107ec0 0x7f4dd0114380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.584+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/2194679103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dd0107500 0x7f4dd0107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.584+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/2194679103 >> 172.21.15.39:0/2194679103 conn(0x7f4dd0076dc0 msgr2=0x7f4dd0105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.585+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/2194679103 shutdown_connections 2024-05-24T11:42:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.586+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/2194679103 wait complete. 2024-05-24T11:42:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd7ef8700 1 Processor -- start 2024-05-24T11:42:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd7ef8700 1 -- start start 2024-05-24T11:42:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd7ef8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dd0107500 0x7f4dd01a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd7ef8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dd0107ec0 0x7f4dd01a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd7ef8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dd010d5c0 0x7f4dd019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd7ef8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd019dd60 con 0x7f4dd0107500 2024-05-24T11:42:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd7ef8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd019ded0 con 0x7f4dd010d5c0 2024-05-24T11:42:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd7ef8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd019e090 con 0x7f4dd0107ec0 2024-05-24T11:42:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd5493700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dd0107ec0 0x7f4dd01a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd6495700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dd010d5c0 0x7f4dd019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd5493700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dd0107ec0 0x7f4dd01a3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:48672/0 (socket says 172.21.15.39:48672) 2024-05-24T11:42:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.587+0000 7f4dd5493700 1 -- 172.21.15.39:0/961609382 learned_addr learned my addr 172.21.15.39:0/961609382 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.588+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dd0107500 0x7f4dd01a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.588+0000 7f4dd5493700 1 -- 172.21.15.39:0/961609382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dd010d5c0 msgr2=0x7f4dd019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.588+0000 7f4dd5493700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dd010d5c0 0x7f4dd019d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.588+0000 7f4dd5493700 1 -- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dd0107500 msgr2=0x7f4dd01a35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.588+0000 7f4dd5493700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dd0107500 0x7f4dd01a35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.588+0000 7f4dd5493700 1 -- 172.21.15.39:0/961609382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc8009d00 con 0x7f4dd0107ec0 2024-05-24T11:42:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.588+0000 7f4dd5493700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dd0107ec0 0x7f4dd01a3b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00b550 tx=0x7f4dcc00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.589+0000 7f4dc6ffd700 1 -- 172.21.15.39:0/961609382 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc011830 con 0x7f4dd0107ec0 2024-05-24T11:42:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.589+0000 7f4dc6ffd700 1 -- 172.21.15.39:0/961609382 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dcc011e20 con 0x7f4dd0107ec0 2024-05-24T11:42:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.589+0000 7f4dc6ffd700 1 -- 172.21.15.39:0/961609382 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc010d70 con 0x7f4dd0107ec0 2024-05-24T11:42:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.589+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/961609382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd019e370 con 0x7f4dd0107ec0 2024-05-24T11:42:02.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.589+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/961609382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd01aa5c0 con 0x7f4dd0107ec0 2024-05-24T11:42:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.591+0000 7f4dc6ffd700 1 -- 172.21.15.39:0/961609382 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4dcc011990 con 0x7f4dd0107ec0 2024-05-24T11:42:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.592+0000 7f4dc6ffd700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4dbc0a1310 0x7f4dbc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.592+0000 7f4dc6ffd700 1 -- 172.21.15.39:0/961609382 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4dcc0cc580 con 0x7f4dd0107ec0 2024-05-24T11:42:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.593+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4dbc0a1310 0x7f4dbc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.593+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4dbc0a1310 0x7f4dbc0a37d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006fd0 tx=0x7f4dc0009190 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.596+0000 7f4de40b3b80 1 Processor -- start 2024-05-24T11:42:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.596+0000 7f4de40b3b80 1 -- start start 2024-05-24T11:42:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 0x561c6aa40d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa3af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa3b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6a9920f0 con 0x561c6aa3b480 2024-05-24T11:42:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6a98fab0 con 0x561c6aa40950 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6a98b3a0 con 0x561c6aa41270 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd5493700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa3af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd5493700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa3af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:48686/0 (socket says 172.21.15.39:48686) 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd5493700 1 -- 172.21.15.39:0/486474341 learned_addr learned my addr 172.21.15.39:0/486474341 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 0x561c6aa40d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd6495700 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa3b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd5493700 1 -- 172.21.15.39:0/486474341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 msgr2=0x561c6aa40d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd5493700 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 0x561c6aa40d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd5493700 1 -- 172.21.15.39:0/486474341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 msgr2=0x561c6aa3b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd5493700 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa3b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd5493700 1 -- 172.21.15.39:0/486474341 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c6a80c350 con 0x561c6aa41270 2024-05-24T11:42:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.597+0000 7f4dd6495700 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa3b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:42:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.598+0000 7f4dd5493700 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa3af40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0a1730 tx=0x7f4dcc0995b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76f40a3543c6163a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.598+0000 7f4dbbfff700 1 -- 172.21.15.39:0/486474341 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc011c40 con 0x561c6aa41270 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.598+0000 7f4dbbfff700 1 -- 172.21.15.39:0/486474341 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dcc099ee0 con 0x561c6aa41270 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.598+0000 7f4dbbfff700 1 -- 172.21.15.39:0/486474341 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc099a40 con 0x561c6aa41270 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.599+0000 7f4de40b3b80 1 -- 172.21.15.39:0/486474341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 msgr2=0x561c6aa3af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.599+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa3af40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0a1730 tx=0x7f4dcc0995b0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.599+0000 7f4de40b3b80 1 -- 172.21.15.39:0/486474341 shutdown_connections 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.599+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa3b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.599+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 0x561c6aa40d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.599+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/486474341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa3af40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.599+0000 7f4de40b3b80 1 -- 172.21.15.39:0/486474341 >> 172.21.15.39:0/486474341 conn(0x561c6a9991b0 msgr2=0x561c6aa35840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.599+0000 7f4de40b3b80 1 -- 172.21.15.39:0/486474341 shutdown_connections 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.599+0000 7f4de40b3b80 1 -- 172.21.15.39:0/486474341 wait complete. 2024-05-24T11:42:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.600+0000 7f4de40b3b80 1 -- start start 2024-05-24T11:42:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.600+0000 7f4de40b3b80 1 Processor -- start 2024-05-24T11:42:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa47250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa44590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 0x561c6aa44cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6a927500 con 0x561c6aa3b480 2024-05-24T11:42:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6a9271a0 con 0x561c6aa41270 2024-05-24T11:42:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6a80e640 con 0x561c6aa40950 2024-05-24T11:42:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4de40b3b80 10 client.0 _ll_register_callbacks cb 0x561c6a930ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4dd5493700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa44590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4dd5493700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa44590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:48690/0 (socket says 172.21.15.39:48690) 2024-05-24T11:42:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4dd5493700 1 -- 172.21.15.39:0/3039368271 learned_addr learned my addr 172.21.15.39:0/3039368271 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.601+0000 7f4dd6495700 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 0x561c6aa44cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa47250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4dd5493700 1 -- 172.21.15.39:0/3039368271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 msgr2=0x561c6aa44cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4dd5493700 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 0x561c6aa44cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4dd5493700 1 -- 172.21.15.39:0/3039368271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 msgr2=0x561c6aa47250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4dd5493700 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa47250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4dd5493700 1 -- 172.21.15.39:0/3039368271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd01aa5c0 con 0x561c6aa40950 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa47250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4dd5493700 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa44590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00be30 tx=0x7f4dcc011c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b76cb6adaae11b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4db97fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4db97fa700 1 -- 172.21.15.39:0/3039368271 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0a1c60 con 0x561c6aa40950 2024-05-24T11:42:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3039368271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c6a8e4210 con 0x561c6aa40950 2024-05-24T11:42:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4db97fa700 1 -- 172.21.15.39:0/3039368271 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4dcc0b9040 con 0x561c6aa40950 2024-05-24T11:42:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4db97fa700 1 -- 172.21.15.39:0/3039368271 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dcc0b9200 con 0x561c6aa40950 2024-05-24T11:42:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4de40b3b80 10 client.25312 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.602+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3039368271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561c6a7b8b00 con 0x561c6aa40950 2024-05-24T11:42:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.603+0000 7f4db97fa700 1 -- 172.21.15.39:0/3039368271 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561c6a8e4210 con 0x561c6aa40950 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.603+0000 7f4db97fa700 1 -- 172.21.15.39:0/3039368271 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4dcc099b20 con 0x561c6aa40950 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.603+0000 7f4de40b3b80 10 client.25312 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.603+0000 7f4de40b3b80 10 client.25312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.603+0000 7f4de40b3b80 20 client.25312 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.603+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561c6aa657e0 0x561c6aa67c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.603+0000 7f4de40b3b80 4 client.25312 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.603+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3039368271 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c6aa0a7b0 con 0x561c6aa657e0 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.604+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561c6aa657e0 0x561c6aa67c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.604+0000 7f4db97fa700 10 client.25312 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.604+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561c6aa657e0 0x561c6aa67c00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4a1caa38195399 server_cookie=951e88c1bad13295 in_seq=0 out_seq=0 2024-05-24T11:42:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4db97fa700 1 -- 172.21.15.39:0/3039368271 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561c6aa0a7b0 con 0x561c6aa657e0 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4db97fa700 10 client.25312 handle_command_reply: tid=0 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4de40b3b80 1 client.25312 shutdown 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3039368271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 msgr2=0x561c6aa44590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa44590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00be30 tx=0x7f4dcc011c40 comp rx=0 tx=0).stop 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3039368271 shutdown_connections 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561c6aa657e0 0x561c6aa67c00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 0x561c6aa44cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa47250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3039368271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa44590 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4db97fa700 0 client.25312 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.605+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3039368271 >> 172.21.15.39:0/3039368271 conn(0x561c6a9991b0 msgr2=0x561c6aa39f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.606+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3039368271 shutdown_connections 2024-05-24T11:42:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.606+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3039368271 wait complete. 2024-05-24T11:42:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.606+0000 7f4de40b3b80 20 client.25312 trim_cache size 0 max 16384 2024-05-24T11:42:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.626+0000 7f4de40b3b80 1 Processor -- start 2024-05-24T11:42:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.626+0000 7f4de40b3b80 1 -- start start 2024-05-24T11:42:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa44550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 0x561c6aa35910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa37540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6aa37a80 con 0x561c6aa3b480 2024-05-24T11:42:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6aa47810 con 0x561c6aa40950 2024-05-24T11:42:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6aa42ae0 con 0x561c6aa41270 2024-05-24T11:42:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd6495700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa37540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd5c94700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa44550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd6495700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa37540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:48698/0 (socket says 172.21.15.39:48698) 2024-05-24T11:42:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd5c94700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa44550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39594/0 (socket says 172.21.15.39:39594) 2024-05-24T11:42:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd6495700 1 -- 172.21.15.39:0/3556006967 learned_addr learned my addr 172.21.15.39:0/3556006967 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd5493700 1 --2- 172.21.15.39:0/3556006967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 0x561c6aa35910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd6495700 1 -- 172.21.15.39:0/3556006967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 msgr2=0x561c6aa35910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd6495700 1 --2- 172.21.15.39:0/3556006967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 0x561c6aa35910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd6495700 1 -- 172.21.15.39:0/3556006967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 msgr2=0x561c6aa44550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd6495700 1 --2- 172.21.15.39:0/3556006967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa44550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd6495700 1 -- 172.21.15.39:0/3556006967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c6a951500 con 0x561c6aa41270 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.627+0000 7f4dd6495700 1 --2- 172.21.15.39:0/3556006967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa37540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8007e80 tx=0x7f4dc800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44a57d63070134a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.628+0000 7f4dc4ff9700 1 -- 172.21.15.39:0/3556006967 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800e040 con 0x561c6aa41270 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.628+0000 7f4dc4ff9700 1 -- 172.21.15.39:0/3556006967 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dc8021070 con 0x561c6aa41270 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.628+0000 7f4dc4ff9700 1 -- 172.21.15.39:0/3556006967 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8007970 con 0x561c6aa41270 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.628+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3556006967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 msgr2=0x561c6aa37540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.628+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3556006967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa37540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8007e80 tx=0x7f4dc800b040 comp rx=0 tx=0).stop 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.629+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3556006967 shutdown_connections 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.629+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3556006967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa41270 0x561c6aa37540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.629+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3556006967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa44550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.629+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/3556006967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa40950 0x561c6aa35910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.629+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3556006967 >> 172.21.15.39:0/3556006967 conn(0x561c6a9991b0 msgr2=0x561c6aa6c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.629+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3556006967 shutdown_connections 2024-05-24T11:42:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.629+0000 7f4de40b3b80 1 -- 172.21.15.39:0/3556006967 wait complete. 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.629+0000 7f4de40b3b80 1 -- start start 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.629+0000 7f4de40b3b80 1 Processor -- start 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa49b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa4a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4de40b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 0x561c6aa4a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6aabaca0 con 0x561c6aa3b480 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6aabade0 con 0x561c6aa41270 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4de40b3b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6aabaf20 con 0x561c6aa40950 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4de40b3b80 10 client.0 _ll_register_callbacks cb 0x561c6a930ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4dd5c94700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa49b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4dd5c94700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa49b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39604/0 (socket says 172.21.15.39:39604) 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4dd5c94700 1 -- 172.21.15.39:0/1489791137 learned_addr learned my addr 172.21.15.39:0/1489791137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4dd6495700 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 0x561c6aa4a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.630+0000 7f4dd5493700 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa4a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4dd5493700 1 -- 172.21.15.39:0/1489791137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 msgr2=0x561c6aa4a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4dd5493700 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 0x561c6aa4a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4dd5493700 1 -- 172.21.15.39:0/1489791137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 msgr2=0x561c6aa49b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4dd5493700 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa49b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4dd5493700 1 -- 172.21.15.39:0/1489791137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd0114c80 con 0x561c6aa40950 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4dd5493700 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa4a0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0994e0 tx=0x7f4dcc097d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30c941311689a081 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4db97fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4db97fa700 1 -- 172.21.15.39:0/1489791137 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc098ed0 con 0x561c6aa40950 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4db97fa700 1 -- 172.21.15.39:0/1489791137 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4dcc0a1990 con 0x561c6aa40950 2024-05-24T11:42:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4de40b3b80 1 -- 172.21.15.39:0/1489791137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dcc099ee0 con 0x561c6aa40950 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4db97fa700 1 -- 172.21.15.39:0/1489791137 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dcc095b20 con 0x561c6aa40950 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4de40b3b80 10 client.25318 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4de40b3b80 1 -- 172.21.15.39:0/1489791137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561c6a810c70 con 0x561c6aa40950 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.631+0000 7f4db97fa700 1 -- 172.21.15.39:0/1489791137 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dcc099ee0 con 0x561c6aa40950 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.632+0000 7f4db97fa700 1 -- 172.21.15.39:0/1489791137 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4dcc09d170 con 0x561c6aa40950 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.632+0000 7f4de40b3b80 10 client.25318 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.632+0000 7f4de40b3b80 10 client.25318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.632+0000 7f4de40b3b80 20 client.25318 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.632+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561c6aa50cb0 0x561c6aa530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.632+0000 7f4de40b3b80 4 client.25318 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.632+0000 7f4de40b3b80 1 -- 172.21.15.39:0/1489791137 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561c6a983e60 con 0x561c6aa50cb0 2024-05-24T11:42:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.632+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561c6aa50cb0 0x561c6aa530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.633+0000 7f4dd5c94700 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561c6aa50cb0 0x561c6aa530a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b69627dda14f5f server_cookie=70a20300765fdb6c in_seq=0 out_seq=0 2024-05-24T11:42:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.633+0000 7f4db97fa700 10 client.25318 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.698+0000 7f4db97fa700 1 -- 172.21.15.39:0/1489791137 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561c6a983e60 con 0x561c6aa50cb0 2024-05-24T11:42:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.698+0000 7f4db97fa700 10 client.25318 handle_command_reply: tid=0 2024-05-24T11:42:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.698+0000 7f4de40b3b80 1 client.25318 shutdown 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 -- 172.21.15.39:0/1489791137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 msgr2=0x561c6aa4a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa4a0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0994e0 tx=0x7f4dcc097d90 comp rx=0 tx=0).stop 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 -- 172.21.15.39:0/1489791137 shutdown_connections 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561c6aa50cb0 0x561c6aa530a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c6aa41270 0x561c6aa4a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c6aa3b480 0x561c6aa49b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 --2- 172.21.15.39:0/1489791137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c6aa40950 0x561c6aa4a0d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 -- 172.21.15.39:0/1489791137 >> 172.21.15.39:0/1489791137 conn(0x561c6a9991b0 msgr2=0x561c6aa3e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4db97fa700 0 client.25318 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 -- 172.21.15.39:0/1489791137 shutdown_connections 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 1 -- 172.21.15.39:0/1489791137 wait complete. 2024-05-24T11:42:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.699+0000 7f4de40b3b80 20 client.25318 trim_cache size 0 max 16384 2024-05-24T11:42:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:42:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:42:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:42:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:42:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.700+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4dbc0a1310 msgr2=0x7f4dbc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.700+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4dbc0a1310 0x7f4dbc0a37d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006fd0 tx=0x7f4dc0009190 comp rx=0 tx=0).stop 2024-05-24T11:42:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.700+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/961609382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dd0107ec0 msgr2=0x7f4dd01a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.700+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dd0107ec0 0x7f4dd01a3b10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00b550 tx=0x7f4dcc00b860 comp rx=0 tx=0).stop 2024-05-24T11:42:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.700+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/961609382 shutdown_connections 2024-05-24T11:42:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.701+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4dbc0a1310 0x7f4dbc0a37d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.701+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dd010d5c0 0x7f4dd019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.701+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dd0107500 0x7f4dd01a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.701+0000 7f4dd7ef8700 1 --2- 172.21.15.39:0/961609382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dd0107ec0 0x7f4dd01a3b10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.701+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/961609382 >> 172.21.15.39:0/961609382 conn(0x7f4dd0076dc0 msgr2=0x7f4dd0077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.701+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/961609382 shutdown_connections 2024-05-24T11:42:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:02.701+0000 7f4dd7ef8700 1 -- 172.21.15.39:0/961609382 wait complete. 2024-05-24T11:42:04.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:03 smithi039 ceph-mon[42919]: pgmap v118: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:04.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 160.0 s, 1338.2 tps, lat 74.723 ms stddev 25.701 2024-05-24T11:42:04.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:03 smithi053 ceph-mon[47448]: pgmap v118: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:04.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:03 smithi137 ceph-mon[47907]: pgmap v118: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:06.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:05 smithi039 ceph-mon[42919]: pgmap v119: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 23 MiB/s wr, 1.88k op/s 2024-05-24T11:42:06.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:05 smithi053 ceph-mon[47448]: pgmap v119: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 23 MiB/s wr, 1.88k op/s 2024-05-24T11:42:06.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:05 smithi137 ceph-mon[47907]: pgmap v119: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 23 MiB/s wr, 1.88k op/s 2024-05-24T11:42:08.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:07 smithi039 ceph-mon[42919]: pgmap v120: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-05-24T11:42:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:07 smithi053 ceph-mon[47448]: pgmap v120: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-05-24T11:42:08.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:07 smithi137 ceph-mon[47907]: pgmap v120: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2024-05-24T11:42:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 165.0 s, 1307.0 tps, lat 76.257 ms stddev 25.496 2024-05-24T11:42:10.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:09 smithi039 ceph-mon[42919]: pgmap v121: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-05-24T11:42:10.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:09 smithi053 ceph-mon[47448]: pgmap v121: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-05-24T11:42:10.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:09 smithi137 ceph-mon[47907]: pgmap v121: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-05-24T11:42:12.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:11 smithi039 ceph-mon[42919]: pgmap v122: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:42:12.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:11 smithi053 ceph-mon[47448]: pgmap v122: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:42:12.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:11 smithi137 ceph-mon[47907]: pgmap v122: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:42:14.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:13 smithi039 ceph-mon[42919]: pgmap v123: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:42:14.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 170.0 s, 1268.4 tps, lat 79.087 ms stddev 41.688 2024-05-24T11:42:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:13 smithi053 ceph-mon[47448]: pgmap v123: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:42:14.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:13 smithi137 ceph-mon[47907]: pgmap v123: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:42:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.871+0000 7f5ac2a45700 1 -- 172.21.15.39:0/1633993422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc107ea0 msgr2=0x7f5abc114360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.871+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/1633993422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc107ea0 0x7f5abc114360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400b0d0 tx=0x7f5ab400b3e0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.871+0000 7f5ac2a45700 1 -- 172.21.15.39:0/1633993422 shutdown_connections 2024-05-24T11:42:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.871+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/1633993422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5abc10d5a0 0x7f5abc10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.871+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/1633993422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc107ea0 0x7f5abc114360 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.871+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/1633993422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5abc1074e0 0x7f5abc107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.871+0000 7f5ac2a45700 1 -- 172.21.15.39:0/1633993422 >> 172.21.15.39:0/1633993422 conn(0x7f5abc076dc0 msgr2=0x7f5abc105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.872+0000 7f5ac2a45700 1 -- 172.21.15.39:0/1633993422 shutdown_connections 2024-05-24T11:42:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.872+0000 7f5ac2a45700 1 -- 172.21.15.39:0/1633993422 wait complete. 2024-05-24T11:42:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5ac2a45700 1 Processor -- start 2024-05-24T11:42:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5ac2a45700 1 -- start start 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5ac2a45700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5abc1074e0 0x7f5abc1a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5ac2a45700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5abc107ea0 0x7f5abc1a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5ac2a45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc10d5a0 0x7f5abc19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5ac2a45700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc19dd70 con 0x7f5abc10d5a0 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5ac2a45700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc19dee0 con 0x7f5abc1074e0 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5ac2a45700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc19e0a0 con 0x7f5abc107ea0 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5abb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5abc107ea0 0x7f5abc1a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5abbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5abc1074e0 0x7f5abc1a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5abb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5abc107ea0 0x7f5abc1a3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59626/0 (socket says 172.21.15.39:59626) 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.873+0000 7f5abb7fe700 1 -- 172.21.15.39:0/3781278606 learned_addr learned my addr 172.21.15.39:0/3781278606 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5ac0fe2700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc10d5a0 0x7f5abc19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5abb7fe700 1 -- 172.21.15.39:0/3781278606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5abc1074e0 msgr2=0x7f5abc1a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5abc1074e0 0x7f5abc1a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5abb7fe700 1 -- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc10d5a0 msgr2=0x7f5abc19d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc10d5a0 0x7f5abc19d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5abb7fe700 1 -- 172.21.15.39:0/3781278606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab4009d00 con 0x7f5abc107ea0 2024-05-24T11:42:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5abc107ea0 0x7f5abc1a3b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00d980 tx=0x7f5aac00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5ab97fa700 1 -- 172.21.15.39:0/3781278606 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac004220 con 0x7f5abc107ea0 2024-05-24T11:42:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5ac2a45700 1 -- 172.21.15.39:0/3781278606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc19e380 con 0x7f5abc107ea0 2024-05-24T11:42:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5ab97fa700 1 -- 172.21.15.39:0/3781278606 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aac007640 con 0x7f5abc107ea0 2024-05-24T11:42:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.874+0000 7f5ab97fa700 1 -- 172.21.15.39:0/3781278606 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac005660 con 0x7f5abc107ea0 2024-05-24T11:42:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.875+0000 7f5ac2a45700 1 -- 172.21.15.39:0/3781278606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc06aba0 con 0x7f5abc107ea0 2024-05-24T11:42:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.878+0000 7f5ab97fa700 1 -- 172.21.15.39:0/3781278606 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5aac004750 con 0x7f5abc107ea0 2024-05-24T11:42:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.879+0000 7f5ab97fa700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5aa40a1310 0x7f5aa40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.879+0000 7f5ab97fa700 1 -- 172.21.15.39:0/3781278606 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5aac0cca40 con 0x7f5abc107ea0 2024-05-24T11:42:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.879+0000 7f5abbfff700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5aa40a1310 0x7f5aa40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5abbfff700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5aa40a1310 0x7f5aa40a37d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009be0 tx=0x7f5ab0009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5acec00b80 1 Processor -- start 2024-05-24T11:42:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5acec00b80 1 -- start start 2024-05-24T11:42:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6e950 0x56416ad6ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6f270 0x56416ad68f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad69480 0x56416ad698a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5acec00b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416acc00f0 con 0x56416ad6e950 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5acec00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416acbdab0 con 0x56416ad69480 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5acec00b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416acb93a0 con 0x56416ad6f270 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5abbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6e950 0x56416ad6ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5ac0fe2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad69480 0x56416ad698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5abb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6f270 0x56416ad68f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5abb7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6f270 0x56416ad68f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59638/0 (socket says 172.21.15.39:59638) 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5abb7fe700 1 -- 172.21.15.39:0/3470534326 learned_addr learned my addr 172.21.15.39:0/3470534326 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.880+0000 7f5abbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6e950 0x56416ad6ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37014/0 (socket says 172.21.15.39:37014) 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.881+0000 7f5abb7fe700 1 -- 172.21.15.39:0/3470534326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad69480 msgr2=0x56416ad698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.881+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/3470534326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad69480 0x56416ad698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.881+0000 7f5abb7fe700 1 -- 172.21.15.39:0/3470534326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6e950 msgr2=0x56416ad6ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.881+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/3470534326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6e950 0x56416ad6ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.881+0000 7f5abb7fe700 1 -- 172.21.15.39:0/3470534326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56416ab3a350 con 0x56416ad6f270 2024-05-24T11:42:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.881+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/3470534326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6f270 0x56416ad68f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0965a0 tx=0x7f5aac004380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce4e23e91efa0075 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.881+0000 7f5aa27fc700 1 -- 172.21.15.39:0/3470534326 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac099b30 con 0x56416ad6f270 2024-05-24T11:42:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.881+0000 7f5aa27fc700 1 -- 172.21.15.39:0/3470534326 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aac09d960 con 0x56416ad6f270 2024-05-24T11:42:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.881+0000 7f5aa27fc700 1 -- 172.21.15.39:0/3470534326 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac099280 con 0x56416ad6f270 2024-05-24T11:42:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.882+0000 7f5acec00b80 1 -- 172.21.15.39:0/3470534326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6f270 msgr2=0x56416ad68f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.882+0000 7f5acec00b80 1 --2- 172.21.15.39:0/3470534326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6f270 0x56416ad68f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0965a0 tx=0x7f5aac004380 comp rx=0 tx=0).stop 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.882+0000 7f5acec00b80 1 -- 172.21.15.39:0/3470534326 shutdown_connections 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.882+0000 7f5acec00b80 1 --2- 172.21.15.39:0/3470534326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad69480 0x56416ad698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.882+0000 7f5acec00b80 1 --2- 172.21.15.39:0/3470534326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6e950 0x56416ad6ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.882+0000 7f5acec00b80 1 --2- 172.21.15.39:0/3470534326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6f270 0x56416ad68f40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.882+0000 7f5acec00b80 1 -- 172.21.15.39:0/3470534326 >> 172.21.15.39:0/3470534326 conn(0x56416acc71b0 msgr2=0x56416ad63840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.882+0000 7f5acec00b80 1 -- 172.21.15.39:0/3470534326 shutdown_connections 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.882+0000 7f5acec00b80 1 -- 172.21.15.39:0/3470534326 wait complete. 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5acec00b80 1 -- start start 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5acec00b80 1 Processor -- start 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad75260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 0x56416ad72550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad72c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5acec00b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416ac55500 con 0x56416ad6f270 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5acec00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416ac551a0 con 0x56416ad6e950 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5acec00b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416ab3c640 con 0x56416ad69480 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5acec00b80 10 client.0 _ll_register_callbacks cb 0x56416ac5ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.883+0000 7f5ac0fe2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad72c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5ac0fe2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad72c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37024/0 (socket says 172.21.15.39:37024) 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5ac0fe2700 1 -- 172.21.15.39:0/589471247 learned_addr learned my addr 172.21.15.39:0/589471247 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5abbfff700 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad75260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5ac0fe2700 1 -- 172.21.15.39:0/589471247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 msgr2=0x56416ad75260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5ac0fe2700 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad75260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5ac0fe2700 1 -- 172.21.15.39:0/589471247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 msgr2=0x56416ad72550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5ac0fe2700 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 0x56416ad72550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5ac0fe2700 1 -- 172.21.15.39:0/589471247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc06aba0 con 0x56416ad6f270 2024-05-24T11:42:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5ac0fe2700 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad72c90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4015000 tx=0x7f5ab400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74de434c04b11817 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.884+0000 7f5a9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:42:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/589471247 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400e040 con 0x56416ad6f270 2024-05-24T11:42:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/589471247 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5ab4003a00 con 0x56416ad6f270 2024-05-24T11:42:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5acec00b80 1 -- 172.21.15.39:0/589471247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56416ac12210 con 0x56416ad6f270 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/589471247 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ab40078f0 con 0x56416ad6f270 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5acec00b80 10 client.15732 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5acec00b80 1 -- 172.21.15.39:0/589471247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56416aae6b00 con 0x56416ad6f270 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/589471247 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56416ac12210 con 0x56416ad6f270 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/589471247 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5ab4019030 con 0x56416ad6f270 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5acec00b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5acec00b80 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5acec00b80 20 client.15732 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5acec00b80 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56416ad93b70 0x56416ad95f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5acec00b80 4 client.15732 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.885+0000 7f5acec00b80 1 -- 172.21.15.39:0/589471247 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56416ad387b0 con 0x56416ad93b70 2024-05-24T11:42:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.886+0000 7f5abbfff700 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56416ad93b70 0x56416ad95f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.886+0000 7f5abbfff700 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56416ad93b70 0x56416ad95f90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd40c6675a96ba server_cookie=2f9ab3856c95e938 in_seq=0 out_seq=0 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.886+0000 7f5a9b7fe700 10 client.15732 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/589471247 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56416ad387b0 con 0x56416ad93b70 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5a9b7fe700 10 client.15732 handle_command_reply: tid=0 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 client.15732 shutdown 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 -- 172.21.15.39:0/589471247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 msgr2=0x56416ad72c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad72c90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4015000 tx=0x7f5ab400bd80 comp rx=0 tx=0).stop 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 -- 172.21.15.39:0/589471247 shutdown_connections 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad75260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56416ad93b70 0x56416ad95f90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad72c90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 --2- 172.21.15.39:0/589471247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 0x56416ad72550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 -- 172.21.15.39:0/589471247 >> 172.21.15.39:0/589471247 conn(0x56416acc71b0 msgr2=0x56416ad67f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5a9b7fe700 0 client.15732 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 -- 172.21.15.39:0/589471247 shutdown_connections 2024-05-24T11:42:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.887+0000 7f5acec00b80 1 -- 172.21.15.39:0/589471247 wait complete. 2024-05-24T11:42:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.888+0000 7f5acec00b80 20 client.15732 trim_cache size 0 max 16384 2024-05-24T11:42:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.910+0000 7f5acec00b80 1 Processor -- start 2024-05-24T11:42:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5acec00b80 1 -- start start 2024-05-24T11:42:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad69480 0x56416ad6ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6e950 0x56416ad72510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6f270 0x56416ad63910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5acec00b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416ad654b0 con 0x56416ad69480 2024-05-24T11:42:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5acec00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416ad65290 con 0x56416ad6f270 2024-05-24T11:42:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5acec00b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416ad64580 con 0x56416ad6e950 2024-05-24T11:42:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5abbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad69480 0x56416ad6ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5abbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad69480 0x56416ad6ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37038/0 (socket says 172.21.15.39:37038) 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.911+0000 7f5abbfff700 1 -- 172.21.15.39:0/3728281590 learned_addr learned my addr 172.21.15.39:0/3728281590 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.912+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6e950 0x56416ad72510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.912+0000 7f5abbfff700 1 -- 172.21.15.39:0/3728281590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6e950 msgr2=0x56416ad72510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.912+0000 7f5abbfff700 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6e950 0x56416ad72510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.912+0000 7f5abbfff700 1 -- 172.21.15.39:0/3728281590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6f270 msgr2=0x56416ad63910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.912+0000 7f5ac0fe2700 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6f270 0x56416ad63910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.912+0000 7f5abbfff700 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6f270 0x56416ad63910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.912+0000 7f5abbfff700 1 -- 172.21.15.39:0/3728281590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56416ac7f500 con 0x56416ad69480 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.912+0000 7f5ac0fe2700 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6f270 0x56416ad63910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:42:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.912+0000 7f5abbfff700 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad69480 0x56416ad6ed30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x56416ad96870 tx=0x7f5ab0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fcb7a84d0ac603 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.913+0000 7f5aa2ffd700 1 -- 172.21.15.39:0/3728281590 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0025070 con 0x56416ad69480 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.913+0000 7f5aa2ffd700 1 -- 172.21.15.39:0/3728281590 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ab0027e90 con 0x56416ad69480 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.913+0000 7f5aa2ffd700 1 -- 172.21.15.39:0/3728281590 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0027930 con 0x56416ad69480 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.914+0000 7f5acec00b80 1 -- 172.21.15.39:0/3728281590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad69480 msgr2=0x56416ad6ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.914+0000 7f5acec00b80 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad69480 0x56416ad6ed30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x56416ad96870 tx=0x7f5ab0014000 comp rx=0 tx=0).stop 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.914+0000 7f5acec00b80 1 -- 172.21.15.39:0/3728281590 shutdown_connections 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.914+0000 7f5acec00b80 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6f270 0x56416ad63910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.914+0000 7f5acec00b80 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad69480 0x56416ad6ed30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.914+0000 7f5acec00b80 1 --2- 172.21.15.39:0/3728281590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad6e950 0x56416ad72510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.914+0000 7f5acec00b80 1 -- 172.21.15.39:0/3728281590 >> 172.21.15.39:0/3728281590 conn(0x56416acc71b0 msgr2=0x56416ad687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.914+0000 7f5acec00b80 1 -- 172.21.15.39:0/3728281590 shutdown_connections 2024-05-24T11:42:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.914+0000 7f5acec00b80 1 -- 172.21.15.39:0/3728281590 wait complete. 2024-05-24T11:42:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.915+0000 7f5acec00b80 1 -- start start 2024-05-24T11:42:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.915+0000 7f5acec00b80 1 Processor -- start 2024-05-24T11:42:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad79dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 0x56416ad77ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5acec00b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad78010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5acec00b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416ad78720 con 0x56416ad6f270 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5acec00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416ad78890 con 0x56416ad6e950 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5acec00b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56416ade9070 con 0x56416ad69480 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5acec00b80 10 client.0 _ll_register_callbacks cb 0x56416ac5ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad79dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abbfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad79dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59670/0 (socket says 172.21.15.39:59670) 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abbfff700 1 -- 172.21.15.39:0/1099788313 learned_addr learned my addr 172.21.15.39:0/1099788313 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5ac0fe2700 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad78010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 0x56416ad77ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abbfff700 1 -- 172.21.15.39:0/1099788313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 msgr2=0x56416ad77ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abbfff700 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 0x56416ad77ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abbfff700 1 -- 172.21.15.39:0/1099788313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 msgr2=0x56416ad78010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abbfff700 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad78010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abbfff700 1 -- 172.21.15.39:0/1099788313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab0017070 con 0x56416ad69480 2024-05-24T11:42:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 0x56416ad77ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:42:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.916+0000 7f5abbfff700 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad79dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0005950 tx=0x7f5ab00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a438221731ae5549 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.917+0000 7f5a9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:42:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.917+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/1099788313 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0025040 con 0x56416ad69480 2024-05-24T11:42:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.917+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/1099788313 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f5ab0004250 con 0x56416ad69480 2024-05-24T11:42:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.917+0000 7f5acec00b80 1 -- 172.21.15.39:0/1099788313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5aac09d960 con 0x56416ad69480 2024-05-24T11:42:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.917+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/1099788313 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ab0004410 con 0x56416ad69480 2024-05-24T11:42:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5acec00b80 10 client.25330 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5acec00b80 1 -- 172.21.15.39:0/1099788313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56416ab3ec70 con 0x56416ad69480 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/1099788313 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aac09d960 con 0x56416ad69480 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/1099788313 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5ab0003880 con 0x56416ad69480 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5acec00b80 10 client.25330 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5acec00b80 10 client.25330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5acec00b80 20 client.25330 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5acec00b80 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56416adea490 0x56416adec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5acec00b80 4 client.25330 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5acec00b80 1 -- 172.21.15.39:0/1099788313 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56416acb1e60 con 0x56416adea490 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.918+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56416adea490 0x56416adec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.919+0000 7f5a9b7fe700 10 client.25330 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.919+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56416adea490 0x56416adec8b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8464e21164da5f42 server_cookie=b3de611749f9dcfc in_seq=0 out_seq=0 2024-05-24T11:42:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.970+0000 7f5a9b7fe700 1 -- 172.21.15.39:0/1099788313 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56416acb1e60 con 0x56416adea490 2024-05-24T11:42:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.970+0000 7f5a9b7fe700 10 client.25330 handle_command_reply: tid=0 2024-05-24T11:42:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.970+0000 7f5acec00b80 1 client.25330 shutdown 2024-05-24T11:42:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 -- 172.21.15.39:0/1099788313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 msgr2=0x56416ad79dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad79dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0005950 tx=0x7f5ab00091a0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 -- 172.21.15.39:0/1099788313 shutdown_connections 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56416ad69480 0x56416ad79dc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56416adea490 0x56416adec8b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56416ad6f270 0x56416ad78010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 --2- 172.21.15.39:0/1099788313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56416ad6e950 0x56416ad77ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 -- 172.21.15.39:0/1099788313 >> 172.21.15.39:0/1099788313 conn(0x56416acc71b0 msgr2=0x56416ad70cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5a9b7fe700 0 client.25330 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 -- 172.21.15.39:0/1099788313 shutdown_connections 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 1 -- 172.21.15.39:0/1099788313 wait complete. 2024-05-24T11:42:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.971+0000 7f5acec00b80 20 client.25330 trim_cache size 0 max 16384 2024-05-24T11:42:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:42:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:42:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:42:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:42:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 -- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5aa40a1310 msgr2=0x7f5aa40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5aa40a1310 0x7f5aa40a37d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009be0 tx=0x7f5ab0009260 comp rx=0 tx=0).stop 2024-05-24T11:42:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 -- 172.21.15.39:0/3781278606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5abc107ea0 msgr2=0x7f5abc1a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5abc107ea0 0x7f5abc1a3b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00d980 tx=0x7f5aac00dc90 comp rx=0 tx=0).stop 2024-05-24T11:42:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 -- 172.21.15.39:0/3781278606 shutdown_connections 2024-05-24T11:42:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5aa40a1310 0x7f5aa40a37d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc10d5a0 0x7f5abc19d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5abc1074e0 0x7f5abc1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 --2- 172.21.15.39:0/3781278606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5abc107ea0 0x7f5abc1a3b20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.972+0000 7f5ac2a45700 1 -- 172.21.15.39:0/3781278606 >> 172.21.15.39:0/3781278606 conn(0x7f5abc076dc0 msgr2=0x7f5abc077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.973+0000 7f5ac2a45700 1 -- 172.21.15.39:0/3781278606 shutdown_connections 2024-05-24T11:42:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:15.973+0000 7f5ac2a45700 1 -- 172.21.15.39:0/3781278606 wait complete. 2024-05-24T11:42:16.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:15 smithi039 ceph-mon[42919]: pgmap v124: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 21 MiB/s wr, 1.80k op/s 2024-05-24T11:42:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:15 smithi053 ceph-mon[47448]: pgmap v124: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 21 MiB/s wr, 1.80k op/s 2024-05-24T11:42:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:15 smithi137 ceph-mon[47907]: pgmap v124: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 21 MiB/s wr, 1.80k op/s 2024-05-24T11:42:18.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:17 smithi039 ceph-mon[42919]: pgmap v125: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:42:18.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:17 smithi053 ceph-mon[47448]: pgmap v125: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:42:18.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:17 smithi137 ceph-mon[47907]: pgmap v125: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:42:19.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:18 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:42:19.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 175.0 s, 1324.2 tps, lat 75.550 ms stddev 21.527 2024-05-24T11:42:19.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:18 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:42:19.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:18 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:42:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: pgmap v126: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:42:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:42:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:42:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: pgmap v126: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:42:20.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:42:20.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:42:20.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: pgmap v126: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:42:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:42:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:42:20.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:42:22.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:21 smithi053 ceph-mon[47448]: pgmap v127: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:22.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:21 smithi137 ceph-mon[47907]: pgmap v127: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:22.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:21 smithi039 ceph-mon[42919]: pgmap v127: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 180.0 s, 1408.2 tps, lat 71.041 ms stddev 21.650 2024-05-24T11:42:24.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:23 smithi053 ceph-mon[47448]: pgmap v128: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:24.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:23 smithi137 ceph-mon[47907]: pgmap v128: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:24.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:23 smithi039 ceph-mon[42919]: pgmap v128: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:26.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:25 smithi053 ceph-mon[47448]: pgmap v129: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 20 MiB/s wr, 1.79k op/s 2024-05-24T11:42:26.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:25 smithi137 ceph-mon[47907]: pgmap v129: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 20 MiB/s wr, 1.79k op/s 2024-05-24T11:42:26.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:25 smithi039 ceph-mon[42919]: pgmap v129: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 20 MiB/s wr, 1.79k op/s 2024-05-24T11:42:28.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:27 smithi053 ceph-mon[47448]: pgmap v130: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-05-24T11:42:28.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:27 smithi137 ceph-mon[47907]: pgmap v130: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-05-24T11:42:28.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:27 smithi039 ceph-mon[42919]: pgmap v130: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-05-24T11:42:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 185.0 s, 1336.0 tps, lat 74.623 ms stddev 19.866 2024-05-24T11:42:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.148+0000 7f04bdd62700 1 -- 172.21.15.39:0/2758724605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8108450 msgr2=0x7f04b810ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.148+0000 7f04bdd62700 1 --2- 172.21.15.39:0/2758724605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8108450 0x7f04b810ffd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009990 tx=0x7f04b0009ca0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.149+0000 7f04bdd62700 1 -- 172.21.15.39:0/2758724605 shutdown_connections 2024-05-24T11:42:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.149+0000 7f04bdd62700 1 --2- 172.21.15.39:0/2758724605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b8103330 0x7f04b8103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.149+0000 7f04bdd62700 1 --2- 172.21.15.39:0/2758724605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8108450 0x7f04b810ffd0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.149+0000 7f04bdd62700 1 --2- 172.21.15.39:0/2758724605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04b8103ce0 0x7f04b8107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.149+0000 7f04bdd62700 1 -- 172.21.15.39:0/2758724605 >> 172.21.15.39:0/2758724605 conn(0x7f04b80feb70 msgr2=0x7f04b8100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.149+0000 7f04bdd62700 1 -- 172.21.15.39:0/2758724605 shutdown_connections 2024-05-24T11:42:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.149+0000 7f04bdd62700 1 -- 172.21.15.39:0/2758724605 wait complete. 2024-05-24T11:42:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.149+0000 7f04bdd62700 1 Processor -- start 2024-05-24T11:42:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04bdd62700 1 -- start start 2024-05-24T11:42:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04bdd62700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8103330 0x7f04b810fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04bdd62700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04b8108450 0x7f04b810dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04bdd62700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b810e2b0 0x7f04b810e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04bdd62700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b81a9f50 con 0x7f04b8103330 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04bdd62700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b81aa090 con 0x7f04b8108450 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04bdd62700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b81aa1d0 con 0x7f04b810e2b0 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8103330 0x7f04b810fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8103330 0x7f04b810fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36074/0 (socket says 172.21.15.39:36074) 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b77fe700 1 -- 172.21.15.39:0/1278438245 learned_addr learned my addr 172.21.15.39:0/1278438245 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b6ffd700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04b8108450 0x7f04b810dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b7fff700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b810e2b0 0x7f04b810e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b77fe700 1 -- 172.21.15.39:0/1278438245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b810e2b0 msgr2=0x7f04b810e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b77fe700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b810e2b0 0x7f04b810e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b77fe700 1 -- 172.21.15.39:0/1278438245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04b8108450 msgr2=0x7f04b810dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b77fe700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04b8108450 0x7f04b810dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b77fe700 1 -- 172.21.15.39:0/1278438245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b00095f0 con 0x7f04b8103330 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.150+0000 7f04b7fff700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b810e2b0 0x7f04b810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.151+0000 7f04b77fe700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8103330 0x7f04b810fbf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f04a800ba50 tx=0x7f04a800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.151+0000 7f04b4ff9700 1 -- 172.21.15.39:0/1278438245 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a800c9b0 con 0x7f04b8103330 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.151+0000 7f04b4ff9700 1 -- 172.21.15.39:0/1278438245 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04a8014440 con 0x7f04b8103330 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.151+0000 7f04b4ff9700 1 -- 172.21.15.39:0/1278438245 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a80075b0 con 0x7f04b8103330 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.151+0000 7f04bdd62700 1 -- 172.21.15.39:0/1278438245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b81aa480 con 0x7f04b8103330 2024-05-24T11:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.151+0000 7f04bdd62700 1 -- 172.21.15.39:0/1278438245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b81aa940 con 0x7f04b8103330 2024-05-24T11:42:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.155+0000 7f04c9f1db80 1 Processor -- start 2024-05-24T11:42:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.155+0000 7f04b4ff9700 1 -- 172.21.15.39:0/1278438245 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f04a80145b0 con 0x7f04b8103330 2024-05-24T11:42:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.155+0000 7f04c9f1db80 1 -- start start 2024-05-24T11:42:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.155+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 0x558ca925bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.155+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925c270 0x558ca9255f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.155+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca92568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.155+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca91ad0f0 con 0x558ca925c270 2024-05-24T11:42:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.155+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca91aaab0 con 0x558ca9256480 2024-05-24T11:42:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.155+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca91a63a0 con 0x558ca925b950 2024-05-24T11:42:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.156+0000 7f04b6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca92568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.156+0000 7f04b6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca92568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:40724/0 (socket says 172.21.15.39:40724) 2024-05-24T11:42:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.156+0000 7f04b6ffd700 1 -- 172.21.15.39:0/4148559748 learned_addr learned my addr 172.21.15.39:0/4148559748 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.156+0000 7f04b7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 0x558ca925bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.156+0000 7f04b77fe700 1 --2- 172.21.15.39:0/4148559748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925c270 0x558ca9255f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.156+0000 7f04b4ff9700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f04a00a1290 0x7f04a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.157+0000 7f04b4ff9700 1 -- 172.21.15.39:0/1278438245 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f04a80cc330 con 0x7f04b8103330 2024-05-24T11:42:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.157+0000 7f04b77fe700 1 -- 172.21.15.39:0/4148559748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 msgr2=0x558ca925bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.157+0000 7f04b77fe700 1 --2- 172.21.15.39:0/4148559748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 0x558ca925bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.157+0000 7f04b77fe700 1 -- 172.21.15.39:0/4148559748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 msgr2=0x558ca92568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.157+0000 7f04b77fe700 1 --2- 172.21.15.39:0/4148559748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca92568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.157+0000 7f04b77fe700 1 -- 172.21.15.39:0/4148559748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ca9027350 con 0x558ca925c270 2024-05-24T11:42:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.157+0000 7f04b77fe700 1 --2- 172.21.15.39:0/4148559748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925c270 0x558ca9255f40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f04a80c9b70 tx=0x7f04a8099300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853d39f8d85f3f8b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.158+0000 7f04b7fff700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f04a00a1290 0x7f04a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.158+0000 7f049dffb700 1 -- 172.21.15.39:0/4148559748 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8099070 con 0x558ca925c270 2024-05-24T11:42:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.158+0000 7f049dffb700 1 -- 172.21.15.39:0/4148559748 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04a8095070 con 0x558ca925c270 2024-05-24T11:42:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.158+0000 7f049dffb700 1 -- 172.21.15.39:0/4148559748 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a80a19b0 con 0x558ca925c270 2024-05-24T11:42:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.159+0000 7f04c9f1db80 1 -- 172.21.15.39:0/4148559748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925c270 msgr2=0x558ca9255f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.159+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/4148559748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925c270 0x558ca9255f40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f04a80c9b70 tx=0x7f04a8099300 comp rx=0 tx=0).stop 2024-05-24T11:42:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.159+0000 7f04b7fff700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f04a00a1290 0x7f04a00a3750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04b00036c0 tx=0x7f04b001b040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.159+0000 7f04c9f1db80 1 -- 172.21.15.39:0/4148559748 shutdown_connections 2024-05-24T11:42:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.159+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/4148559748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 0x558ca925bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.159+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/4148559748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca92568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.159+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/4148559748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925c270 0x558ca9255f40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.159+0000 7f04c9f1db80 1 -- 172.21.15.39:0/4148559748 >> 172.21.15.39:0/4148559748 conn(0x558ca91b41b0 msgr2=0x558ca9250840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.159+0000 7f04c9f1db80 1 -- 172.21.15.39:0/4148559748 shutdown_connections 2024-05-24T11:42:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.160+0000 7f04c9f1db80 1 -- 172.21.15.39:0/4148559748 wait complete. 2024-05-24T11:42:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.160+0000 7f04c9f1db80 1 -- start start 2024-05-24T11:42:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.160+0000 7f04c9f1db80 1 Processor -- start 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca9256480 0x558ca925f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca925b950 0x558ca925fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca92600d0 0x558ca926bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca9142500 con 0x558ca9256480 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca91421a0 con 0x558ca925b950 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca9029640 con 0x558ca92600d0 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04c9f1db80 10 client.0 _ll_register_callbacks cb 0x558ca914bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04b6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca925b950 0x558ca925fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04b6ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca925b950 0x558ca925fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:40726/0 (socket says 172.21.15.39:40726) 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04b6ffd700 1 -- 172.21.15.39:0/3648158156 learned_addr learned my addr 172.21.15.39:0/3648158156 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.161+0000 7f04b7fff700 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca92600d0 0x558ca926bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.162+0000 7f04b6ffd700 1 -- 172.21.15.39:0/3648158156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca92600d0 msgr2=0x558ca926bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.162+0000 7f04b6ffd700 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca92600d0 0x558ca926bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.162+0000 7f04b6ffd700 1 -- 172.21.15.39:0/3648158156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca9256480 msgr2=0x558ca925f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.162+0000 7f04b6ffd700 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca9256480 0x558ca925f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.162+0000 7f04b6ffd700 1 -- 172.21.15.39:0/3648158156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b81aa940 con 0x558ca925b950 2024-05-24T11:42:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.162+0000 7f04b6ffd700 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca925b950 0x558ca925fb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b960 tx=0x7f04ac00bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32056bbdbbb41588 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.162+0000 7f0496ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:42:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.162+0000 7f0496ffd700 1 -- 172.21.15.39:0/3648158156 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac016070 con 0x558ca925b950 2024-05-24T11:42:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.162+0000 7f0496ffd700 1 -- 172.21.15.39:0/3648158156 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f04ac003d00 con 0x558ca925b950 2024-05-24T11:42:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.163+0000 7f04c9f1db80 1 -- 172.21.15.39:0/3648158156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ca90ff210 con 0x558ca925b950 2024-05-24T11:42:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.163+0000 7f0496ffd700 1 -- 172.21.15.39:0/3648158156 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04ac011440 con 0x558ca925b950 2024-05-24T11:42:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.163+0000 7f04c9f1db80 10 client.25439 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.163+0000 7f04c9f1db80 1 -- 172.21.15.39:0/3648158156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558ca8fd3b00 con 0x558ca925b950 2024-05-24T11:42:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.163+0000 7f0496ffd700 1 -- 172.21.15.39:0/3648158156 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558ca90ff210 con 0x558ca925b950 2024-05-24T11:42:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.164+0000 7f0496ffd700 1 -- 172.21.15.39:0/3648158156 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f04ac0115a0 con 0x558ca925b950 2024-05-24T11:42:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.164+0000 7f04c9f1db80 10 client.25439 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.164+0000 7f04c9f1db80 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.164+0000 7f04c9f1db80 20 client.25439 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.164+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558ca92811e0 0x558ca9283600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.164+0000 7f04c9f1db80 4 client.25439 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:42:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.164+0000 7f04c9f1db80 1 -- 172.21.15.39:0/3648158156 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ca92257b0 con 0x558ca92811e0 2024-05-24T11:42:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.165+0000 7f04b77fe700 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558ca92811e0 0x558ca9283600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.165+0000 7f04b77fe700 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558ca92811e0 0x558ca9283600 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bac938b6f64bf84 server_cookie=e3502df0bcb31452 in_seq=0 out_seq=0 2024-05-24T11:42:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.165+0000 7f0496ffd700 10 client.25439 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f0496ffd700 1 -- 172.21.15.39:0/3648158156 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ca92257b0 con 0x558ca92811e0 2024-05-24T11:42:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f0496ffd700 10 client.25439 handle_command_reply: tid=0 2024-05-24T11:42:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 client.25439 shutdown 2024-05-24T11:42:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 -- 172.21.15.39:0/3648158156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca925b950 msgr2=0x558ca925fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca925b950 0x558ca925fb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b960 tx=0x7f04ac00bc70 comp rx=0 tx=0).stop 2024-05-24T11:42:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 -- 172.21.15.39:0/3648158156 shutdown_connections 2024-05-24T11:42:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca92600d0 0x558ca926bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558ca92811e0 0x558ca9283600 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca9256480 0x558ca925f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/3648158156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca925b950 0x558ca925fb90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f0496ffd700 0 client.25439 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 -- 172.21.15.39:0/3648158156 >> 172.21.15.39:0/3648158156 conn(0x558ca91b41b0 msgr2=0x558ca925a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 -- 172.21.15.39:0/3648158156 shutdown_connections 2024-05-24T11:42:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 1 -- 172.21.15.39:0/3648158156 wait complete. 2024-05-24T11:42:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.166+0000 7f04c9f1db80 20 client.25439 trim_cache size 0 max 16384 2024-05-24T11:42:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.189+0000 7f04c9f1db80 1 Processor -- start 2024-05-24T11:42:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.189+0000 7f04c9f1db80 1 -- start start 2024-05-24T11:42:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.189+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9250910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925b950 0x558ca9252540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca92600d0 0x558ca9256860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca92621c0 con 0x558ca925b950 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca9256da0 con 0x558ca9256480 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca92520f0 con 0x558ca92600d0 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9250910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9250910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:40728/0 (socket says 172.21.15.39:40728) 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b77fe700 1 -- 172.21.15.39:0/1226556224 learned_addr learned my addr 172.21.15.39:0/1226556224 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b6ffd700 1 --2- 172.21.15.39:0/1226556224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925b950 0x558ca9252540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b77fe700 1 -- 172.21.15.39:0/1226556224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca92600d0 msgr2=0x558ca9256860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b77fe700 1 --2- 172.21.15.39:0/1226556224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca92600d0 0x558ca9256860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b77fe700 1 -- 172.21.15.39:0/1226556224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925b950 msgr2=0x558ca9252540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b77fe700 1 --2- 172.21.15.39:0/1226556224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925b950 0x558ca9252540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b77fe700 1 -- 172.21.15.39:0/1226556224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ca916c500 con 0x558ca9256480 2024-05-24T11:42:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.190+0000 7f04b77fe700 1 --2- 172.21.15.39:0/1226556224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9250910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04a80c9bd0 tx=0x7f04a8099090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6189032cce3c5ca7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.191+0000 7f049e7fc700 1 -- 172.21.15.39:0/1226556224 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a80a19b0 con 0x558ca9256480 2024-05-24T11:42:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.191+0000 7f049e7fc700 1 -- 172.21.15.39:0/1226556224 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04a8099530 con 0x558ca9256480 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.191+0000 7f049e7fc700 1 -- 172.21.15.39:0/1226556224 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a80840d0 con 0x558ca9256480 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.191+0000 7f04c9f1db80 1 -- 172.21.15.39:0/1226556224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 msgr2=0x558ca9250910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.191+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/1226556224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9250910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04a80c9bd0 tx=0x7f04a8099090 comp rx=0 tx=0).stop 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.192+0000 7f04c9f1db80 1 -- 172.21.15.39:0/1226556224 shutdown_connections 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.192+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/1226556224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca92600d0 0x558ca9256860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.192+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/1226556224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9250910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.192+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/1226556224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca925b950 0x558ca9252540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.192+0000 7f04c9f1db80 1 -- 172.21.15.39:0/1226556224 >> 172.21.15.39:0/1226556224 conn(0x558ca91b41b0 msgr2=0x558ca9254670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.192+0000 7f04c9f1db80 1 -- 172.21.15.39:0/1226556224 shutdown_connections 2024-05-24T11:42:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.192+0000 7f04c9f1db80 1 -- 172.21.15.39:0/1226556224 wait complete. 2024-05-24T11:42:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.192+0000 7f04c9f1db80 1 -- start start 2024-05-24T11:42:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.193+0000 7f04c9f1db80 1 Processor -- start 2024-05-24T11:42:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9266e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 0x558ca924ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04c9f1db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca92600d0 0x558ca92504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca9250a90 con 0x558ca92600d0 2024-05-24T11:42:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca9250c00 con 0x558ca9256480 2024-05-24T11:42:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04c9f1db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca9259380 con 0x558ca925b950 2024-05-24T11:42:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04c9f1db80 10 client.0 _ll_register_callbacks cb 0x558ca914bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04b77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9266e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04b77fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9266e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:40732/0 (socket says 172.21.15.39:40732) 2024-05-24T11:42:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04b77fe700 1 -- 172.21.15.39:0/54536309 learned_addr learned my addr 172.21.15.39:0/54536309 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.194+0000 7f04b6ffd700 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 0x558ca924ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f04b77fe700 1 -- 172.21.15.39:0/54536309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 msgr2=0x558ca924ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f04b77fe700 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 0x558ca924ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f04b77fe700 1 -- 172.21.15.39:0/54536309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca92600d0 msgr2=0x558ca92504c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:42:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f04b77fe700 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca92600d0 0x558ca92504c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f04b77fe700 1 -- 172.21.15.39:0/54536309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ca9027350 con 0x558ca9256480 2024-05-24T11:42:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f04b77fe700 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9266e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04a80b9150 tx=0x7f04a8099100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c35b48b42ea01d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f0496ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:42:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f0496ffd700 1 -- 172.21.15.39:0/54536309 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8099c30 con 0x558ca9256480 2024-05-24T11:42:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f0496ffd700 1 -- 172.21.15.39:0/54536309 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f04a80a19b0 con 0x558ca9256480 2024-05-24T11:42:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.195+0000 7f04c9f1db80 1 -- 172.21.15.39:0/54536309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04a8095070 con 0x558ca9256480 2024-05-24T11:42:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f0496ffd700 1 -- 172.21.15.39:0/54536309 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a80840d0 con 0x558ca9256480 2024-05-24T11:42:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f04c9f1db80 10 client.25445 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f04c9f1db80 1 -- 172.21.15.39:0/54536309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558ca902bc70 con 0x558ca9256480 2024-05-24T11:42:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f0496ffd700 1 -- 172.21.15.39:0/54536309 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04a8095070 con 0x558ca9256480 2024-05-24T11:42:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f0496ffd700 1 -- 172.21.15.39:0/54536309 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f04a8099800 con 0x558ca9256480 2024-05-24T11:42:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f04c9f1db80 10 client.25445 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f04c9f1db80 10 client.25445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f04c9f1db80 20 client.25445 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558ca92d7940 0x558ca92d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f04c9f1db80 4 client.25445 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:42:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f04c9f1db80 1 -- 172.21.15.39:0/54536309 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ca919ee60 con 0x558ca92d7940 2024-05-24T11:42:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.196+0000 7f04b6ffd700 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558ca92d7940 0x558ca92d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.197+0000 7f04b6ffd700 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558ca92d7940 0x558ca92d9d30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fca493313f0a0c7 server_cookie=54748f940f2b9a02 in_seq=0 out_seq=0 2024-05-24T11:42:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.197+0000 7f0496ffd700 10 client.25445 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.223+0000 7f0496ffd700 1 -- 172.21.15.39:0/54536309 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ca919ee60 con 0x558ca92d7940 2024-05-24T11:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.223+0000 7f0496ffd700 10 client.25445 handle_command_reply: tid=0 2024-05-24T11:42:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.223+0000 7f04c9f1db80 1 client.25445 shutdown 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 -- 172.21.15.39:0/54536309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 msgr2=0x558ca9266e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9266e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04a80b9150 tx=0x7f04a8099100 comp rx=0 tx=0).stop 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 -- 172.21.15.39:0/54536309 shutdown_connections 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558ca92d7940 0x558ca92d9d30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ca92600d0 0x558ca92504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ca9256480 0x558ca9266e60 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04a80b9150 tx=0x7f04a8099100 comp rx=0 tx=0).stop 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 --2- 172.21.15.39:0/54536309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ca925b950 0x558ca924ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 -- 172.21.15.39:0/54536309 >> 172.21.15.39:0/54536309 conn(0x558ca91b41b0 msgr2=0x558ca926b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f0496ffd700 0 client.25445 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 -- 172.21.15.39:0/54536309 shutdown_connections 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 1 -- 172.21.15.39:0/54536309 wait complete. 2024-05-24T11:42:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.224+0000 7f04c9f1db80 20 client.25445 trim_cache size 0 max 16384 2024-05-24T11:42:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:42:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:42:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:42:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:42:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 -- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f04a00a1290 msgr2=0x7f04a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f04a00a1290 0x7f04a00a3750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04b00036c0 tx=0x7f04b001b040 comp rx=0 tx=0).stop 2024-05-24T11:42:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 -- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8103330 msgr2=0x7f04b810fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8103330 0x7f04b810fbf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f04a800ba50 tx=0x7f04a800bd60 comp rx=0 tx=0).stop 2024-05-24T11:42:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 -- 172.21.15.39:0/1278438245 shutdown_connections 2024-05-24T11:42:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b810e2b0 0x7f04b810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f04a00a1290 0x7f04a00a3750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04b8103330 0x7f04b810fbf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 --2- 172.21.15.39:0/1278438245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04b8108450 0x7f04b810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.225+0000 7f04bdd62700 1 -- 172.21.15.39:0/1278438245 >> 172.21.15.39:0/1278438245 conn(0x7f04b80feb70 msgr2=0x7f04b80ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.226+0000 7f04bdd62700 1 -- 172.21.15.39:0/1278438245 shutdown_connections 2024-05-24T11:42:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:29.227+0000 7f04bdd62700 1 -- 172.21.15.39:0/1278438245 wait complete. 2024-05-24T11:42:30.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:29 smithi053 ceph-mon[47448]: pgmap v131: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 20 MiB/s wr, 1.79k op/s 2024-05-24T11:42:30.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:29 smithi137 ceph-mon[47907]: pgmap v131: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 20 MiB/s wr, 1.79k op/s 2024-05-24T11:42:30.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:29 smithi039 ceph-mon[42919]: pgmap v131: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 20 MiB/s wr, 1.79k op/s 2024-05-24T11:42:32.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:31 smithi053 ceph-mon[47448]: pgmap v132: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:32.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:31 smithi137 ceph-mon[47907]: pgmap v132: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:32.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:31 smithi039 ceph-mon[42919]: pgmap v132: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:34.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 190.0 s, 1315.0 tps, lat 76.262 ms stddev 23.150 2024-05-24T11:42:34.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:33 smithi053 ceph-mon[47448]: pgmap v133: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:34.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:33 smithi137 ceph-mon[47907]: pgmap v133: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:34.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:33 smithi039 ceph-mon[42919]: pgmap v133: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-05-24T11:42:36.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:35 smithi053 ceph-mon[47448]: pgmap v134: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 19 MiB/s wr, 1.79k op/s 2024-05-24T11:42:36.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:35 smithi137 ceph-mon[47907]: pgmap v134: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 19 MiB/s wr, 1.79k op/s 2024-05-24T11:42:36.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:35 smithi039 ceph-mon[42919]: pgmap v134: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 19 MiB/s wr, 1.79k op/s 2024-05-24T11:42:38.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:37 smithi053 ceph-mon[47448]: pgmap v135: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 1.20k op/s 2024-05-24T11:42:38.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:37 smithi137 ceph-mon[47907]: pgmap v135: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 1.20k op/s 2024-05-24T11:42:38.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:37 smithi039 ceph-mon[42919]: pgmap v135: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 1.20k op/s 2024-05-24T11:42:39.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 195.0 s, 1285.6 tps, lat 77.741 ms stddev 23.883 2024-05-24T11:42:40.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:39 smithi053 ceph-mon[47448]: pgmap v136: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 18 MiB/s wr, 1.83k op/s 2024-05-24T11:42:40.315 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:39 smithi137 ceph-mon[47907]: pgmap v136: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 18 MiB/s wr, 1.83k op/s 2024-05-24T11:42:40.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:39 smithi039 ceph-mon[42919]: pgmap v136: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 18 MiB/s wr, 1.83k op/s 2024-05-24T11:42:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:41 smithi053 ceph-mon[47448]: pgmap v137: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-05-24T11:42:42.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:41 smithi137 ceph-mon[47907]: pgmap v137: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-05-24T11:42:42.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:41 smithi039 ceph-mon[42919]: pgmap v137: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-05-24T11:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.403+0000 7f89c435b700 1 -- 172.21.15.39:0/155760716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89bc10d950 msgr2=0x7f89bc10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.403+0000 7f89c435b700 1 --2- 172.21.15.39:0/155760716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89bc10d950 0x7f89bc10dd30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009990 tx=0x7f89ac009ca0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.403+0000 7f89c435b700 1 -- 172.21.15.39:0/155760716 shutdown_connections 2024-05-24T11:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.403+0000 7f89c435b700 1 --2- 172.21.15.39:0/155760716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89bc107fa0 0x7f89bc115920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.403+0000 7f89c435b700 1 --2- 172.21.15.39:0/155760716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89bc10d950 0x7f89bc10dd30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.403+0000 7f89c435b700 1 --2- 172.21.15.39:0/155760716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89bc068bb0 0x7f89bc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.403+0000 7f89c435b700 1 -- 172.21.15.39:0/155760716 >> 172.21.15.39:0/155760716 conn(0x7f89bc06c3c0 msgr2=0x7f89bc06c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.403+0000 7f89c435b700 1 -- 172.21.15.39:0/155760716 shutdown_connections 2024-05-24T11:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.403+0000 7f89c435b700 1 -- 172.21.15.39:0/155760716 wait complete. 2024-05-24T11:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.404+0000 7f89c435b700 1 Processor -- start 2024-05-24T11:42:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.404+0000 7f89c435b700 1 -- start start 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c435b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89bc068bb0 0x7f89bc1a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c435b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89bc107fa0 0x7f89bc1a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c435b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89bc10d950 0x7f89bc1a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c435b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc1a6c90 con 0x7f89bc10d950 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c435b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc1a6e00 con 0x7f89bc107fa0 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c435b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc1a6fc0 con 0x7f89bc068bb0 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c20f7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89bc068bb0 0x7f89bc1a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c20f7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89bc068bb0 0x7f89bc1a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:35946/0 (socket says 172.21.15.39:35946) 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c20f7700 1 -- 172.21.15.39:0/473482474 learned_addr learned my addr 172.21.15.39:0/473482474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c18f6700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89bc107fa0 0x7f89bc1a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c20f7700 1 -- 172.21.15.39:0/473482474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89bc107fa0 msgr2=0x7f89bc1a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c20f7700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89bc107fa0 0x7f89bc1a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c20f7700 1 -- 172.21.15.39:0/473482474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89bc10d950 msgr2=0x7f89bc1a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c20f7700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89bc10d950 0x7f89bc1a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c20f7700 1 -- 172.21.15.39:0/473482474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0095f0 con 0x7f89bc068bb0 2024-05-24T11:42:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.405+0000 7f89c18f6700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89bc107fa0 0x7f89bc1a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:42:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.406+0000 7f89c20f7700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89bc068bb0 0x7f89bc1a1dc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f89ac004730 tx=0x7f89ac009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.406+0000 7f89b37fe700 1 -- 172.21.15.39:0/473482474 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac01d070 con 0x7f89bc068bb0 2024-05-24T11:42:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.406+0000 7f89c435b700 1 -- 172.21.15.39:0/473482474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89bc1a7240 con 0x7f89bc068bb0 2024-05-24T11:42:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.406+0000 7f89b37fe700 1 -- 172.21.15.39:0/473482474 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89ac022470 con 0x7f89bc068bb0 2024-05-24T11:42:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.406+0000 7f89b37fe700 1 -- 172.21.15.39:0/473482474 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac017650 con 0x7f89bc068bb0 2024-05-24T11:42:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.406+0000 7f89c435b700 1 -- 172.21.15.39:0/473482474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc118250 con 0x7f89bc068bb0 2024-05-24T11:42:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.409+0000 7f89b37fe700 1 -- 172.21.15.39:0/473482474 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f89ac0225e0 con 0x7f89bc068bb0 2024-05-24T11:42:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.409+0000 7f89b37fe700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f89a80a1310 0x7f89a80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.409+0000 7f89b37fe700 1 -- 172.21.15.39:0/473482474 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f89ac0ce350 con 0x7f89bc068bb0 2024-05-24T11:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.410+0000 7f89c18f6700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f89a80a1310 0x7f89a80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.411+0000 7f89c18f6700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f89a80a1310 0x7f89a80a37d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f89b8005fd0 tx=0x7f89b8005ee0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.411+0000 7f89d0516b80 1 Processor -- start 2024-05-24T11:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89d0516b80 1 -- start start 2024-05-24T11:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a94950 0x558d70a94d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a95270 0x558d70a8ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 0x558d70a8f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89d0516b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d709e60f0 con 0x558d70a8f480 2024-05-24T11:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89d0516b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d709e3ab0 con 0x558d70a94950 2024-05-24T11:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89d0516b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d709df3a0 con 0x558d70a95270 2024-05-24T11:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89c18f6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a95270 0x558d70a8ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89c28f8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 0x558d70a8f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89c18f6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a95270 0x558d70a8ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:35952/0 (socket says 172.21.15.39:35952) 2024-05-24T11:42:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89c18f6700 1 -- 172.21.15.39:0/3425900670 learned_addr learned my addr 172.21.15.39:0/3425900670 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.412+0000 7f89c20f7700 1 --2- 172.21.15.39:0/3425900670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a94950 0x558d70a94d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.413+0000 7f89c18f6700 1 -- 172.21.15.39:0/3425900670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a94950 msgr2=0x558d70a94d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.413+0000 7f89c18f6700 1 --2- 172.21.15.39:0/3425900670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a94950 0x558d70a94d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.413+0000 7f89c18f6700 1 -- 172.21.15.39:0/3425900670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 msgr2=0x558d70a8f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.413+0000 7f89c18f6700 1 --2- 172.21.15.39:0/3425900670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 0x558d70a8f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.413+0000 7f89c18f6700 1 -- 172.21.15.39:0/3425900670 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d70860350 con 0x558d70a95270 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.413+0000 7f89c18f6700 1 --2- 172.21.15.39:0/3425900670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a95270 0x558d70a8ef40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f89b800ed80 tx=0x7f89b8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e971d0c6b8d9535 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.413+0000 7f89b0da5700 1 -- 172.21.15.39:0/3425900670 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8022070 con 0x558d70a95270 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.413+0000 7f89b0da5700 1 -- 172.21.15.39:0/3425900670 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89b800edb0 con 0x558d70a95270 2024-05-24T11:42:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.413+0000 7f89b0da5700 1 -- 172.21.15.39:0/3425900670 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8005310 con 0x558d70a95270 2024-05-24T11:42:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.414+0000 7f89d0516b80 1 -- 172.21.15.39:0/3425900670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a95270 msgr2=0x558d70a8ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.414+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3425900670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a95270 0x558d70a8ef40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f89b800ed80 tx=0x7f89b8009f90 comp rx=0 tx=0).stop 2024-05-24T11:42:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.415+0000 7f89d0516b80 1 -- 172.21.15.39:0/3425900670 shutdown_connections 2024-05-24T11:42:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.415+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3425900670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 0x558d70a8f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.415+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3425900670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a94950 0x558d70a94d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.415+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3425900670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a95270 0x558d70a8ef40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.415+0000 7f89d0516b80 1 -- 172.21.15.39:0/3425900670 >> 172.21.15.39:0/3425900670 conn(0x558d709ed1b0 msgr2=0x558d70a89840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.415+0000 7f89d0516b80 1 -- 172.21.15.39:0/3425900670 shutdown_connections 2024-05-24T11:42:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.415+0000 7f89d0516b80 1 -- 172.21.15.39:0/3425900670 wait complete. 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.415+0000 7f89d0516b80 1 -- start start 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89d0516b80 1 Processor -- start 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 0x558d70a985e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a94950 0x558d70a98b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a99060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89d0516b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7097b500 con 0x558d70a8f480 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89d0516b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7097b1a0 con 0x558d70a95270 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89d0516b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d70862640 con 0x558d70a94950 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89d0516b80 10 client.0 _ll_register_callbacks cb 0x558d70984ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89c18f6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a94950 0x558d70a98b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89c18f6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a94950 0x558d70a98b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:35956/0 (socket says 172.21.15.39:35956) 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89c18f6700 1 -- 172.21.15.39:0/2784672626 learned_addr learned my addr 172.21.15.39:0/2784672626 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89c20f7700 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 0x558d70a985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.416+0000 7f89c28f8700 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a99060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89c18f6700 1 -- 172.21.15.39:0/2784672626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 msgr2=0x558d70a99060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89c18f6700 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a99060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89c18f6700 1 -- 172.21.15.39:0/2784672626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 msgr2=0x558d70a985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89c18f6700 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 0x558d70a985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89c18f6700 1 -- 172.21.15.39:0/2784672626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b800ea30 con 0x558d70a94950 2024-05-24T11:42:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89c28f8700 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a99060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:42:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89c20f7700 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 0x558d70a985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:42:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89c18f6700 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a94950 0x558d70a98b20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f89b800ed50 tx=0x7f89b8004cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=484f67a208fd2b99 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:42.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89a5ffb700 1 -- 172.21.15.39:0/2784672626 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8022070 con 0x558d70a94950 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.417+0000 7f89a5ffb700 1 -- 172.21.15.39:0/2784672626 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f89b8004020 con 0x558d70a94950 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89d0516b80 1 -- 172.21.15.39:0/2784672626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d70938210 con 0x558d70a94950 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89a5ffb700 1 -- 172.21.15.39:0/2784672626 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89b8011cf0 con 0x558d70a94950 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89d0516b80 10 client.25351 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89d0516b80 1 -- 172.21.15.39:0/2784672626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558d7080cb00 con 0x558d70a94950 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89a5ffb700 1 -- 172.21.15.39:0/2784672626 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558d70938210 con 0x558d70a94950 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89a5ffb700 1 -- 172.21.15.39:0/2784672626 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f89b8003680 con 0x558d70a94950 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89d0516b80 10 client.25351 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89d0516b80 10 client.25351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89d0516b80 20 client.25351 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:42.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89d0516b80 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558d70ab9c00 0x558d70abc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89d0516b80 4 client.25351 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:42:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.418+0000 7f89d0516b80 1 -- 172.21.15.39:0/2784672626 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d70a5e7b0 con 0x558d70ab9c00 2024-05-24T11:42:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.419+0000 7f89c20f7700 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558d70ab9c00 0x558d70abc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.420+0000 7f89c20f7700 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558d70ab9c00 0x558d70abc020 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0dc03c715be6bc server_cookie=8b511a83c6054ef5 in_seq=0 out_seq=0 2024-05-24T11:42:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.420+0000 7f89a5ffb700 10 client.25351 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89a5ffb700 1 -- 172.21.15.39:0/2784672626 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d70a5e7b0 con 0x558d70ab9c00 2024-05-24T11:42:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89a5ffb700 10 client.25351 handle_command_reply: tid=0 2024-05-24T11:42:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 client.25351 shutdown 2024-05-24T11:42:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 -- 172.21.15.39:0/2784672626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a94950 msgr2=0x558d70a98b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a94950 0x558d70a98b20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f89b800ed50 tx=0x7f89b8004cc0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 -- 172.21.15.39:0/2784672626 shutdown_connections 2024-05-24T11:42:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558d70ab9c00 0x558d70abc020 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a99060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89a5ffb700 0 client.25351 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a8f480 0x558d70a985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 --2- 172.21.15.39:0/2784672626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a94950 0x558d70a98b20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 -- 172.21.15.39:0/2784672626 >> 172.21.15.39:0/2784672626 conn(0x558d709ed1b0 msgr2=0x558d70a934a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 -- 172.21.15.39:0/2784672626 shutdown_connections 2024-05-24T11:42:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.421+0000 7f89d0516b80 1 -- 172.21.15.39:0/2784672626 wait complete. 2024-05-24T11:42:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.422+0000 7f89d0516b80 20 client.25351 trim_cache size 0 max 16384 2024-05-24T11:42:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.442+0000 7f89d0516b80 1 Processor -- start 2024-05-24T11:42:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.442+0000 7f89d0516b80 1 -- start start 2024-05-24T11:42:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 0x558d70a89910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a8b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89d0516b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d70a8b0f0 con 0x558d70a94950 2024-05-24T11:42:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89d0516b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d70a8a330 con 0x558d70a95270 2024-05-24T11:42:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89d0516b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d70ab6860 con 0x558d70a8f480 2024-05-24T11:42:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c18f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 0x558d70a89910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c20f7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c18f6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 0x558d70a89910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57594/0 (socket says 172.21.15.39:57594) 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c20f7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:35972/0 (socket says 172.21.15.39:35972) 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c18f6700 1 -- 172.21.15.39:0/3193699045 learned_addr learned my addr 172.21.15.39:0/3193699045 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c28f8700 1 --2- 172.21.15.39:0/3193699045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a8b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c20f7700 1 -- 172.21.15.39:0/3193699045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 msgr2=0x558d70a8b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c20f7700 1 --2- 172.21.15.39:0/3193699045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a8b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c20f7700 1 -- 172.21.15.39:0/3193699045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 msgr2=0x558d70a89910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c20f7700 1 --2- 172.21.15.39:0/3193699045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 0x558d70a89910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.443+0000 7f89c20f7700 1 -- 172.21.15.39:0/3193699045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d709a5500 con 0x558d70a8f480 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.444+0000 7f89c20f7700 1 --2- 172.21.15.39:0/3193699045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab5700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0cbb90 tx=0x7f89ac009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa29580278008af6 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.444+0000 7f89b15a6700 1 -- 172.21.15.39:0/3193699045 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac09be00 con 0x558d70a8f480 2024-05-24T11:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.444+0000 7f89b15a6700 1 -- 172.21.15.39:0/3193699045 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89ac0b3070 con 0x558d70a8f480 2024-05-24T11:42:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.444+0000 7f89b15a6700 1 -- 172.21.15.39:0/3193699045 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac099920 con 0x558d70a8f480 2024-05-24T11:42:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.445+0000 7f89d0516b80 1 -- 172.21.15.39:0/3193699045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 msgr2=0x558d70ab5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.445+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3193699045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab5700 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0cbb90 tx=0x7f89ac009310 comp rx=0 tx=0).stop 2024-05-24T11:42:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.445+0000 7f89d0516b80 1 -- 172.21.15.39:0/3193699045 shutdown_connections 2024-05-24T11:42:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.445+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3193699045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab5700 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.445+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3193699045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a8b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.445+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3193699045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 0x558d70a89910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.445+0000 7f89d0516b80 1 -- 172.21.15.39:0/3193699045 >> 172.21.15.39:0/3193699045 conn(0x558d709ed1b0 msgr2=0x558d70ac0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.446+0000 7f89d0516b80 1 -- 172.21.15.39:0/3193699045 shutdown_connections 2024-05-24T11:42:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.446+0000 7f89d0516b80 1 -- 172.21.15.39:0/3193699045 wait complete. 2024-05-24T11:42:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.446+0000 7f89d0516b80 1 -- start start 2024-05-24T11:42:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.446+0000 7f89d0516b80 1 Processor -- start 2024-05-24T11:42:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 0x558d70a92e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89d0516b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a93380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89d0516b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d70a938c0 con 0x558d70a94950 2024-05-24T11:42:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89d0516b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d70a93a30 con 0x558d70a95270 2024-05-24T11:42:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89d0516b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d70a93bf0 con 0x558d70a8f480 2024-05-24T11:42:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89d0516b80 10 client.0 _ll_register_callbacks cb 0x558d70984ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89c28f8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a93380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89c28f8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a93380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:40856/0 (socket says 172.21.15.39:40856) 2024-05-24T11:42:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89c28f8700 1 -- 172.21.15.39:0/3965011473 learned_addr learned my addr 172.21.15.39:0/3965011473 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.447+0000 7f89c20f7700 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89c28f8700 1 -- 172.21.15.39:0/3965011473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 msgr2=0x558d70ab0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89c28f8700 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89c28f8700 1 -- 172.21.15.39:0/3965011473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 msgr2=0x558d70a92e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89c28f8700 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 0x558d70a92e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89c28f8700 1 -- 172.21.15.39:0/3965011473 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc118250 con 0x558d70a95270 2024-05-24T11:42:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89c28f8700 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a93380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89b4007b30 tx=0x7f89b4007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c639e024ea711584 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:42:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89a5ffb700 1 -- 172.21.15.39:0/3965011473 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4010040 con 0x558d70a95270 2024-05-24T11:42:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89a5ffb700 1 -- 172.21.15.39:0/3965011473 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f89b4003e10 con 0x558d70a95270 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89d0516b80 1 -- 172.21.15.39:0/3965011473 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89b800edb0 con 0x558d70a95270 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89a5ffb700 1 -- 172.21.15.39:0/3965011473 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89b40168a0 con 0x558d70a95270 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89d0516b80 10 client.25457 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.448+0000 7f89d0516b80 1 -- 172.21.15.39:0/3965011473 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558d70864c70 con 0x558d70a95270 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.449+0000 7f89a5ffb700 1 -- 172.21.15.39:0/3965011473 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89b800edb0 con 0x558d70a95270 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.449+0000 7f89a5ffb700 1 -- 172.21.15.39:0/3965011473 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f89b401e020 con 0x558d70a95270 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.449+0000 7f89d0516b80 10 client.25457 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.449+0000 7f89d0516b80 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.449+0000 7f89d0516b80 20 client.25457 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.449+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558d70b106a0 0x558d70b12ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.449+0000 7f89d0516b80 4 client.25457 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:42:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.449+0000 7f89d0516b80 1 -- 172.21.15.39:0/3965011473 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d709d7e60 con 0x558d70b106a0 2024-05-24T11:42:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.450+0000 7f89c20f7700 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558d70b106a0 0x558d70b12ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.450+0000 7f89c20f7700 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558d70b106a0 0x558d70b12ac0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eeef9b77adedffd server_cookie=fe561dc296233a64 in_seq=0 out_seq=0 2024-05-24T11:42:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.450+0000 7f89a5ffb700 10 client.25457 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:42:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:42:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:42:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:42:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.503+0000 7f89a5ffb700 1 -- 172.21.15.39:0/3965011473 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d709d7e60 con 0x558d70b106a0 2024-05-24T11:42:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.503+0000 7f89a5ffb700 10 client.25457 handle_command_reply: tid=0 2024-05-24T11:42:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.503+0000 7f89d0516b80 1 client.25457 shutdown 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.503+0000 7f89d0516b80 1 -- 172.21.15.39:0/3965011473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 msgr2=0x558d70a93380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a93380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89b4007b30 tx=0x7f89b4007ef0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 1 -- 172.21.15.39:0/3965011473 shutdown_connections 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d70a8f480 0x558d70ab0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558d70b106a0 0x558d70b12ac0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558d70a95270 0x558d70a93380 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 1 --2- 172.21.15.39:0/3965011473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558d70a94950 0x558d70a92e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 1 -- 172.21.15.39:0/3965011473 >> 172.21.15.39:0/3965011473 conn(0x558d709ed1b0 msgr2=0x558d70a8e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89a5ffb700 0 client.25457 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 1 -- 172.21.15.39:0/3965011473 shutdown_connections 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 1 -- 172.21.15.39:0/3965011473 wait complete. 2024-05-24T11:42:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.504+0000 7f89d0516b80 20 client.25457 trim_cache size 0 max 16384 2024-05-24T11:42:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 -- 172.21.15.39:0/473482474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f89a80a1310 msgr2=0x7f89a80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f89a80a1310 0x7f89a80a37d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f89b8005fd0 tx=0x7f89b8005ee0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 -- 172.21.15.39:0/473482474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89bc068bb0 msgr2=0x7f89bc1a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89bc068bb0 0x7f89bc1a1dc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f89ac004730 tx=0x7f89ac009570 comp rx=0 tx=0).stop 2024-05-24T11:42:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 -- 172.21.15.39:0/473482474 shutdown_connections 2024-05-24T11:42:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89bc068bb0 0x7f89bc1a1dc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f89a80a1310 0x7f89a80a37d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89bc10d950 0x7f89bc1a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 --2- 172.21.15.39:0/473482474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89bc107fa0 0x7f89bc1a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 -- 172.21.15.39:0/473482474 >> 172.21.15.39:0/473482474 conn(0x7f89bc06c3c0 msgr2=0x7f89bc10b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.508+0000 7f89c435b700 1 -- 172.21.15.39:0/473482474 shutdown_connections 2024-05-24T11:42:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:42.509+0000 7f89c435b700 1 -- 172.21.15.39:0/473482474 wait complete. 2024-05-24T11:42:44.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 200.0 s, 1455.4 tps, lat 68.767 ms stddev 18.576 2024-05-24T11:42:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:43 smithi053 ceph-mon[47448]: pgmap v138: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-05-24T11:42:44.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:43 smithi137 ceph-mon[47907]: pgmap v138: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-05-24T11:42:44.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:43 smithi039 ceph-mon[42919]: pgmap v138: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-05-24T11:42:46.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:45 smithi053 ceph-mon[47448]: pgmap v139: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 18 MiB/s wr, 1.83k op/s 2024-05-24T11:42:46.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:45 smithi137 ceph-mon[47907]: pgmap v139: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 18 MiB/s wr, 1.83k op/s 2024-05-24T11:42:46.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:45 smithi039 ceph-mon[42919]: pgmap v139: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 18 MiB/s wr, 1.83k op/s 2024-05-24T11:42:48.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:47 smithi053 ceph-mon[47448]: pgmap v140: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 12 MiB/s wr, 1.24k op/s 2024-05-24T11:42:48.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:47 smithi137 ceph-mon[47907]: pgmap v140: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 12 MiB/s wr, 1.24k op/s 2024-05-24T11:42:48.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:47 smithi039 ceph-mon[42919]: pgmap v140: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 12 MiB/s wr, 1.24k op/s 2024-05-24T11:42:49.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 205.0 s, 1394.6 tps, lat 71.607 ms stddev 20.254 2024-05-24T11:42:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:49 smithi053 ceph-mon[47448]: pgmap v141: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 22 MiB/s wr, 1.87k op/s 2024-05-24T11:42:50.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:49 smithi137 ceph-mon[47907]: pgmap v141: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 22 MiB/s wr, 1.87k op/s 2024-05-24T11:42:50.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:49 smithi039 ceph-mon[42919]: pgmap v141: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 22 MiB/s wr, 1.87k op/s 2024-05-24T11:42:52.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:51 smithi053 ceph-mon[47448]: pgmap v142: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:52.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:51 smithi137 ceph-mon[47907]: pgmap v142: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:52.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:51 smithi039 ceph-mon[42919]: pgmap v142: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 210.0 s, 1354.0 tps, lat 73.839 ms stddev 22.851 2024-05-24T11:42:54.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:53 smithi053 ceph-mon[47448]: pgmap v143: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:54.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:53 smithi137 ceph-mon[47907]: pgmap v143: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:54.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:53 smithi039 ceph-mon[42919]: pgmap v143: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:42:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.687+0000 7fef50b67700 1 -- 172.21.15.39:0/2045598267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c10f630 msgr2=0x7fef4c10fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.687+0000 7fef50b67700 1 --2- 172.21.15.39:0/2045598267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c10f630 0x7fef4c10fa10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fef4400b0a0 tx=0x7fef4400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.688+0000 7fef50b67700 1 -- 172.21.15.39:0/2045598267 shutdown_connections 2024-05-24T11:42:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.688+0000 7fef50b67700 1 --2- 172.21.15.39:0/2045598267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef4c107fa0 0x7fef4c108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.688+0000 7fef50b67700 1 --2- 172.21.15.39:0/2045598267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c10f630 0x7fef4c10fa10 secure :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fef4400b0a0 tx=0x7fef4400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.688+0000 7fef50b67700 1 --2- 172.21.15.39:0/2045598267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c10d440 0x7fef4c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.688+0000 7fef50b67700 1 -- 172.21.15.39:0/2045598267 >> 172.21.15.39:0/2045598267 conn(0x7fef4c06bf80 msgr2=0x7fef4c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.689+0000 7fef50b67700 1 -- 172.21.15.39:0/2045598267 shutdown_connections 2024-05-24T11:42:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.690+0000 7fef50b67700 1 -- 172.21.15.39:0/2045598267 wait complete. 2024-05-24T11:42:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.691+0000 7fef50b67700 1 Processor -- start 2024-05-24T11:42:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.691+0000 7fef50b67700 1 -- start start 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.691+0000 7fef50b67700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c107fa0 0x7fef4c1a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.691+0000 7fef50b67700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c10d440 0x7fef4c1a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.691+0000 7fef50b67700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef4c1aadd0 0x7fef4c1ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.691+0000 7fef50b67700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1ab850 con 0x7fef4c107fa0 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.691+0000 7fef50b67700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1ab9c0 con 0x7fef4c10d440 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.691+0000 7fef50b67700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1abb80 con 0x7fef4c1aadd0 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c107fa0 0x7fef4c1a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c107fa0 0x7fef4c1a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52490/0 (socket says 172.21.15.39:52490) 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4a59c700 1 -- 172.21.15.39:0/1804122486 learned_addr learned my addr 172.21.15.39:0/1804122486 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef49d9b700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c10d440 0x7fef4c1a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef4c1aadd0 0x7fef4c1ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4a59c700 1 -- 172.21.15.39:0/1804122486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef4c1aadd0 msgr2=0x7fef4c1ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4a59c700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef4c1aadd0 0x7fef4c1ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4a59c700 1 -- 172.21.15.39:0/1804122486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c10d440 msgr2=0x7fef4c1a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4a59c700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c10d440 0x7fef4c1a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4a59c700 1 -- 172.21.15.39:0/1804122486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef34009520 con 0x7fef4c107fa0 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef4a59c700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c107fa0 0x7fef4c1a64f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fef44006070 tx=0x7fef44006230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.692+0000 7fef437fe700 1 -- 172.21.15.39:0/1804122486 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4400e040 con 0x7fef4c107fa0 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.693+0000 7fef50b67700 1 -- 172.21.15.39:0/1804122486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef44009d00 con 0x7fef4c107fa0 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.693+0000 7fef437fe700 1 -- 172.21.15.39:0/1804122486 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef44003c00 con 0x7fef4c107fa0 2024-05-24T11:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.693+0000 7fef50b67700 1 -- 172.21.15.39:0/1804122486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c1b3320 con 0x7fef4c107fa0 2024-05-24T11:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.694+0000 7fef437fe700 1 -- 172.21.15.39:0/1804122486 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4401b460 con 0x7fef4c107fa0 2024-05-24T11:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.695+0000 7fef437fe700 1 -- 172.21.15.39:0/1804122486 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fef44005040 con 0x7fef4c107fa0 2024-05-24T11:42:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.696+0000 7fef437fe700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef380a1350 0x7fef380a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.696+0000 7fef437fe700 1 -- 172.21.15.39:0/1804122486 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fef440cd500 con 0x7fef4c107fa0 2024-05-24T11:42:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.696+0000 7fef49d9b700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef380a1350 0x7fef380a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.697+0000 7fef49d9b700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef380a1350 0x7fef380a3810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef4c1a7a40 tx=0x7fef3c009190 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.698+0000 7fef5cd22b80 1 Processor -- start 2024-05-24T11:42:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.698+0000 7fef5cd22b80 1 -- start start 2024-05-24T11:42:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.698+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d2950 0x564e750d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.698+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d3270 0x564e750ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.698+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.698+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e750240f0 con 0x564e750cd480 2024-05-24T11:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.698+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e75021ab0 con 0x564e750d2950 2024-05-24T11:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.698+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7501d3a0 con 0x564e750d3270 2024-05-24T11:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d2950 0x564e750d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4a59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d2950 0x564e750d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:54216/0 (socket says 172.21.15.39:54216) 2024-05-24T11:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4a59c700 1 -- 172.21.15.39:0/1759633049 learned_addr learned my addr 172.21.15.39:0/1759633049 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef49d9b700 1 --2- 172.21.15.39:0/1759633049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d3270 0x564e750ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/1759633049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4a59c700 1 -- 172.21.15.39:0/1759633049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d3270 msgr2=0x564e750ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4a59c700 1 --2- 172.21.15.39:0/1759633049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d3270 0x564e750ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4a59c700 1 -- 172.21.15.39:0/1759633049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 msgr2=0x564e750cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4a59c700 1 --2- 172.21.15.39:0/1759633049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4a59c700 1 -- 172.21.15.39:0/1759633049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e74e9e350 con 0x564e750d2950 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.699+0000 7fef4a59c700 1 --2- 172.21.15.39:0/1759633049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d2950 0x564e750d2d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef440cad80 tx=0x7fef44097410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=795a9e61f33fb30d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.700+0000 7fef40ff9700 1 -- 172.21.15.39:0/1759633049 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef440bb0b0 con 0x564e750d2950 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.700+0000 7fef40ff9700 1 -- 172.21.15.39:0/1759633049 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef4409b6d0 con 0x564e750d2950 2024-05-24T11:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.700+0000 7fef40ff9700 1 -- 172.21.15.39:0/1759633049 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44097de0 con 0x564e750d2950 2024-05-24T11:42:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.700+0000 7fef5cd22b80 1 -- 172.21.15.39:0/1759633049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d2950 msgr2=0x564e750d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.700+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/1759633049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d2950 0x564e750d2d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef440cad80 tx=0x7fef44097410 comp rx=0 tx=0).stop 2024-05-24T11:42:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.701+0000 7fef5cd22b80 1 -- 172.21.15.39:0/1759633049 shutdown_connections 2024-05-24T11:42:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.701+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/1759633049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.701+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/1759633049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d2950 0x564e750d2d30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.701+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/1759633049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d3270 0x564e750ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.701+0000 7fef5cd22b80 1 -- 172.21.15.39:0/1759633049 >> 172.21.15.39:0/1759633049 conn(0x564e7502b1b0 msgr2=0x564e750c7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.701+0000 7fef5cd22b80 1 -- 172.21.15.39:0/1759633049 shutdown_connections 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.701+0000 7fef5cd22b80 1 -- 172.21.15.39:0/1759633049 wait complete. 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.701+0000 7fef5cd22b80 1 -- start start 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef5cd22b80 1 Processor -- start 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750d9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 0x564e750d6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e74fb9500 con 0x564e750cd480 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e74fb91a0 con 0x564e750d3270 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e74ea0640 con 0x564e750d2950 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef5cd22b80 10 client.0 _ll_register_callbacks cb 0x564e74fc2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef4a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750d9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef4a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750d9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52510/0 (socket says 172.21.15.39:52510) 2024-05-24T11:42:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef4a59c700 1 -- 172.21.15.39:0/3056863615 learned_addr learned my addr 172.21.15.39:0/3056863615 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef49d9b700 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 0x564e750d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.702+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef4ad9d700 1 -- 172.21.15.39:0/3056863615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 msgr2=0x564e750d6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 0x564e750d6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef4ad9d700 1 -- 172.21.15.39:0/3056863615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 msgr2=0x564e750d9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750d9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef4ad9d700 1 -- 172.21.15.39:0/3056863615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c1b3320 con 0x564e750d3270 2024-05-24T11:42:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750d6c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef34013710 tx=0x7fef34009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd4af7d85482e190 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef31ffb700 1 -- 172.21.15.39:0/3056863615 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef34004830 con 0x564e750d3270 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef31ffb700 1 -- 172.21.15.39:0/3056863615 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fef34004990 con 0x564e750d3270 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.703+0000 7fef5cd22b80 1 -- 172.21.15.39:0/3056863615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e74f76210 con 0x564e750d3270 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.704+0000 7fef31ffb700 1 -- 172.21.15.39:0/3056863615 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef34004b50 con 0x564e750d3270 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.704+0000 7fef5cd22b80 10 client.25466 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.704+0000 7fef5cd22b80 1 -- 172.21.15.39:0/3056863615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564e74e4ab00 con 0x564e750d3270 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.704+0000 7fef31ffb700 1 -- 172.21.15.39:0/3056863615 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564e74f76210 con 0x564e750d3270 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.704+0000 7fef31ffb700 1 -- 172.21.15.39:0/3056863615 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fef3401c880 con 0x564e750d3270 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.704+0000 7fef5cd22b80 10 client.25466 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.704+0000 7fef5cd22b80 10 client.25466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.704+0000 7fef5cd22b80 20 client.25466 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.705+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564e750e6b40 0x564e750e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.705+0000 7fef5cd22b80 4 client.25466 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:42:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.705+0000 7fef5cd22b80 1 -- 172.21.15.39:0/3056863615 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e7509c7b0 con 0x564e750e6b40 2024-05-24T11:42:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.705+0000 7fef4a59c700 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564e750e6b40 0x564e750e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.705+0000 7fef4a59c700 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564e750e6b40 0x564e750e8f60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c5636a2cee4236 server_cookie=2d8301dcb1d31138 in_seq=0 out_seq=0 2024-05-24T11:42:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.705+0000 7fef31ffb700 10 client.25466 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.706+0000 7fef31ffb700 1 -- 172.21.15.39:0/3056863615 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e7509c7b0 con 0x564e750e6b40 2024-05-24T11:42:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.706+0000 7fef31ffb700 10 client.25466 handle_command_reply: tid=0 2024-05-24T11:42:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.706+0000 7fef5cd22b80 1 client.25466 shutdown 2024-05-24T11:42:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 -- 172.21.15.39:0/3056863615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 msgr2=0x564e750d6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750d6c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef34013710 tx=0x7fef34009f90 comp rx=0 tx=0).stop 2024-05-24T11:42:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 -- 172.21.15.39:0/3056863615 shutdown_connections 2024-05-24T11:42:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564e750e6b40 0x564e750e8f60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750d6c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750d9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef31ffb700 0 client.25466 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/3056863615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 0x564e750d6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 -- 172.21.15.39:0/3056863615 >> 172.21.15.39:0/3056863615 conn(0x564e7502b1b0 msgr2=0x564e750cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 -- 172.21.15.39:0/3056863615 shutdown_connections 2024-05-24T11:42:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.707+0000 7fef5cd22b80 1 -- 172.21.15.39:0/3056863615 wait complete. 2024-05-24T11:42:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.708+0000 7fef5cd22b80 20 client.25466 trim_cache size 0 max 16384 2024-05-24T11:42:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.730+0000 7fef5cd22b80 1 Processor -- start 2024-05-24T11:42:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.730+0000 7fef5cd22b80 1 -- start start 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750c7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 0x564e750e2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e750e4160 con 0x564e750cd480 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e750c9290 con 0x564e750d3270 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e750d6320 con 0x564e750d2950 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef4ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef4ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:54232/0 (socket says 172.21.15.39:54232) 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef4ad9d700 1 -- 172.21.15.39:0/444826258 learned_addr learned my addr 172.21.15.39:0/444826258 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef49d9b700 1 --2- 172.21.15.39:0/444826258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 0x564e750e2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef4a59c700 1 --2- 172.21.15.39:0/444826258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef4ad9d700 1 -- 172.21.15.39:0/444826258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 msgr2=0x564e750e2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/444826258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 0x564e750e2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef4ad9d700 1 -- 172.21.15.39:0/444826258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 msgr2=0x564e750c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/444826258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.731+0000 7fef4ad9d700 1 -- 172.21.15.39:0/444826258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e74fe3500 con 0x564e750d3270 2024-05-24T11:42:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.732+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/444826258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750c9520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fef34013f80 tx=0x7fef34013e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab768f2fbb8d6f4c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.732+0000 7fef417fa700 1 -- 172.21.15.39:0/444826258 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef340048b0 con 0x564e750d3270 2024-05-24T11:42:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.732+0000 7fef417fa700 1 -- 172.21.15.39:0/444826258 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef34004a10 con 0x564e750d3270 2024-05-24T11:42:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.732+0000 7fef417fa700 1 -- 172.21.15.39:0/444826258 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef34011e30 con 0x564e750d3270 2024-05-24T11:42:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 -- 172.21.15.39:0/444826258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 msgr2=0x564e750c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/444826258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750c9520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fef34013f80 tx=0x7fef34013e50 comp rx=0 tx=0).stop 2024-05-24T11:42:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 -- 172.21.15.39:0/444826258 shutdown_connections 2024-05-24T11:42:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/444826258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750c9520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/444826258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750cd480 0x564e750c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/444826258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750d2950 0x564e750e2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 -- 172.21.15.39:0/444826258 >> 172.21.15.39:0/444826258 conn(0x564e7502b1b0 msgr2=0x564e750c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 -- 172.21.15.39:0/444826258 shutdown_connections 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 -- 172.21.15.39:0/444826258 wait complete. 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 -- start start 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.733+0000 7fef5cd22b80 1 Processor -- start 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750cd480 0x564e750ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750d2950 0x564e750db9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef5cd22b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e750dc7c0 con 0x564e750d2950 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7513c040 con 0x564e750d3270 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef5cd22b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e7513c180 con 0x564e750cd480 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef5cd22b80 10 client.0 _ll_register_callbacks cb 0x564e74fc2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef4a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750cd480 0x564e750ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef4a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750cd480 0x564e750ddea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:51418/0 (socket says 172.21.15.39:51418) 2024-05-24T11:42:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef4a59c700 1 -- 172.21.15.39:0/898563999 learned_addr learned my addr 172.21.15.39:0/898563999 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:42:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef4ad9d700 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.734+0000 7fef49d9b700 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750d2950 0x564e750db9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef4a59c700 1 -- 172.21.15.39:0/898563999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 msgr2=0x564e750dc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef4a59c700 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750dc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef4a59c700 1 -- 172.21.15.39:0/898563999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750d2950 msgr2=0x564e750db9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:42:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef4a59c700 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750d2950 0x564e750db9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef4a59c700 1 -- 172.21.15.39:0/898563999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e74e4ab00 con 0x564e750cd480 2024-05-24T11:42:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef4a59c700 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750cd480 0x564e750ddea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef440062a0 tx=0x7fef4409b030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a0d90c4592871d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:42:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:42:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef31ffb700 1 -- 172.21.15.39:0/898563999 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4409f580 con 0x564e750cd480 2024-05-24T11:42:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef31ffb700 1 -- 172.21.15.39:0/898563999 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fef440a3ac0 con 0x564e750cd480 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.735+0000 7fef5cd22b80 1 -- 172.21.15.39:0/898563999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef4409b6d0 con 0x564e750cd480 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef31ffb700 1 -- 172.21.15.39:0/898563999 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef440caf10 con 0x564e750cd480 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef5cd22b80 10 client.25369 fetch_fsmap learned FSMap version 27 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef5cd22b80 1 -- 172.21.15.39:0/898563999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564e74ea2c70 con 0x564e750cd480 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef31ffb700 1 -- 172.21.15.39:0/898563999 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef4409b6d0 con 0x564e750cd480 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef31ffb700 1 -- 172.21.15.39:0/898563999 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fef4409f130 con 0x564e750cd480 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef5cd22b80 10 client.25369 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef5cd22b80 10 client.25369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef5cd22b80 20 client.25369 populate_metadata read hostname 'smithi039' 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564e7513d320 0x564e7513f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef5cd22b80 4 client.25369 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef5cd22b80 1 -- 172.21.15.39:0/898563999 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564e75015e60 con 0x564e7513d320 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.736+0000 7fef49d9b700 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564e7513d320 0x564e7513f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.737+0000 7fef49d9b700 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564e7513d320 0x564e7513f710 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ef6fe0cb7672cb5 server_cookie=cc810f58a82c1b57 in_seq=0 out_seq=0 2024-05-24T11:42:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:55.737+0000 7fef31ffb700 10 client.25369 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:56.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:55 smithi053 ceph-mon[47448]: pgmap v144: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 25 MiB/s wr, 1.83k op/s 2024-05-24T11:42:56.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:55 smithi137 ceph-mon[47907]: pgmap v144: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 25 MiB/s wr, 1.83k op/s 2024-05-24T11:42:56.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:55 smithi039 ceph-mon[42919]: pgmap v144: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 25 MiB/s wr, 1.83k op/s 2024-05-24T11:42:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.954+0000 7fef31ffb700 1 -- 172.21.15.39:0/898563999 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564e75015e60 con 0x564e7513d320 2024-05-24T11:42:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.954+0000 7fef31ffb700 10 client.25369 handle_command_reply: tid=0 2024-05-24T11:42:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.954+0000 7fef5cd22b80 1 client.25369 shutdown 2024-05-24T11:42:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 -- 172.21.15.39:0/898563999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750cd480 msgr2=0x564e750ddea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750cd480 0x564e750ddea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef440062a0 tx=0x7fef4409b030 comp rx=0 tx=0).stop 2024-05-24T11:42:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 -- 172.21.15.39:0/898563999 shutdown_connections 2024-05-24T11:42:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e750cd480 0x564e750ddea0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fef440062a0 tx=0x7fef4409b030 comp rx=0 tx=0).stop 2024-05-24T11:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564e7513d320 0x564e7513f710 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e750d3270 0x564e750dc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 --2- 172.21.15.39:0/898563999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564e750d2950 0x564e750db9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 -- 172.21.15.39:0/898563999 >> 172.21.15.39:0/898563999 conn(0x564e7502b1b0 msgr2=0x564e750ed8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef31ffb700 0 client.25369 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 -- 172.21.15.39:0/898563999 shutdown_connections 2024-05-24T11:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 1 -- 172.21.15.39:0/898563999 wait complete. 2024-05-24T11:42:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.955+0000 7fef5cd22b80 20 client.25369 trim_cache size 0 max 16384 2024-05-24T11:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:42:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 -- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef380a1350 msgr2=0x7fef380a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef380a1350 0x7fef380a3810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef4c1a7a40 tx=0x7fef3c009190 comp rx=0 tx=0).stop 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 -- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c107fa0 msgr2=0x7fef4c1a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c107fa0 0x7fef4c1a64f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fef44006070 tx=0x7fef44006230 comp rx=0 tx=0).stop 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 -- 172.21.15.39:0/1804122486 shutdown_connections 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef4c1aadd0 0x7fef4c1ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef380a1350 0x7fef380a3810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef4c107fa0 0x7fef4c1a64f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 --2- 172.21.15.39:0/1804122486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef4c10d440 0x7fef4c1a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.956+0000 7fef50b67700 1 -- 172.21.15.39:0/1804122486 >> 172.21.15.39:0/1804122486 conn(0x7fef4c06bf80 msgr2=0x7fef4c1090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.957+0000 7fef50b67700 1 -- 172.21.15.39:0/1804122486 shutdown_connections 2024-05-24T11:42:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:42:56.957+0000 7fef50b67700 1 -- 172.21.15.39:0/1804122486 wait complete. 2024-05-24T11:42:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:57 smithi053 ceph-mon[47448]: pgmap v145: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-05-24T11:42:58.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:57 smithi137 ceph-mon[47907]: pgmap v145: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-05-24T11:42:58.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:57 smithi039 ceph-mon[42919]: pgmap v145: 97 pgs: 97 active+clean; 3.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-05-24T11:42:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 215.0 s, 1436.8 tps, lat 69.622 ms stddev 19.037 2024-05-24T11:43:00.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:42:59 smithi053 ceph-mon[47448]: pgmap v146: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 27 MiB/s wr, 1.83k op/s 2024-05-24T11:43:00.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:42:59 smithi137 ceph-mon[47907]: pgmap v146: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 27 MiB/s wr, 1.83k op/s 2024-05-24T11:43:00.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:42:59 smithi039 ceph-mon[42919]: pgmap v146: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 27 MiB/s wr, 1.83k op/s 2024-05-24T11:43:02.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:01 smithi053 ceph-mon[47448]: pgmap v147: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-05-24T11:43:02.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:01 smithi137 ceph-mon[47907]: pgmap v147: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-05-24T11:43:02.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:01 smithi039 ceph-mon[42919]: pgmap v147: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-05-24T11:43:04.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 220.0 s, 1213.6 tps, lat 82.422 ms stddev 45.254 2024-05-24T11:43:04.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:03 smithi053 ceph-mon[47448]: pgmap v148: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-05-24T11:43:04.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:03 smithi137 ceph-mon[47907]: pgmap v148: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-05-24T11:43:04.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:03 smithi039 ceph-mon[42919]: pgmap v148: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-05-24T11:43:06.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:05 smithi053 ceph-mon[47448]: pgmap v149: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 25 MiB/s wr, 1.79k op/s 2024-05-24T11:43:06.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:05 smithi137 ceph-mon[47907]: pgmap v149: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 25 MiB/s wr, 1.79k op/s 2024-05-24T11:43:06.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:05 smithi039 ceph-mon[42919]: pgmap v149: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 25 MiB/s wr, 1.79k op/s 2024-05-24T11:43:08.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:07 smithi053 ceph-mon[47448]: pgmap v150: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2024-05-24T11:43:08.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:07 smithi137 ceph-mon[47907]: pgmap v150: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2024-05-24T11:43:08.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:07 smithi039 ceph-mon[42919]: pgmap v150: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2024-05-24T11:43:09.118 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 225.0 s, 1272.2 tps, lat 78.591 ms stddev 22.899 2024-05-24T11:43:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.127+0000 7f024bff4700 1 -- 172.21.15.39:0/2896621250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02440ff410 msgr2=0x7f02440ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.127+0000 7f024bff4700 1 --2- 172.21.15.39:0/2896621250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02440ff410 0x7f02440ff890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f023800b0a0 tx=0x7f023800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.127+0000 7f024bff4700 1 -- 172.21.15.39:0/2896621250 shutdown_connections 2024-05-24T11:43:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.127+0000 7f024bff4700 1 --2- 172.21.15.39:0/2896621250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02440ffdd0 0x7f024410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.127+0000 7f024bff4700 1 --2- 172.21.15.39:0/2896621250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0244107020 0x7f0244107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.127+0000 7f024bff4700 1 --2- 172.21.15.39:0/2896621250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02440ff410 0x7f02440ff890 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.128+0000 7f024bff4700 1 -- 172.21.15.39:0/2896621250 >> 172.21.15.39:0/2896621250 conn(0x7f02440763a0 msgr2=0x7f02440fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.128+0000 7f024bff4700 1 -- 172.21.15.39:0/2896621250 shutdown_connections 2024-05-24T11:43:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.128+0000 7f024bff4700 1 -- 172.21.15.39:0/2896621250 wait complete. 2024-05-24T11:43:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.129+0000 7f024bff4700 1 Processor -- start 2024-05-24T11:43:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.129+0000 7f024bff4700 1 -- start start 2024-05-24T11:43:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.129+0000 7f024bff4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02440ff410 0x7f024410dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.129+0000 7f024bff4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02440ffdd0 0x7f024410bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.129+0000 7f024bff4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0244107020 0x7f024410c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.129+0000 7f024bff4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024410caa0 con 0x7f02440ffdd0 2024-05-24T11:43:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.129+0000 7f024bff4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024410cc10 con 0x7f0244107020 2024-05-24T11:43:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.129+0000 7f024bff4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0244112a00 con 0x7f02440ff410 2024-05-24T11:43:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f024958f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02440ffdd0 0x7f024410bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f024958f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02440ffdd0 0x7f024410bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49158/0 (socket says 172.21.15.39:49158) 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f024958f700 1 -- 172.21.15.39:0/1323738835 learned_addr learned my addr 172.21.15.39:0/1323738835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f0249d90700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02440ff410 0x7f024410dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f024a591700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0244107020 0x7f024410c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f0249d90700 1 -- 172.21.15.39:0/1323738835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0244107020 msgr2=0x7f024410c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f0249d90700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0244107020 0x7f024410c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f0249d90700 1 -- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02440ffdd0 msgr2=0x7f024410bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f0249d90700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02440ffdd0 0x7f024410bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f0249d90700 1 -- 172.21.15.39:0/1323738835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0238009d00 con 0x7f02440ff410 2024-05-24T11:43:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f024958f700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02440ffdd0 0x7f024410bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:43:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f024a591700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0244107020 0x7f024410c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:43:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f0249d90700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02440ff410 0x7f024410dbf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f024000b450 tx=0x7f024000b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.130+0000 7f0236ffd700 1 -- 172.21.15.39:0/1323738835 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240011830 con 0x7f02440ff410 2024-05-24T11:43:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.131+0000 7f024bff4700 1 -- 172.21.15.39:0/1323738835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0244112ce0 con 0x7f02440ff410 2024-05-24T11:43:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.131+0000 7f024bff4700 1 -- 172.21.15.39:0/1323738835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0244113240 con 0x7f02440ff410 2024-05-24T11:43:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.131+0000 7f0236ffd700 1 -- 172.21.15.39:0/1323738835 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0240011e20 con 0x7f02440ff410 2024-05-24T11:43:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.131+0000 7f0236ffd700 1 -- 172.21.15.39:0/1323738835 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240010d10 con 0x7f02440ff410 2024-05-24T11:43:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.136+0000 7f02581afb80 1 Processor -- start 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f02581afb80 1 -- start start 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f02581afb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f1126f950 0x559f1126fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f02581afb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 0x559f11269f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f02581afb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1126a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f02581afb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f111c10f0 con 0x559f11270270 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f02581afb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f111beab0 con 0x559f1126f950 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f02581afb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f111ba3a0 con 0x559f1126a480 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f024958f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1126a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f024958f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1126a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:46048/0 (socket says 172.21.15.39:46048) 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.137+0000 7f024958f700 1 -- 172.21.15.39:0/2634370253 learned_addr learned my addr 172.21.15.39:0/2634370253 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f024958f700 1 -- 172.21.15.39:0/2634370253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f1126f950 msgr2=0x559f1126fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f024958f700 1 --2- 172.21.15.39:0/2634370253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f1126f950 0x559f1126fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f024958f700 1 -- 172.21.15.39:0/2634370253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 msgr2=0x559f11269f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f024958f700 1 --2- 172.21.15.39:0/2634370253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 0x559f11269f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f024958f700 1 -- 172.21.15.39:0/2634370253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f1103b350 con 0x559f1126a480 2024-05-24T11:43:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f024958f700 1 --2- 172.21.15.39:0/2634370253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1126a8a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f023800b070 tx=0x7f0238009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d3a2c3f9777a543 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f0236ffd700 1 -- 172.21.15.39:0/1323738835 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f024001a400 con 0x7f02440ff410 2024-05-24T11:43:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f022ffff700 1 -- 172.21.15.39:0/2634370253 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023800e040 con 0x559f1126a480 2024-05-24T11:43:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f022ffff700 1 -- 172.21.15.39:0/2634370253 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02380094d0 con 0x559f1126a480 2024-05-24T11:43:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.138+0000 7f022ffff700 1 -- 172.21.15.39:0/2634370253 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023801b400 con 0x559f1126a480 2024-05-24T11:43:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.139+0000 7f0236ffd700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f02300a1390 0x7f02300a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.139+0000 7f0236ffd700 1 -- 172.21.15.39:0/1323738835 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f02400cc530 con 0x7f02440ff410 2024-05-24T11:43:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.139+0000 7f02581afb80 1 -- 172.21.15.39:0/2634370253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 msgr2=0x559f1126a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.139+0000 7f02581afb80 1 --2- 172.21.15.39:0/2634370253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1126a8a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f023800b070 tx=0x7f0238009200 comp rx=0 tx=0).stop 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.140+0000 7f02581afb80 1 -- 172.21.15.39:0/2634370253 shutdown_connections 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.140+0000 7f02581afb80 1 --2- 172.21.15.39:0/2634370253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1126a8a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.140+0000 7f02581afb80 1 --2- 172.21.15.39:0/2634370253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f1126f950 0x559f1126fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.140+0000 7f02581afb80 1 --2- 172.21.15.39:0/2634370253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 0x559f11269f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.140+0000 7f02581afb80 1 -- 172.21.15.39:0/2634370253 >> 172.21.15.39:0/2634370253 conn(0x559f111c81b0 msgr2=0x559f11264840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.140+0000 7f024a591700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f02300a1390 0x7f02300a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.140+0000 7f02581afb80 1 -- 172.21.15.39:0/2634370253 shutdown_connections 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.140+0000 7f02581afb80 1 -- 172.21.15.39:0/2634370253 wait complete. 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.141+0000 7f02581afb80 1 -- start start 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.141+0000 7f024a591700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f02300a1390 0x7f02300a3850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f024410e2f0 tx=0x7f023c008040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.141+0000 7f02581afb80 1 Processor -- start 2024-05-24T11:43:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.142+0000 7f02581afb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f112735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.142+0000 7f02581afb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f11273b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.142+0000 7f02581afb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f11274050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.142+0000 7f02581afb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f11156500 con 0x559f1126f950 2024-05-24T11:43:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.142+0000 7f02581afb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f111561a0 con 0x559f11270270 2024-05-24T11:43:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.142+0000 7f02581afb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f1103d640 con 0x559f1126a480 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f02581afb80 10 client.0 _ll_register_callbacks cb 0x559f1115fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f024a591700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f11274050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f024a591700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f11274050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49768/0 (socket says 172.21.15.39:49768) 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f024a591700 1 -- 172.21.15.39:0/2765670223 learned_addr learned my addr 172.21.15.39:0/2765670223 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f024958f700 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f11273b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f024a591700 1 -- 172.21.15.39:0/2765670223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 msgr2=0x559f112735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f024a591700 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f112735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f024a591700 1 -- 172.21.15.39:0/2765670223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 msgr2=0x559f11273b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f024a591700 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f11273b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.143+0000 7f024a591700 1 -- 172.21.15.39:0/2765670223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02441006e0 con 0x559f11270270 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.144+0000 7f024a591700 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f11274050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f023c014d10 tx=0x7f023c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e2bbad0395db5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.144+0000 7f022d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:43:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.144+0000 7f022d7fa700 1 -- 172.21.15.39:0/2765670223 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c012e10 con 0x559f11270270 2024-05-24T11:43:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.144+0000 7f02581afb80 1 -- 172.21.15.39:0/2765670223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f11113210 con 0x559f11270270 2024-05-24T11:43:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.144+0000 7f022d7fa700 1 -- 172.21.15.39:0/2765670223 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f023c014e00 con 0x559f11270270 2024-05-24T11:43:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.144+0000 7f022d7fa700 1 -- 172.21.15.39:0/2765670223 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f023c029470 con 0x559f11270270 2024-05-24T11:43:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.144+0000 7f02581afb80 10 client.25478 fetch_fsmap learned FSMap version 27 2024-05-24T11:43:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.144+0000 7f02581afb80 1 -- 172.21.15.39:0/2765670223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559f10fe7b00 con 0x559f11270270 2024-05-24T11:43:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.144+0000 7f022d7fa700 1 -- 172.21.15.39:0/2765670223 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559f11113210 con 0x559f11270270 2024-05-24T11:43:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.145+0000 7f022d7fa700 1 -- 172.21.15.39:0/2765670223 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f023c013400 con 0x559f11270270 2024-05-24T11:43:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.145+0000 7f02581afb80 10 client.25478 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:43:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.145+0000 7f02581afb80 10 client.25478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:43:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.145+0000 7f02581afb80 20 client.25478 populate_metadata read hostname 'smithi039' 2024-05-24T11:43:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.145+0000 7f02581afb80 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559f11283c80 0x559f112860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.145+0000 7f02581afb80 4 client.25478 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:43:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.145+0000 7f02581afb80 1 -- 172.21.15.39:0/2765670223 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f112397b0 con 0x559f11283c80 2024-05-24T11:43:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.145+0000 7f0249d90700 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559f11283c80 0x559f112860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.146+0000 7f0249d90700 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559f11283c80 0x559f112860a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b8115d061b0277 server_cookie=761598712f41e6ae in_seq=0 out_seq=0 2024-05-24T11:43:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.146+0000 7f022d7fa700 10 client.25478 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.147+0000 7f022d7fa700 1 -- 172.21.15.39:0/2765670223 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f112397b0 con 0x559f11283c80 2024-05-24T11:43:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.147+0000 7f022d7fa700 10 client.25478 handle_command_reply: tid=0 2024-05-24T11:43:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.147+0000 7f02581afb80 1 client.25478 shutdown 2024-05-24T11:43:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 -- 172.21.15.39:0/2765670223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 msgr2=0x559f11274050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f11274050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f023c014d10 tx=0x7f023c004770 comp rx=0 tx=0).stop 2024-05-24T11:43:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 -- 172.21.15.39:0/2765670223 shutdown_connections 2024-05-24T11:43:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f112735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559f11283c80 0x559f112860a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f11274050 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 --2- 172.21.15.39:0/2765670223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f11273b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 -- 172.21.15.39:0/2765670223 >> 172.21.15.39:0/2765670223 conn(0x559f111c81b0 msgr2=0x559f1126e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f022d7fa700 0 client.25478 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 -- 172.21.15.39:0/2765670223 shutdown_connections 2024-05-24T11:43:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.148+0000 7f02581afb80 1 -- 172.21.15.39:0/2765670223 wait complete. 2024-05-24T11:43:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.149+0000 7f02581afb80 20 client.25478 trim_cache size 0 max 16384 2024-05-24T11:43:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.172+0000 7f02581afb80 1 Processor -- start 2024-05-24T11:43:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.172+0000 7f02581afb80 1 -- start start 2024-05-24T11:43:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.172+0000 7f02581afb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f1126a480 0x559f11264910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.172+0000 7f02581afb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126f950 0x559f11272eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f02581afb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 0x559f112664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f02581afb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f11273420 con 0x559f11270270 2024-05-24T11:43:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f02581afb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f11274bb0 con 0x559f1126a480 2024-05-24T11:43:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f02581afb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f112669f0 con 0x559f1126f950 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024a591700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 0x559f112664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024a591700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 0x559f112664b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49184/0 (socket says 172.21.15.39:49184) 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024a591700 1 -- 172.21.15.39:0/1387032084 learned_addr learned my addr 172.21.15.39:0/1387032084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024958f700 1 --2- 172.21.15.39:0/1387032084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126f950 0x559f11272eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f0249d90700 1 --2- 172.21.15.39:0/1387032084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f1126a480 0x559f11264910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024a591700 1 -- 172.21.15.39:0/1387032084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126f950 msgr2=0x559f11272eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024a591700 1 --2- 172.21.15.39:0/1387032084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126f950 0x559f11272eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024a591700 1 -- 172.21.15.39:0/1387032084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f1126a480 msgr2=0x559f11264910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024a591700 1 --2- 172.21.15.39:0/1387032084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f1126a480 0x559f11264910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024a591700 1 -- 172.21.15.39:0/1387032084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f11180500 con 0x559f11270270 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.173+0000 7f024a591700 1 --2- 172.21.15.39:0/1387032084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 0x559f112664b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f023c028000 tx=0x7f023c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6389f538de5e7a2d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.174+0000 7f0234ff9700 1 -- 172.21.15.39:0/1387032084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c012e10 con 0x559f11270270 2024-05-24T11:43:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.174+0000 7f0234ff9700 1 -- 172.21.15.39:0/1387032084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f023c005e90 con 0x559f11270270 2024-05-24T11:43:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.174+0000 7f0234ff9700 1 -- 172.21.15.39:0/1387032084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c006280 con 0x559f11270270 2024-05-24T11:43:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.175+0000 7f02581afb80 1 -- 172.21.15.39:0/1387032084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 msgr2=0x559f112664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.175+0000 7f02581afb80 1 --2- 172.21.15.39:0/1387032084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 0x559f112664b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f023c028000 tx=0x7f023c012000 comp rx=0 tx=0).stop 2024-05-24T11:43:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.175+0000 7f02581afb80 1 -- 172.21.15.39:0/1387032084 shutdown_connections 2024-05-24T11:43:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.175+0000 7f02581afb80 1 --2- 172.21.15.39:0/1387032084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f11270270 0x559f112664b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.175+0000 7f02581afb80 1 --2- 172.21.15.39:0/1387032084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f1126a480 0x559f11264910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.175+0000 7f02581afb80 1 --2- 172.21.15.39:0/1387032084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126f950 0x559f11272eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.175+0000 7f02581afb80 1 -- 172.21.15.39:0/1387032084 >> 172.21.15.39:0/1387032084 conn(0x559f111c81b0 msgr2=0x559f112695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.176+0000 7f02581afb80 1 -- 172.21.15.39:0/1387032084 shutdown_connections 2024-05-24T11:43:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.176+0000 7f02581afb80 1 -- 172.21.15.39:0/1387032084 wait complete. 2024-05-24T11:43:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.176+0000 7f02581afb80 1 -- start start 2024-05-24T11:43:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.176+0000 7f02581afb80 1 Processor -- start 2024-05-24T11:43:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f02581afb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1127b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f02581afb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f112d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f02581afb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f112d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f02581afb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f112d2170 con 0x559f1126f950 2024-05-24T11:43:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f02581afb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f112d22e0 con 0x559f11270270 2024-05-24T11:43:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f02581afb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f112d24a0 con 0x559f1126a480 2024-05-24T11:43:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f02581afb80 10 client.0 _ll_register_callbacks cb 0x559f1115fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:43:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f024958f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f112d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f024958f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f112d32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49198/0 (socket says 172.21.15.39:49198) 2024-05-24T11:43:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f024958f700 1 -- 172.21.15.39:0/2463580110 learned_addr learned my addr 172.21.15.39:0/2463580110 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f0249d90700 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1127b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.177+0000 7f024a591700 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f112d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.178+0000 7f024958f700 1 -- 172.21.15.39:0/2463580110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 msgr2=0x559f1127b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.178+0000 7f024958f700 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1127b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.178+0000 7f024958f700 1 -- 172.21.15.39:0/2463580110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 msgr2=0x559f112d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.178+0000 7f024958f700 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f112d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.178+0000 7f024958f700 1 -- 172.21.15.39:0/2463580110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f10fe7b00 con 0x559f1126f950 2024-05-24T11:43:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.178+0000 7f024958f700 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f112d32d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0238009320 tx=0x7f0238025710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5e51332ad351ca server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.178+0000 7f022d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:43:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f022d7fa700 1 -- 172.21.15.39:0/2463580110 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023800e040 con 0x559f1126f950 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f022d7fa700 1 -- 172.21.15.39:0/2463580110 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f0238003a00 con 0x559f1126f950 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f02581afb80 1 -- 172.21.15.39:0/2463580110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02380094d0 con 0x559f1126f950 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f022d7fa700 1 -- 172.21.15.39:0/2463580110 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0238003bc0 con 0x559f1126f950 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f02581afb80 10 client.15792 fetch_fsmap learned FSMap version 27 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f02581afb80 1 -- 172.21.15.39:0/2463580110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559f1103fc70 con 0x559f1126f950 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f022d7fa700 1 -- 172.21.15.39:0/2463580110 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02380094d0 con 0x559f1126f950 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f022d7fa700 1 -- 172.21.15.39:0/2463580110 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f023802f070 con 0x559f1126f950 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f02581afb80 10 client.15792 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f02581afb80 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f02581afb80 20 client.15792 populate_metadata read hostname 'smithi039' 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f02581afb80 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559f112eb6b0 0x559f112edad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f02581afb80 4 client.15792 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:43:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.179+0000 7f02581afb80 1 -- 172.21.15.39:0/2463580110 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f111b2e60 con 0x559f112eb6b0 2024-05-24T11:43:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.180+0000 7f0249d90700 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559f112eb6b0 0x559f112edad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.180+0000 7f0249d90700 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559f112eb6b0 0x559f112edad0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ec97f61c2881f4 server_cookie=6ee1cefd9a3347c2 in_seq=0 out_seq=0 2024-05-24T11:43:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.180+0000 7f022d7fa700 10 client.15792 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:10.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:09 smithi053 ceph-mon[47448]: pgmap v151: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 24 MiB/s wr, 1.77k op/s 2024-05-24T11:43:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.233+0000 7f022d7fa700 1 -- 172.21.15.39:0/2463580110 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f111b2e60 con 0x559f112eb6b0 2024-05-24T11:43:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.233+0000 7f022d7fa700 10 client.15792 handle_command_reply: tid=0 2024-05-24T11:43:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.233+0000 7f02581afb80 1 client.15792 shutdown 2024-05-24T11:43:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.234+0000 7f02581afb80 1 -- 172.21.15.39:0/2463580110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 msgr2=0x559f112d32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.234+0000 7f02581afb80 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f112d32d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0238009320 tx=0x7f0238025710 comp rx=0 tx=0).stop 2024-05-24T11:43:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.235+0000 7f02581afb80 1 -- 172.21.15.39:0/2463580110 shutdown_connections 2024-05-24T11:43:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.235+0000 7f02581afb80 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f1126a480 0x559f1127b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.235+0000 7f02581afb80 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559f112eb6b0 0x559f112edad0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.235+0000 7f02581afb80 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559f11270270 0x559f112d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.235+0000 7f02581afb80 1 --2- 172.21.15.39:0/2463580110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f1126f950 0x559f112d32d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.235+0000 7f02581afb80 1 -- 172.21.15.39:0/2463580110 >> 172.21.15.39:0/2463580110 conn(0x559f111c81b0 msgr2=0x559f11270650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.235+0000 7f022d7fa700 0 client.15792 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.236+0000 7f02581afb80 1 -- 172.21.15.39:0/2463580110 shutdown_connections 2024-05-24T11:43:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.236+0000 7f02581afb80 1 -- 172.21.15.39:0/2463580110 wait complete. 2024-05-24T11:43:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.236+0000 7f02581afb80 20 client.15792 trim_cache size 0 max 16384 2024-05-24T11:43:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:43:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:43:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:43:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:43:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.236+0000 7f024bff4700 1 -- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f02300a1390 msgr2=0x7f02300a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.236+0000 7f024bff4700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f02300a1390 0x7f02300a3850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f024410e2f0 tx=0x7f023c008040 comp rx=0 tx=0).stop 2024-05-24T11:43:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.236+0000 7f024bff4700 1 -- 172.21.15.39:0/1323738835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02440ff410 msgr2=0x7f024410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.236+0000 7f024bff4700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02440ff410 0x7f024410dbf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f024000b450 tx=0x7f024000b760 comp rx=0 tx=0).stop 2024-05-24T11:43:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.237+0000 7f024bff4700 1 -- 172.21.15.39:0/1323738835 shutdown_connections 2024-05-24T11:43:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.237+0000 7f024bff4700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02440ff410 0x7f024410dbf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.237+0000 7f024bff4700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f02300a1390 0x7f02300a3850 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.237+0000 7f024bff4700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0244107020 0x7f024410c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.237+0000 7f024bff4700 1 --2- 172.21.15.39:0/1323738835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02440ffdd0 0x7f024410bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.237+0000 7f024bff4700 1 -- 172.21.15.39:0/1323738835 >> 172.21.15.39:0/1323738835 conn(0x7f02440763a0 msgr2=0x7f02441041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.237+0000 7f024bff4700 1 -- 172.21.15.39:0/1323738835 shutdown_connections 2024-05-24T11:43:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:10.237+0000 7f024bff4700 1 -- 172.21.15.39:0/1323738835 wait complete. 2024-05-24T11:43:10.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:09 smithi137 ceph-mon[47907]: pgmap v151: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 24 MiB/s wr, 1.77k op/s 2024-05-24T11:43:10.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:09 smithi039 ceph-mon[42919]: pgmap v151: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 24 MiB/s wr, 1.77k op/s 2024-05-24T11:43:12.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:11 smithi053 ceph-mon[47448]: pgmap v152: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:43:12.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:11 smithi137 ceph-mon[47907]: pgmap v152: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:43:12.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:11 smithi039 ceph-mon[42919]: pgmap v152: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:43:14.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 230.0 s, 1338.4 tps, lat 74.827 ms stddev 24.298 2024-05-24T11:43:14.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:13 smithi053 ceph-mon[47448]: pgmap v153: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:43:14.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:13 smithi137 ceph-mon[47907]: pgmap v153: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:43:14.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:13 smithi039 ceph-mon[42919]: pgmap v153: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:43:16.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:15 smithi053 ceph-mon[47448]: pgmap v154: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2024-05-24T11:43:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:15 smithi137 ceph-mon[47907]: pgmap v154: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2024-05-24T11:43:16.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:15 smithi039 ceph-mon[42919]: pgmap v154: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2024-05-24T11:43:18.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:17 smithi053 ceph-mon[47448]: pgmap v155: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2024-05-24T11:43:18.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:17 smithi137 ceph-mon[47907]: pgmap v155: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2024-05-24T11:43:18.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:17 smithi039 ceph-mon[42919]: pgmap v155: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2024-05-24T11:43:19.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 235.0 s, 1412.8 tps, lat 70.780 ms stddev 19.839 2024-05-24T11:43:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:19 smithi053 ceph-mon[47448]: pgmap v156: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 22 MiB/s wr, 1.73k op/s 2024-05-24T11:43:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:43:20.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:19 smithi137 ceph-mon[47907]: pgmap v156: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 22 MiB/s wr, 1.73k op/s 2024-05-24T11:43:20.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:43:20.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:19 smithi039 ceph-mon[42919]: pgmap v156: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 22 MiB/s wr, 1.73k op/s 2024-05-24T11:43:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:43:21.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:20 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:43:21.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:20 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:43:21.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:20 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:43:21.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:20 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:43:21.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:20 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:43:21.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:20 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:43:21.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:20 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:43:21.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:20 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:43:21.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:20 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:43:22.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:21 smithi053 ceph-mon[47448]: pgmap v157: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:43:22.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:21 smithi137 ceph-mon[47907]: pgmap v157: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:43:22.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:21 smithi039 ceph-mon[42919]: pgmap v157: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:43:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.418+0000 7f9b56504700 1 -- 172.21.15.39:0/2958528688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5011a640 msgr2=0x7f9b5011ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.418+0000 7f9b56504700 1 --2- 172.21.15.39:0/2958528688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5011a640 0x7f9b5011ca30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009990 tx=0x7f9b48009ca0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.418+0000 7f9b56504700 1 -- 172.21.15.39:0/2958528688 shutdown_connections 2024-05-24T11:43:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.418+0000 7f9b56504700 1 --2- 172.21.15.39:0/2958528688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5011a640 0x7f9b5011ca30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.418+0000 7f9b56504700 1 --2- 172.21.15.39:0/2958528688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5006c970 0x7f9b5006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.418+0000 7f9b56504700 1 --2- 172.21.15.39:0/2958528688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b5006d290 0x7f9b5011a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.418+0000 7f9b56504700 1 -- 172.21.15.39:0/2958528688 >> 172.21.15.39:0/2958528688 conn(0x7f9b5006c020 msgr2=0x7f9b50109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.418+0000 7f9b56504700 1 -- 172.21.15.39:0/2958528688 shutdown_connections 2024-05-24T11:43:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.419+0000 7f9b56504700 1 -- 172.21.15.39:0/2958528688 wait complete. 2024-05-24T11:43:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.419+0000 7f9b56504700 1 Processor -- start 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.419+0000 7f9b56504700 1 -- start start 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b56504700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b5006c970 0x7f9b501a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b56504700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5006d290 0x7f9b501a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b56504700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5011a640 0x7f9b501aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b56504700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b501aaf20 con 0x7f9b5006c970 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b56504700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b501ab090 con 0x7f9b5011a640 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b56504700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b501ab250 con 0x7f9b5006d290 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b477fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5006d290 0x7f9b501a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b54aa1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5011a640 0x7f9b501aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b477fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5006d290 0x7f9b501a6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47398/0 (socket says 172.21.15.39:47398) 2024-05-24T11:43:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b477fe700 1 -- 172.21.15.39:0/2641945367 learned_addr learned my addr 172.21.15.39:0/2641945367 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b5006c970 0x7f9b501a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b477fe700 1 -- 172.21.15.39:0/2641945367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5011a640 msgr2=0x7f9b501aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b477fe700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5011a640 0x7f9b501aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b477fe700 1 -- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b5006c970 msgr2=0x7f9b501a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b477fe700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b5006c970 0x7f9b501a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b477fe700 1 -- 172.21.15.39:0/2641945367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b480095f0 con 0x7f9b5006d290 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.420+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b5006c970 0x7f9b501a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.421+0000 7f9b477fe700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5006d290 0x7f9b501a6540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c9b0 tx=0x7f9b3800cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.421+0000 7f9b4dffb700 1 -- 172.21.15.39:0/2641945367 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38007b00 con 0x7f9b5006d290 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.421+0000 7f9b56504700 1 -- 172.21.15.39:0/2641945367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b501ab530 con 0x7f9b5006d290 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.421+0000 7f9b4dffb700 1 -- 172.21.15.39:0/2641945367 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b38007c60 con 0x7f9b5006d290 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.421+0000 7f9b4dffb700 1 -- 172.21.15.39:0/2641945367 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38018660 con 0x7f9b5006d290 2024-05-24T11:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.421+0000 7f9b56504700 1 -- 172.21.15.39:0/2641945367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b501b2d70 con 0x7f9b5006d290 2024-05-24T11:43:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.429+0000 7f9b4dffb700 1 -- 172.21.15.39:0/2641945367 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9b38004830 con 0x7f9b5006d290 2024-05-24T11:43:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.429+0000 7f9b626bfb80 1 Processor -- start 2024-05-24T11:43:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.429+0000 7f9b4dffb700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9b3c0a13d0 0x7f9b3c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.430+0000 7f9b4dffb700 1 -- 172.21.15.39:0/2641945367 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9b380ccbe0 con 0x7f9b5006d290 2024-05-24T11:43:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.430+0000 7f9b54aa1700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9b3c0a13d0 0x7f9b3c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.430+0000 7f9b626bfb80 1 -- start start 2024-05-24T11:43:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.430+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.430+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 0x562f6d0e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.430+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0e18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.430+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6d0380f0 con 0x562f6d0e6950 2024-05-24T11:43:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.430+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6d035ab0 con 0x562f6d0e7270 2024-05-24T11:43:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.430+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6d0313a0 con 0x562f6d0e1480 2024-05-24T11:43:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9b3c0a13d0 0x7f9b3c0a3890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800b2e0 tx=0x7f9b48005fd0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0e18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0e18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47408/0 (socket says 172.21.15.39:47408) 2024-05-24T11:43:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 -- 172.21.15.39:0/2222926107 learned_addr learned my addr 172.21.15.39:0/2222926107 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b477fe700 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 0x562f6d0e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 -- 172.21.15.39:0/2222926107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 msgr2=0x562f6d0e0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 0x562f6d0e0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 -- 172.21.15.39:0/2222926107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 msgr2=0x562f6d0e6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0e6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 -- 172.21.15.39:0/2222926107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f6ceb2350 con 0x562f6d0e1480 2024-05-24T11:43:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:43:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.431+0000 7f9b54aa1700 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0e18a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9b48005780 tx=0x7f9b4801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc24268fa23247ed server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.432+0000 7f9b467fc700 1 -- 172.21.15.39:0/2222926107 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4802b070 con 0x562f6d0e1480 2024-05-24T11:43:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.432+0000 7f9b467fc700 1 -- 172.21.15.39:0/2222926107 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b4801e040 con 0x562f6d0e1480 2024-05-24T11:43:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.432+0000 7f9b467fc700 1 -- 172.21.15.39:0/2222926107 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48004d80 con 0x562f6d0e1480 2024-05-24T11:43:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.433+0000 7f9b626bfb80 1 -- 172.21.15.39:0/2222926107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 msgr2=0x562f6d0e18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.433+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0e18a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9b48005780 tx=0x7f9b4801f000 comp rx=0 tx=0).stop 2024-05-24T11:43:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.435+0000 7f9b626bfb80 1 -- 172.21.15.39:0/2222926107 shutdown_connections 2024-05-24T11:43:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.435+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0e18a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.435+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.435+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/2222926107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 0x562f6d0e0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.435+0000 7f9b626bfb80 1 -- 172.21.15.39:0/2222926107 >> 172.21.15.39:0/2222926107 conn(0x562f6d03f1b0 msgr2=0x562f6d0db840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.435+0000 7f9b626bfb80 1 -- 172.21.15.39:0/2222926107 shutdown_connections 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.435+0000 7f9b626bfb80 1 -- 172.21.15.39:0/2222926107 wait complete. 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.435+0000 7f9b626bfb80 1 -- start start 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b626bfb80 1 Processor -- start 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0ed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0ea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 0x562f6d0eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6cfcd500 con 0x562f6d0e6950 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6cfcd1a0 con 0x562f6d0e7270 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6ceb4640 con 0x562f6d0e1480 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b626bfb80 10 client.0 _ll_register_callbacks cb 0x562f6cfd6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b477fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0ea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b477fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0ea4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46850/0 (socket says 172.21.15.39:46850) 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b477fe700 1 -- 172.21.15.39:0/1539770252 learned_addr learned my addr 172.21.15.39:0/1539770252 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.436+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0ed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b54aa1700 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 0x562f6d0eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b477fe700 1 -- 172.21.15.39:0/1539770252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 msgr2=0x562f6d0ed1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b477fe700 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0ed1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b477fe700 1 -- 172.21.15.39:0/1539770252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 msgr2=0x562f6d0eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b477fe700 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 0x562f6d0eab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b477fe700 1 -- 172.21.15.39:0/1539770252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b40009050 con 0x562f6d0e6950 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b477fe700 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0ea4b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9b3809abf0 tx=0x7f9b3809b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac94cc6e07bbc4e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/1539770252 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3801f500 con 0x562f6d0e6950 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/1539770252 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9b3809ea90 con 0x562f6d0e6950 2024-05-24T11:43:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.437+0000 7f9b626bfb80 1 -- 172.21.15.39:0/1539770252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f6cf8a210 con 0x562f6d0e6950 2024-05-24T11:43:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.438+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/1539770252 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b380a30e0 con 0x562f6d0e6950 2024-05-24T11:43:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.438+0000 7f9b626bfb80 10 client.15801 fetch_fsmap learned FSMap version 27 2024-05-24T11:43:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.438+0000 7f9b626bfb80 1 -- 172.21.15.39:0/1539770252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9b5011d3e0 con 0x562f6d0e6950 2024-05-24T11:43:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.438+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/1539770252 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562f6cf8a210 con 0x562f6d0e6950 2024-05-24T11:43:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.439+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/1539770252 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9b380a2c50 con 0x562f6d0e6950 2024-05-24T11:43:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.439+0000 7f9b626bfb80 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.439+0000 7f9b626bfb80 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.439+0000 7f9b626bfb80 20 client.15801 populate_metadata read hostname 'smithi039' 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.439+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562f6d10baf0 0x562f6d10df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.439+0000 7f9b626bfb80 4 client.15801 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.439+0000 7f9b626bfb80 1 -- 172.21.15.39:0/1539770252 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f6d0b07b0 con 0x562f6d10baf0 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.441+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562f6d10baf0 0x562f6d10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.441+0000 7f9b2f7fe700 10 client.15801 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.441+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562f6d10baf0 0x562f6d10df10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912242347bca6117 server_cookie=8144eecda422800b in_seq=0 out_seq=0 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/1539770252 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f6d0b07b0 con 0x562f6d10baf0 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b2f7fe700 10 client.15801 handle_command_reply: tid=0 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 client.15801 shutdown 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 -- 172.21.15.39:0/1539770252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 msgr2=0x562f6d0ea4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0ea4b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9b3809abf0 tx=0x7f9b3809b460 comp rx=0 tx=0).stop 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 -- 172.21.15.39:0/1539770252 shutdown_connections 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e1480 0x562f6d0ed1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562f6d10baf0 0x562f6d10df10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e7270 0x562f6d0eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b2f7fe700 0 client.15801 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/1539770252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e6950 0x562f6d0ea4b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 -- 172.21.15.39:0/1539770252 >> 172.21.15.39:0/1539770252 conn(0x562f6d03f1b0 msgr2=0x562f6d0e54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.442+0000 7f9b626bfb80 1 -- 172.21.15.39:0/1539770252 shutdown_connections 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.443+0000 7f9b626bfb80 1 -- 172.21.15.39:0/1539770252 wait complete. 2024-05-24T11:43:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.443+0000 7f9b626bfb80 20 client.15801 trim_cache size 0 max 16384 2024-05-24T11:43:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.465+0000 7f9b626bfb80 1 Processor -- start 2024-05-24T11:43:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.465+0000 7f9b626bfb80 1 -- start start 2024-05-24T11:43:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 0x562f6d0eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0db910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0ea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6d0dd320 con 0x562f6d0e7270 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6d0ed810 con 0x562f6d0e1480 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6d0ea280 con 0x562f6d0e6950 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b54aa1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b477fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0db910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b54aa1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0ea470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46854/0 (socket says 172.21.15.39:46854) 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b54aa1700 1 -- 172.21.15.39:0/3416133013 learned_addr learned my addr 172.21.15.39:0/3416133013 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/3416133013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 0x562f6d0eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b477fe700 1 -- 172.21.15.39:0/3416133013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 msgr2=0x562f6d0eaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b477fe700 1 --2- 172.21.15.39:0/3416133013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 0x562f6d0eaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b477fe700 1 -- 172.21.15.39:0/3416133013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 msgr2=0x562f6d0ea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b477fe700 1 --2- 172.21.15.39:0/3416133013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0ea470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b477fe700 1 -- 172.21.15.39:0/3416133013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f6ce5eb00 con 0x562f6d0e6950 2024-05-24T11:43:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.466+0000 7f9b54aa1700 1 --2- 172.21.15.39:0/3416133013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:43:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.467+0000 7f9b477fe700 1 --2- 172.21.15.39:0/3416133013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0db910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9b380976d0 tx=0x7f9b3809aa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=312b197cc4810ade server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.467+0000 7f9b46ffd700 1 -- 172.21.15.39:0/3416133013 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38018e20 con 0x562f6d0e6950 2024-05-24T11:43:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.467+0000 7f9b46ffd700 1 -- 172.21.15.39:0/3416133013 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b3809ea90 con 0x562f6d0e6950 2024-05-24T11:43:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.467+0000 7f9b46ffd700 1 -- 172.21.15.39:0/3416133013 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3801f560 con 0x562f6d0e6950 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3416133013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 msgr2=0x562f6d0db910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3416133013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0db910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9b380976d0 tx=0x7f9b3809aa30 comp rx=0 tx=0).stop 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3416133013 shutdown_connections 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3416133013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3416133013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 0x562f6d0eaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3416133013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0db910 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3416133013 >> 172.21.15.39:0/3416133013 conn(0x562f6d03f1b0 msgr2=0x562f6d112830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3416133013 shutdown_connections 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3416133013 wait complete. 2024-05-24T11:43:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.468+0000 7f9b626bfb80 1 -- start start 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b626bfb80 1 Processor -- start 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 0x562f6d0efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0f0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b626bfb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0f0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6d160fb0 con 0x562f6d0e7270 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6d1610f0 con 0x562f6d0e1480 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b626bfb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6d161230 con 0x562f6d0e6950 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b626bfb80 10 client.0 _ll_register_callbacks cb 0x562f6cfd6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b477fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b477fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0f0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47438/0 (socket says 172.21.15.39:47438) 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.469+0000 7f9b477fe700 1 -- 172.21.15.39:0/3039382358 learned_addr learned my addr 172.21.15.39:0/3039382358 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 0x562f6d0efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b54aa1700 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b477fe700 1 -- 172.21.15.39:0/3039382358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 msgr2=0x562f6d0efbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b477fe700 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 0x562f6d0efbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b477fe700 1 -- 172.21.15.39:0/3039382358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 msgr2=0x562f6d0f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b477fe700 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b477fe700 1 -- 172.21.15.39:0/3039382358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f6ceb2350 con 0x562f6d0e6950 2024-05-24T11:43:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b54aa1700 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:43:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b477fe700 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0f0100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b38097160 tx=0x7f9b3809a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fbfa912a0edb5d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.470+0000 7f9b2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:43:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.472+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/3039382358 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3801f510 con 0x562f6d0e6950 2024-05-24T11:43:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.472+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/3039382358 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f9b3809aec0 con 0x562f6d0e6950 2024-05-24T11:43:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.472+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3039382358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b4801e040 con 0x562f6d0e6950 2024-05-24T11:43:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.472+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/3039382358 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b3809b080 con 0x562f6d0e6950 2024-05-24T11:43:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b626bfb80 10 client.25402 fetch_fsmap learned FSMap version 27 2024-05-24T11:43:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3039382358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9b5011d3e0 con 0x562f6d0e6950 2024-05-24T11:43:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/3039382358 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b4801e040 con 0x562f6d0e6950 2024-05-24T11:43:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/3039382358 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9b3809f9b0 con 0x562f6d0e6950 2024-05-24T11:43:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b626bfb80 10 client.25402 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:43:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b626bfb80 10 client.25402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:43:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b626bfb80 20 client.25402 populate_metadata read hostname 'smithi039' 2024-05-24T11:43:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562f6d1624e0 0x562f6d164900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b626bfb80 4 client.25402 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:43:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3039382358 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562f6d029e60 con 0x562f6d1624e0 2024-05-24T11:43:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.473+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562f6d1624e0 0x562f6d164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.474+0000 7f9b4ffff700 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562f6d1624e0 0x562f6d164900 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c3eafe7be22efb0 server_cookie=5cf70d60cb43e6ff in_seq=0 out_seq=0 2024-05-24T11:43:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.474+0000 7f9b2f7fe700 10 client.25402 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b2f7fe700 1 -- 172.21.15.39:0/3039382358 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562f6d029e60 con 0x562f6d1624e0 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b2f7fe700 10 client.25402 handle_command_reply: tid=0 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 client.25402 shutdown 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3039382358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 msgr2=0x562f6d0f0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0f0100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b38097160 tx=0x7f9b3809a380 comp rx=0 tx=0).stop 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3039382358 shutdown_connections 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562f6d1624e0 0x562f6d164900 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6d0e7270 0x562f6d0f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562f6d0e1480 0x562f6d0efbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b2f7fe700 0 client.25402 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 --2- 172.21.15.39:0/3039382358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f6d0e6950 0x562f6d0f0100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3039382358 >> 172.21.15.39:0/3039382358 conn(0x562f6d03f1b0 msgr2=0x562f6d0df510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.497+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3039382358 shutdown_connections 2024-05-24T11:43:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.498+0000 7f9b626bfb80 1 -- 172.21.15.39:0/3039382358 wait complete. 2024-05-24T11:43:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.498+0000 7f9b626bfb80 20 client.25402 trim_cache size 0 max 16384 2024-05-24T11:43:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:43:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:43:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:43:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:43:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 -- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9b3c0a13d0 msgr2=0x7f9b3c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9b3c0a13d0 0x7f9b3c0a3890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800b2e0 tx=0x7f9b48005fd0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 -- 172.21.15.39:0/2641945367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5006d290 msgr2=0x7f9b501a6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5006d290 0x7f9b501a6540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c9b0 tx=0x7f9b3800cd70 comp rx=0 tx=0).stop 2024-05-24T11:43:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 -- 172.21.15.39:0/2641945367 shutdown_connections 2024-05-24T11:43:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9b3c0a13d0 0x7f9b3c0a3890 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5011a640 0x7f9b501aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b5006c970 0x7f9b501a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 --2- 172.21.15.39:0/2641945367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b5006d290 0x7f9b501a6540 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 -- 172.21.15.39:0/2641945367 >> 172.21.15.39:0/2641945367 conn(0x7f9b5006c020 msgr2=0x7f9b50109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 -- 172.21.15.39:0/2641945367 shutdown_connections 2024-05-24T11:43:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:23.500+0000 7f9b56504700 1 -- 172.21.15.39:0/2641945367 wait complete. 2024-05-24T11:43:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 240.0 s, 1294.6 tps, lat 77.130 ms stddev 25.909 2024-05-24T11:43:24.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:23 smithi053 ceph-mon[47448]: pgmap v158: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:43:24.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:23 smithi137 ceph-mon[47907]: pgmap v158: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:43:24.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:23 smithi039 ceph-mon[42919]: pgmap v158: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:43:26.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:25 smithi053 ceph-mon[47448]: pgmap v159: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:43:26.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:25 smithi137 ceph-mon[47907]: pgmap v159: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:43:26.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:25 smithi039 ceph-mon[42919]: pgmap v159: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:43:28.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:27 smithi053 ceph-mon[47448]: pgmap v160: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:43:28.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:27 smithi137 ceph-mon[47907]: pgmap v160: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:43:28.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:27 smithi039 ceph-mon[42919]: pgmap v160: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:43:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 245.0 s, 1312.8 tps, lat 76.116 ms stddev 20.193 2024-05-24T11:43:30.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:29 smithi053 ceph-mon[47448]: pgmap v161: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:43:30.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:29 smithi137 ceph-mon[47907]: pgmap v161: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:43:30.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:29 smithi039 ceph-mon[42919]: pgmap v161: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:43:32.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:31 smithi053 ceph-mon[47448]: pgmap v162: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:43:32.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:31 smithi137 ceph-mon[47907]: pgmap v162: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:43:32.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:31 smithi039 ceph-mon[42919]: pgmap v162: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:43:34.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 250.0 s, 1365.2 tps, lat 73.328 ms stddev 20.858 2024-05-24T11:43:34.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:33 smithi137 ceph-mon[47907]: pgmap v163: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:43:34.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:33 smithi039 ceph-mon[42919]: pgmap v163: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:43:34.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:33 smithi053 ceph-mon[47448]: pgmap v163: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:43:36.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:35 smithi137 ceph-mon[47907]: pgmap v164: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-05-24T11:43:36.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:35 smithi039 ceph-mon[42919]: pgmap v164: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-05-24T11:43:36.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:35 smithi053 ceph-mon[47448]: pgmap v164: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-05-24T11:43:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.681+0000 7efed9827700 1 -- 172.21.15.39:0/1204296781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 msgr2=0x7efed41152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.681+0000 7efed9827700 1 --2- 172.21.15.39:0/1204296781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 0x7efed41152e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efec4009990 tx=0x7efec4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.681+0000 7efed9827700 1 -- 172.21.15.39:0/1204296781 shutdown_connections 2024-05-24T11:43:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.681+0000 7efed9827700 1 --2- 172.21.15.39:0/1204296781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 0x7efed41152e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.681+0000 7efed9827700 1 --2- 172.21.15.39:0/1204296781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed4109940 0x7efed4109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.681+0000 7efed9827700 1 --2- 172.21.15.39:0/1204296781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efed410a260 0x7efed41129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.681+0000 7efed9827700 1 -- 172.21.15.39:0/1204296781 >> 172.21.15.39:0/1204296781 conn(0x7efed406bf80 msgr2=0x7efed406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.681+0000 7efed9827700 1 -- 172.21.15.39:0/1204296781 shutdown_connections 2024-05-24T11:43:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.681+0000 7efed9827700 1 -- 172.21.15.39:0/1204296781 wait complete. 2024-05-24T11:43:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.682+0000 7efed9827700 1 Processor -- start 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.682+0000 7efed9827700 1 -- start start 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.682+0000 7efed9827700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed4109940 0x7efed41a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.682+0000 7efed9827700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efed410a260 0x7efed41a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.682+0000 7efed9827700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 0x7efed419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.682+0000 7efed9827700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419dd60 con 0x7efed4112ef0 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.682+0000 7efed9827700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419ded0 con 0x7efed4109940 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.682+0000 7efed9827700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419e090 con 0x7efed410a260 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 0x7efed419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 0x7efed419d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42292/0 (socket says 172.21.15.39:42292) 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed37fe700 1 -- 172.21.15.39:0/1486733932 learned_addr learned my addr 172.21.15.39:0/1486733932 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed27fc700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efed410a260 0x7efed41a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed37fe700 1 -- 172.21.15.39:0/1486733932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efed410a260 msgr2=0x7efed41a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed37fe700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efed410a260 0x7efed41a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed37fe700 1 -- 172.21.15.39:0/1486733932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed4109940 msgr2=0x7efed41a3580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-24T11:43:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed37fe700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed4109940 0x7efed41a3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed37fe700 1 -- 172.21.15.39:0/1486733932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec40095f0 con 0x7efed4112ef0 2024-05-24T11:43:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.683+0000 7efed27fc700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efed410a260 0x7efed41a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:43:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.684+0000 7efed37fe700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 0x7efed419d700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efec4004760 tx=0x7efec4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.684+0000 7efed8825700 1 -- 172.21.15.39:0/1486733932 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec401d070 con 0x7efed4112ef0 2024-05-24T11:43:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.684+0000 7efed8825700 1 -- 172.21.15.39:0/1486733932 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efec400be50 con 0x7efed4112ef0 2024-05-24T11:43:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.684+0000 7efed8825700 1 -- 172.21.15.39:0/1486733932 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4022e40 con 0x7efed4112ef0 2024-05-24T11:43:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.688+0000 7efed9827700 1 -- 172.21.15.39:0/1486733932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed419e310 con 0x7efed4112ef0 2024-05-24T11:43:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.688+0000 7efed9827700 1 -- 172.21.15.39:0/1486733932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed41aa390 con 0x7efed4112ef0 2024-05-24T11:43:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.693+0000 7efee59e2b80 1 Processor -- start 2024-05-24T11:43:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.693+0000 7efed8825700 1 -- 172.21.15.39:0/1486733932 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efec400fa90 con 0x7efed4112ef0 2024-05-24T11:43:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.693+0000 7efee59e2b80 1 -- start start 2024-05-24T11:43:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.693+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c569540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.693+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c563750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.693+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 0x55b14c5640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.693+0000 7efee59e2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c4b20f0 con 0x55b14c563c90 2024-05-24T11:43:36.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.693+0000 7efee59e2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c4afab0 con 0x55b14c569a80 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.693+0000 7efee59e2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c4ab3a0 con 0x55b14c569160 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c563750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c563750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:48822/0 (socket says 172.21.15.39:48822) 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed2ffd700 1 -- 172.21.15.39:0/120826343 learned_addr learned my addr 172.21.15.39:0/120826343 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed27fc700 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c569540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed27fc700 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 0x55b14c5640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed2ffd700 1 -- 172.21.15.39:0/120826343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 msgr2=0x55b14c569540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed2ffd700 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c569540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed2ffd700 1 -- 172.21.15.39:0/120826343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 msgr2=0x55b14c5640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed2ffd700 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 0x55b14c5640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed2ffd700 1 -- 172.21.15.39:0/120826343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b14c32c350 con 0x55b14c569a80 2024-05-24T11:43:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed27fc700 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c569540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:43:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efed2ffd700 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c563750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efecc007730 tx=0x7efecc0094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0ca93748406a3d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.694+0000 7efec9ffb700 1 -- 172.21.15.39:0/120826343 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc010040 con 0x55b14c569a80 2024-05-24T11:43:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.695+0000 7efec9ffb700 1 -- 172.21.15.39:0/120826343 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efecc00b5a0 con 0x55b14c569a80 2024-05-24T11:43:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.695+0000 7efec9ffb700 1 -- 172.21.15.39:0/120826343 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc004230 con 0x55b14c569a80 2024-05-24T11:43:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.695+0000 7efed8825700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efec00a1310 0x7efec00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.695+0000 7efed8825700 1 -- 172.21.15.39:0/1486733932 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7efec40ce210 con 0x7efed4112ef0 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.695+0000 7efee59e2b80 1 -- 172.21.15.39:0/120826343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 msgr2=0x55b14c563750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.695+0000 7efee59e2b80 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c563750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efecc007730 tx=0x7efecc0094c0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.696+0000 7efed27fc700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efec00a1310 0x7efec00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.697+0000 7efee59e2b80 1 -- 172.21.15.39:0/120826343 shutdown_connections 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.697+0000 7efee59e2b80 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c569540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.697+0000 7efee59e2b80 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 0x55b14c5640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.697+0000 7efee59e2b80 1 --2- 172.21.15.39:0/120826343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c563750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.697+0000 7efee59e2b80 1 -- 172.21.15.39:0/120826343 >> 172.21.15.39:0/120826343 conn(0x55b14c4b91b0 msgr2=0x55b14c555840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.697+0000 7efee59e2b80 1 -- 172.21.15.39:0/120826343 shutdown_connections 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.697+0000 7efee59e2b80 1 -- 172.21.15.39:0/120826343 wait complete. 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.697+0000 7efee59e2b80 1 -- start start 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.697+0000 7efed27fc700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efec00a1310 0x7efec00a37d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efebc00ffd0 tx=0x7efebc009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efee59e2b80 1 Processor -- start 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c563c90 0x55b14c55d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c55d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569a80 0x55b14c563020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efee59e2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c447500 con 0x55b14c569a80 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efee59e2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c4471a0 con 0x55b14c563c90 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efee59e2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c32e640 con 0x55b14c569160 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efee59e2b80 10 client.0 _ll_register_callbacks cb 0x55b14c450ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:43:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efed27fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c55d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efed27fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c55d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38254/0 (socket says 172.21.15.39:38254) 2024-05-24T11:43:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.698+0000 7efed27fc700 1 -- 172.21.15.39:0/544791029 learned_addr learned my addr 172.21.15.39:0/544791029 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efed37fe700 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569a80 0x55b14c563020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efed27fc700 1 -- 172.21.15.39:0/544791029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c563c90 msgr2=0x55b14c55d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efed27fc700 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c563c90 0x55b14c55d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efed27fc700 1 -- 172.21.15.39:0/544791029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569a80 msgr2=0x55b14c563020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efed27fc700 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569a80 0x55b14c563020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efed27fc700 1 -- 172.21.15.39:0/544791029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc009400 con 0x55b14c569160 2024-05-24T11:43:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efed37fe700 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569a80 0x55b14c563020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:43:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efed27fc700 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c55d7f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efebc009ae0 tx=0x7efebc011040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e578fb8dbdb5dceb server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efeb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:43:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efeb2ffd700 1 -- 172.21.15.39:0/544791029 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc025070 con 0x55b14c569160 2024-05-24T11:43:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efee59e2b80 1 -- 172.21.15.39:0/544791029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b14c404210 con 0x55b14c569160 2024-05-24T11:43:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.699+0000 7efeb2ffd700 1 -- 172.21.15.39:0/544791029 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7efebc029410 con 0x55b14c569160 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.700+0000 7efeb2ffd700 1 -- 172.21.15.39:0/544791029 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efebc0295d0 con 0x55b14c569160 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.700+0000 7efee59e2b80 10 client.25414 fetch_fsmap learned FSMap version 27 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.700+0000 7efee59e2b80 1 -- 172.21.15.39:0/544791029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7efecc0073e0 con 0x55b14c569160 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.700+0000 7efeb2ffd700 1 -- 172.21.15.39:0/544791029 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efebc037020 con 0x55b14c569160 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.700+0000 7efeb2ffd700 1 -- 172.21.15.39:0/544791029 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efebc01e4a0 con 0x55b14c569160 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.700+0000 7efee59e2b80 10 client.25414 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.700+0000 7efee59e2b80 10 client.25414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.700+0000 7efee59e2b80 20 client.25414 populate_metadata read hostname 'smithi039' 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.700+0000 7efee59e2b80 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b14c585c90 0x55b14c5880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.701+0000 7efee59e2b80 4 client.25414 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:43:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.701+0000 7efee59e2b80 1 -- 172.21.15.39:0/544791029 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b14c52a7b0 con 0x55b14c585c90 2024-05-24T11:43:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.701+0000 7efed2ffd700 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b14c585c90 0x55b14c5880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.702+0000 7efed2ffd700 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b14c585c90 0x55b14c5880b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06aae2395f64981 server_cookie=c26c8dae3fe45637 in_seq=0 out_seq=0 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.703+0000 7efeb2ffd700 10 client.25414 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.703+0000 7efeb2ffd700 1 -- 172.21.15.39:0/544791029 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b14c52a7b0 con 0x55b14c585c90 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.703+0000 7efeb2ffd700 10 client.25414 handle_command_reply: tid=0 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.703+0000 7efee59e2b80 1 client.25414 shutdown 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.703+0000 7efee59e2b80 1 -- 172.21.15.39:0/544791029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 msgr2=0x55b14c55d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.703+0000 7efee59e2b80 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c55d7f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efebc009ae0 tx=0x7efebc011040 comp rx=0 tx=0).stop 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efee59e2b80 1 -- 172.21.15.39:0/544791029 shutdown_connections 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efee59e2b80 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b14c585c90 0x55b14c5880b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efee59e2b80 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569a80 0x55b14c563020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efee59e2b80 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c563c90 0x55b14c55d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efee59e2b80 1 --2- 172.21.15.39:0/544791029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569160 0x55b14c55d7f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efee59e2b80 1 -- 172.21.15.39:0/544791029 >> 172.21.15.39:0/544791029 conn(0x55b14c4b91b0 msgr2=0x55b14c567cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efeb2ffd700 0 client.25414 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efee59e2b80 1 -- 172.21.15.39:0/544791029 shutdown_connections 2024-05-24T11:43:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efee59e2b80 1 -- 172.21.15.39:0/544791029 wait complete. 2024-05-24T11:43:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.704+0000 7efee59e2b80 20 client.25414 trim_cache size 0 max 16384 2024-05-24T11:43:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.728+0000 7efee59e2b80 1 Processor -- start 2024-05-24T11:43:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.728+0000 7efee59e2b80 1 -- start start 2024-05-24T11:43:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.728+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 0x55b14c562f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.728+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569160 0x55b14c56b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.728+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569a80 0x55b14c56ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.728+0000 7efee59e2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c559c60 con 0x55b14c563c90 2024-05-24T11:43:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.728+0000 7efee59e2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c556530 con 0x55b14c569160 2024-05-24T11:43:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.728+0000 7efee59e2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c563af0 con 0x55b14c569a80 2024-05-24T11:43:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569a80 0x55b14c56ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569a80 0x55b14c56ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38266/0 (socket says 172.21.15.39:38266) 2024-05-24T11:43:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed37fe700 1 -- 172.21.15.39:0/1470704493 learned_addr learned my addr 172.21.15.39:0/1470704493 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed27fc700 1 --2- 172.21.15.39:0/1470704493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569160 0x55b14c56b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed2ffd700 1 --2- 172.21.15.39:0/1470704493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 0x55b14c562f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed37fe700 1 -- 172.21.15.39:0/1470704493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569160 msgr2=0x55b14c56b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed37fe700 1 --2- 172.21.15.39:0/1470704493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569160 0x55b14c56b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed37fe700 1 -- 172.21.15.39:0/1470704493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 msgr2=0x55b14c562f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed37fe700 1 --2- 172.21.15.39:0/1470704493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 0x55b14c562f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.729+0000 7efed37fe700 1 -- 172.21.15.39:0/1470704493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b14c2d8b00 con 0x55b14c569a80 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.730+0000 7efed37fe700 1 --2- 172.21.15.39:0/1470704493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569a80 0x55b14c56ba10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efec40cba00 tx=0x7efec409b1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=446bdb8378830b4b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.730+0000 7efeca7fc700 1 -- 172.21.15.39:0/1470704493 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec409ad50 con 0x55b14c569a80 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.730+0000 7efeca7fc700 1 -- 172.21.15.39:0/1470704493 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efec40a1070 con 0x55b14c569a80 2024-05-24T11:43:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.730+0000 7efeca7fc700 1 -- 172.21.15.39:0/1470704493 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec40a37e0 con 0x55b14c569a80 2024-05-24T11:43:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.731+0000 7efee59e2b80 1 -- 172.21.15.39:0/1470704493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569a80 msgr2=0x55b14c56ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.731+0000 7efee59e2b80 1 --2- 172.21.15.39:0/1470704493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569a80 0x55b14c56ba10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efec40cba00 tx=0x7efec409b1c0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.731+0000 7efee59e2b80 1 -- 172.21.15.39:0/1470704493 shutdown_connections 2024-05-24T11:43:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.731+0000 7efee59e2b80 1 --2- 172.21.15.39:0/1470704493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c569a80 0x55b14c56ba10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.731+0000 7efee59e2b80 1 --2- 172.21.15.39:0/1470704493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c563c90 0x55b14c562f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.731+0000 7efee59e2b80 1 --2- 172.21.15.39:0/1470704493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569160 0x55b14c56b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.731+0000 7efee59e2b80 1 -- 172.21.15.39:0/1470704493 >> 172.21.15.39:0/1470704493 conn(0x55b14c4b91b0 msgr2=0x55b14c58c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.731+0000 7efee59e2b80 1 -- 172.21.15.39:0/1470704493 shutdown_connections 2024-05-24T11:43:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.731+0000 7efee59e2b80 1 -- 172.21.15.39:0/1470704493 wait complete. 2024-05-24T11:43:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.732+0000 7efee59e2b80 1 -- start start 2024-05-24T11:43:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.732+0000 7efee59e2b80 1 Processor -- start 2024-05-24T11:43:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.732+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c563c90 0x55b14c562c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.732+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569160 0x55b14c55e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.732+0000 7efee59e2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c55e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.732+0000 7efee59e2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c55eb40 con 0x55b14c569160 2024-05-24T11:43:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.732+0000 7efee59e2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c55ecb0 con 0x55b14c569a80 2024-05-24T11:43:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.732+0000 7efee59e2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14c55ee70 con 0x55b14c563c90 2024-05-24T11:43:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.732+0000 7efee59e2b80 10 client.0 _ll_register_callbacks cb 0x55b14c450ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:43:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.733+0000 7efed2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c563c90 0x55b14c562c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.733+0000 7efed2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c563c90 0x55b14c562c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38274/0 (socket says 172.21.15.39:38274) 2024-05-24T11:43:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.733+0000 7efed2ffd700 1 -- 172.21.15.39:0/43233203 learned_addr learned my addr 172.21.15.39:0/43233203 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.733+0000 7efed27fc700 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569160 0x55b14c55e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.733+0000 7efed37fe700 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c55e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efed37fe700 1 -- 172.21.15.39:0/43233203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c563c90 msgr2=0x55b14c562c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efed37fe700 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c563c90 0x55b14c562c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efed37fe700 1 -- 172.21.15.39:0/43233203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569160 msgr2=0x55b14c55e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efed37fe700 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569160 0x55b14c55e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efed37fe700 1 -- 172.21.15.39:0/43233203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed41aa390 con 0x55b14c569a80 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efed37fe700 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c55e570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55b14c5566a0 tx=0x7efec409f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3d90fb68fcbb3ec server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efeb2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efeb2ffd700 1 -- 172.21.15.39:0/43233203 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4021eb0 con 0x55b14c569a80 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efeb2ffd700 1 -- 172.21.15.39:0/43233203 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7efec40ce030 con 0x55b14c569a80 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.734+0000 7efee59e2b80 1 -- 172.21.15.39:0/43233203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efecc00b5a0 con 0x55b14c569a80 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efeb2ffd700 1 -- 172.21.15.39:0/43233203 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efec409ad50 con 0x55b14c569a80 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efee59e2b80 10 client.25514 fetch_fsmap learned FSMap version 27 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efee59e2b80 1 -- 172.21.15.39:0/43233203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b14c471500 con 0x55b14c569a80 2024-05-24T11:43:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efeb2ffd700 1 -- 172.21.15.39:0/43233203 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efec407e070 con 0x55b14c569a80 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efeb2ffd700 1 -- 172.21.15.39:0/43233203 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efec409a8e0 con 0x55b14c569a80 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efee59e2b80 10 client.25514 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efee59e2b80 10 client.25514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efee59e2b80 20 client.25514 populate_metadata read hostname 'smithi039' 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efee59e2b80 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b14c5dc2b0 0x55b14c5de6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efee59e2b80 4 client.25514 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.735+0000 7efee59e2b80 1 -- 172.21.15.39:0/43233203 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b14c4a3e60 con 0x55b14c5dc2b0 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.736+0000 7efed2ffd700 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b14c5dc2b0 0x55b14c5de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.736+0000 7efeb2ffd700 10 client.25514 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.736+0000 7efed2ffd700 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b14c5dc2b0 0x55b14c5de6d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1b1d7eabf25665 server_cookie=bb15c8baba068426 in_seq=0 out_seq=0 2024-05-24T11:43:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.774+0000 7efeb2ffd700 1 -- 172.21.15.39:0/43233203 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b14c4a3e60 con 0x55b14c5dc2b0 2024-05-24T11:43:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.774+0000 7efeb2ffd700 10 client.25514 handle_command_reply: tid=0 2024-05-24T11:43:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.774+0000 7efee59e2b80 1 client.25514 shutdown 2024-05-24T11:43:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 -- 172.21.15.39:0/43233203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 msgr2=0x55b14c55e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c55e570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55b14c5566a0 tx=0x7efec409f130 comp rx=0 tx=0).stop 2024-05-24T11:43:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 -- 172.21.15.39:0/43233203 shutdown_connections 2024-05-24T11:43:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b14c563c90 0x55b14c562c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b14c5dc2b0 0x55b14c5de6d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b14c569a80 0x55b14c55e570 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 --2- 172.21.15.39:0/43233203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14c569160 0x55b14c55e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 -- 172.21.15.39:0/43233203 >> 172.21.15.39:0/43233203 conn(0x55b14c4b91b0 msgr2=0x55b14c56b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efeb2ffd700 0 client.25514 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 -- 172.21.15.39:0/43233203 shutdown_connections 2024-05-24T11:43:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.775+0000 7efee59e2b80 1 -- 172.21.15.39:0/43233203 wait complete. 2024-05-24T11:43:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efee59e2b80 20 client.25514 trim_cache size 0 max 16384 2024-05-24T11:43:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:43:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:43:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:43:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:43:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 -- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efec00a1310 msgr2=0x7efec00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efec00a1310 0x7efec00a37d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efebc00ffd0 tx=0x7efebc009260 comp rx=0 tx=0).stop 2024-05-24T11:43:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 -- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 msgr2=0x7efed419d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 0x7efed419d700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efec4004760 tx=0x7efec4004790 comp rx=0 tx=0).stop 2024-05-24T11:43:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 -- 172.21.15.39:0/1486733932 shutdown_connections 2024-05-24T11:43:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efec00a1310 0x7efec00a37d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4112ef0 0x7efed419d700 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed4109940 0x7efed41a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 --2- 172.21.15.39:0/1486733932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efed410a260 0x7efed41a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.776+0000 7efed9827700 1 -- 172.21.15.39:0/1486733932 >> 172.21.15.39:0/1486733932 conn(0x7efed406bf80 msgr2=0x7efed406fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.777+0000 7efed9827700 1 -- 172.21.15.39:0/1486733932 shutdown_connections 2024-05-24T11:43:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:36.777+0000 7efed9827700 1 -- 172.21.15.39:0/1486733932 wait complete. 2024-05-24T11:43:38.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:37 smithi039 ceph-mon[42919]: pgmap v165: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-05-24T11:43:38.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:37 smithi053 ceph-mon[47448]: pgmap v165: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-05-24T11:43:38.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:37 smithi137 ceph-mon[47907]: pgmap v165: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-05-24T11:43:39.118 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 255.0 s, 1202.6 tps, lat 83.017 ms stddev 23.934 2024-05-24T11:43:40.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:40 smithi039 ceph-mon[42919]: pgmap v166: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-05-24T11:43:40.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:40 smithi053 ceph-mon[47448]: pgmap v166: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-05-24T11:43:40.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:40 smithi137 ceph-mon[47907]: pgmap v166: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 19 MiB/s wr, 1.74k op/s 2024-05-24T11:43:42.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:42 smithi039 ceph-mon[42919]: pgmap v167: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:43:42.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:42 smithi053 ceph-mon[47448]: pgmap v167: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:43:42.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:42 smithi137 ceph-mon[47907]: pgmap v167: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:43:44.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 260.0 s, 1274.0 tps, lat 78.613 ms stddev 23.375 2024-05-24T11:43:44.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:44 smithi039 ceph-mon[42919]: pgmap v168: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:43:44.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:44 smithi053 ceph-mon[47448]: pgmap v168: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:43:44.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:44 smithi137 ceph-mon[47907]: pgmap v168: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:43:46.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:46 smithi039 ceph-mon[42919]: pgmap v169: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-05-24T11:43:46.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:46 smithi053 ceph-mon[47448]: pgmap v169: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-05-24T11:43:46.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:46 smithi137 ceph-mon[47907]: pgmap v169: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 18 MiB/s wr, 1.70k op/s 2024-05-24T11:43:48.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:48 smithi039 ceph-mon[42919]: pgmap v170: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-24T11:43:48.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:48 smithi053 ceph-mon[47448]: pgmap v170: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-24T11:43:48.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:48 smithi137 ceph-mon[47907]: pgmap v170: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-24T11:43:49.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 265.0 s, 1350.4 tps, lat 74.025 ms stddev 22.925 2024-05-24T11:43:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.956+0000 7f14e69f9700 1 -- 172.21.15.39:0/991004958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00fff20 msgr2=0x7f14e010c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:49.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.956+0000 7f14e69f9700 1 --2- 172.21.15.39:0/991004958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00fff20 0x7f14e010c3e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c240 tx=0x7f14d800c550 comp rx=0 tx=0).stop 2024-05-24T11:43:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.958+0000 7f14e69f9700 1 -- 172.21.15.39:0/991004958 shutdown_connections 2024-05-24T11:43:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.958+0000 7f14e69f9700 1 --2- 172.21.15.39:0/991004958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00fff20 0x7f14e010c3e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.958+0000 7f14e69f9700 1 --2- 172.21.15.39:0/991004958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14e0105620 0x7f14e0105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.958+0000 7f14e69f9700 1 --2- 172.21.15.39:0/991004958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14e00ff560 0x7f14e00ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.958+0000 7f14e69f9700 1 -- 172.21.15.39:0/991004958 >> 172.21.15.39:0/991004958 conn(0x7f14e00773b0 msgr2=0x7f14e00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.959+0000 7f14e69f9700 1 -- 172.21.15.39:0/991004958 shutdown_connections 2024-05-24T11:43:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.960+0000 7f14e69f9700 1 -- 172.21.15.39:0/991004958 wait complete. 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.960+0000 7f14e69f9700 1 Processor -- start 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14e69f9700 1 -- start start 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14e69f9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00ff560 0x7f14e01942f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14e69f9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14e00fff20 0x7f14e0194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14e69f9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14e0105620 0x7f14e0194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14e69f9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e01a9ac0 con 0x7f14e00fff20 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14e69f9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e01a9c00 con 0x7f14e00ff560 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14e69f9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e01a9d70 con 0x7f14e0105620 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14dffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00ff560 0x7f14e01942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14dffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00ff560 0x7f14e01942f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:60358/0 (socket says 172.21.15.39:60358) 2024-05-24T11:43:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.961+0000 7f14dffff700 1 -- 172.21.15.39:0/3835443315 learned_addr learned my addr 172.21.15.39:0/3835443315 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.962+0000 7f14df7fe700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14e00fff20 0x7f14e0194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.962+0000 7f14e4f96700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14e0105620 0x7f14e0194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.962+0000 7f14dffff700 1 -- 172.21.15.39:0/3835443315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14e0105620 msgr2=0x7f14e0194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.962+0000 7f14dffff700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14e0105620 0x7f14e0194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.962+0000 7f14dffff700 1 -- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14e00fff20 msgr2=0x7f14e0194830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.962+0000 7f14dffff700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14e00fff20 0x7f14e0194830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.962+0000 7f14dffff700 1 -- 172.21.15.39:0/3835443315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d800ae10 con 0x7f14e00ff560 2024-05-24T11:43:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.962+0000 7f14dffff700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00ff560 0x7f14e01942f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14d000b530 tx=0x7f14d000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.963+0000 7f14dd7fa700 1 -- 172.21.15.39:0/3835443315 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0011830 con 0x7f14e00ff560 2024-05-24T11:43:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.963+0000 7f14e69f9700 1 -- 172.21.15.39:0/3835443315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e01a9ff0 con 0x7f14e00ff560 2024-05-24T11:43:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.963+0000 7f14e69f9700 1 -- 172.21.15.39:0/3835443315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e01aa520 con 0x7f14e00ff560 2024-05-24T11:43:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.963+0000 7f14dd7fa700 1 -- 172.21.15.39:0/3835443315 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14d0011e20 con 0x7f14e00ff560 2024-05-24T11:43:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.963+0000 7f14dd7fa700 1 -- 172.21.15.39:0/3835443315 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0010db0 con 0x7f14e00ff560 2024-05-24T11:43:49.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.967+0000 7f14f2bb4b80 1 Processor -- start 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14f2bb4b80 1 -- start start 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4567950 0x55b4f4567d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 0x55b4f4561f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4562480 0x55b4f45628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f44b90f0 con 0x55b4f4562480 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f44b6ab0 con 0x55b4f4567950 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f44b23a0 con 0x55b4f4568270 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14e4f96700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4567950 0x55b4f4567d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14e4f96700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4567950 0x55b4f4567d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:60360/0 (socket says 172.21.15.39:60360) 2024-05-24T11:43:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14e4f96700 1 -- 172.21.15.39:0/1289307569 learned_addr learned my addr 172.21.15.39:0/1289307569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14df7fe700 1 --2- 172.21.15.39:0/1289307569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4562480 0x55b4f45628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14e4f96700 1 -- 172.21.15.39:0/1289307569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 msgr2=0x55b4f4561f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:43:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14e4f96700 1 --2- 172.21.15.39:0/1289307569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 0x55b4f4561f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14e4f96700 1 -- 172.21.15.39:0/1289307569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4562480 msgr2=0x55b4f45628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14e4f96700 1 --2- 172.21.15.39:0/1289307569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4562480 0x55b4f45628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.968+0000 7f14e4f96700 1 -- 172.21.15.39:0/1289307569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4f4333350 con 0x55b4f4567950 2024-05-24T11:43:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.969+0000 7f14dd7fa700 1 -- 172.21.15.39:0/3835443315 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f14d001a400 con 0x7f14e00ff560 2024-05-24T11:43:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.970+0000 7f14e4f96700 1 --2- 172.21.15.39:0/1289307569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4567950 0x55b4f4567d30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14d800ad10 tx=0x7f14d800ad40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf0d86bd45df5027 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.970+0000 7f14dd7fa700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f14c80a13d0 0x7f14c80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.970+0000 7f14dd7fa700 1 -- 172.21.15.39:0/3835443315 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f14d00cc420 con 0x7f14e00ff560 2024-05-24T11:43:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.970+0000 7f14dffff700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f14c80a13d0 0x7f14c80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.970+0000 7f14c67fc700 1 -- 172.21.15.39:0/1289307569 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d800f070 con 0x55b4f4567950 2024-05-24T11:43:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.970+0000 7f14c67fc700 1 -- 172.21.15.39:0/1289307569 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14d800a0d0 con 0x55b4f4567950 2024-05-24T11:43:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.970+0000 7f14c67fc700 1 -- 172.21.15.39:0/1289307569 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d80139d0 con 0x55b4f4567950 2024-05-24T11:43:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.971+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/1289307569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4567950 msgr2=0x55b4f4567d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.971+0000 7f14dffff700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f14c80a13d0 0x7f14c80a3890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55b4f455d790 tx=0x7f14d0011f90 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.971+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/1289307569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4567950 0x55b4f4567d30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14d800ad10 tx=0x7f14d800ad40 comp rx=0 tx=0).stop 2024-05-24T11:43:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.972+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/1289307569 shutdown_connections 2024-05-24T11:43:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.972+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/1289307569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4562480 0x55b4f45628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.972+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/1289307569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4567950 0x55b4f4567d30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.972+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/1289307569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 0x55b4f4561f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.972+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/1289307569 >> 172.21.15.39:0/1289307569 conn(0x55b4f44c01b0 msgr2=0x55b4f455c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.972+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/1289307569 shutdown_connections 2024-05-24T11:43:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.972+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/1289307569 wait complete. 2024-05-24T11:43:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.972+0000 7f14f2bb4b80 1 -- start start 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.973+0000 7f14f2bb4b80 1 Processor -- start 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.973+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4562480 0x55b4f456e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.973+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4567950 0x55b4f456b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.973+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 0x55b4f456bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.973+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f444e500 con 0x55b4f4567950 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.973+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f444e1a0 con 0x55b4f4568270 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.973+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f4335640 con 0x55b4f4562480 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.973+0000 7f14f2bb4b80 10 client.0 _ll_register_callbacks cb 0x55b4f4457ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14df7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4567950 0x55b4f456b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14df7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4567950 0x55b4f456b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45500/0 (socket says 172.21.15.39:45500) 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14df7fe700 1 -- 172.21.15.39:0/2529655245 learned_addr learned my addr 172.21.15.39:0/2529655245 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14e4f96700 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4562480 0x55b4f456e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14e4f96700 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 0x55b4f456bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14e4f96700 1 -- 172.21.15.39:0/2529655245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 msgr2=0x55b4f456bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14e4f96700 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 0x55b4f456bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14e4f96700 1 -- 172.21.15.39:0/2529655245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4567950 msgr2=0x55b4f456b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14e4f96700 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4567950 0x55b4f456b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.974+0000 7f14e4f96700 1 -- 172.21.15.39:0/2529655245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e010cce0 con 0x55b4f4562480 2024-05-24T11:43:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.975+0000 7f14e4f96700 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4562480 0x55b4f456e280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14d80061c0 tx=0x7f14d80134b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1895a0cbac8939d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.975+0000 7f14bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.975+0000 7f14bf7fe700 1 -- 172.21.15.39:0/2529655245 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d800f040 con 0x55b4f4562480 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.975+0000 7f14bf7fe700 1 -- 172.21.15.39:0/2529655245 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f14d8008040 con 0x55b4f4562480 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.975+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2529655245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4f440b210 con 0x55b4f4562480 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14bf7fe700 1 -- 172.21.15.39:0/2529655245 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d800a5b0 con 0x55b4f4562480 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14f2bb4b80 10 client.25429 fetch_fsmap learned FSMap version 27 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2529655245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b4f42dfb00 con 0x55b4f4562480 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14bf7fe700 1 -- 172.21.15.39:0/2529655245 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b4f440b210 con 0x55b4f4562480 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14bf7fe700 1 -- 172.21.15.39:0/2529655245 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14d800c270 con 0x55b4f4562480 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14f2bb4b80 10 client.25429 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14f2bb4b80 10 client.25429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:43:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14f2bb4b80 20 client.25429 populate_metadata read hostname 'smithi039' 2024-05-24T11:43:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b4f458c870 0x55b4f458ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14f2bb4b80 4 client.25429 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:43:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.976+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2529655245 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4f45317b0 con 0x55b4f458c870 2024-05-24T11:43:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.977+0000 7f14df7fe700 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b4f458c870 0x55b4f458ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.977+0000 7f14bf7fe700 10 client.25429 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.977+0000 7f14df7fe700 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b4f458c870 0x55b4f458ec90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4ab588f4805078 server_cookie=e63d4a376ab598d9 in_seq=0 out_seq=0 2024-05-24T11:43:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14bf7fe700 1 -- 172.21.15.39:0/2529655245 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4f45317b0 con 0x55b4f458c870 2024-05-24T11:43:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14bf7fe700 10 client.25429 handle_command_reply: tid=0 2024-05-24T11:43:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 client.25429 shutdown 2024-05-24T11:43:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2529655245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4562480 msgr2=0x55b4f456e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4562480 0x55b4f456e280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14d80061c0 tx=0x7f14d80134b0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2529655245 shutdown_connections 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4562480 0x55b4f456e280 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b4f458c870 0x55b4f458ec90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 0x55b4f456bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14bf7fe700 0 client.25429 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2529655245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4567950 0x55b4f456b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2529655245 >> 172.21.15.39:0/2529655245 conn(0x55b4f44c01b0 msgr2=0x55b4f4560f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2529655245 shutdown_connections 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.978+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2529655245 wait complete. 2024-05-24T11:43:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:49.979+0000 7f14f2bb4b80 20 client.25429 trim_cache size 0 max 16384 2024-05-24T11:43:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14f2bb4b80 1 Processor -- start 2024-05-24T11:43:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14f2bb4b80 1 -- start start 2024-05-24T11:43:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4562480 0x55b4f456bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4567950 0x55b4f455c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 0x55b4f456aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f456c530 con 0x55b4f4562480 2024-05-24T11:43:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f456b3e0 con 0x55b4f4568270 2024-05-24T11:43:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f4593e40 con 0x55b4f4567950 2024-05-24T11:43:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14df7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4567950 0x55b4f455c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14df7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4567950 0x55b4f455c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:58050/0 (socket says 172.21.15.39:58050) 2024-05-24T11:43:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.003+0000 7f14df7fe700 1 -- 172.21.15.39:0/2376192555 learned_addr learned my addr 172.21.15.39:0/2376192555 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.004+0000 7f14e4f96700 1 --2- 172.21.15.39:0/2376192555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 0x55b4f456aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.004+0000 7f14df7fe700 1 -- 172.21.15.39:0/2376192555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 msgr2=0x55b4f456aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.004+0000 7f14df7fe700 1 --2- 172.21.15.39:0/2376192555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 0x55b4f456aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.004+0000 7f14df7fe700 1 -- 172.21.15.39:0/2376192555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4562480 msgr2=0x55b4f456bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.004+0000 7f14df7fe700 1 --2- 172.21.15.39:0/2376192555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4562480 0x55b4f456bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.004+0000 7f14df7fe700 1 -- 172.21.15.39:0/2376192555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4f4478500 con 0x55b4f4567950 2024-05-24T11:43:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.004+0000 7f14df7fe700 1 --2- 172.21.15.39:0/2376192555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4567950 0x55b4f455c910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009ec0 tx=0x7f14d4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8f5e6914c51b2c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14c6ffd700 1 -- 172.21.15.39:0/2376192555 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4016070 con 0x55b4f4567950 2024-05-24T11:43:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14c6ffd700 1 -- 172.21.15.39:0/2376192555 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14d4017040 con 0x55b4f4567950 2024-05-24T11:43:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14c6ffd700 1 -- 172.21.15.39:0/2376192555 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4005310 con 0x55b4f4567950 2024-05-24T11:43:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2376192555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4567950 msgr2=0x55b4f455c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2376192555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4567950 0x55b4f455c910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009ec0 tx=0x7f14d4009260 comp rx=0 tx=0).stop 2024-05-24T11:43:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2376192555 shutdown_connections 2024-05-24T11:43:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2376192555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4568270 0x55b4f456aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2376192555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4562480 0x55b4f456bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/2376192555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4567950 0x55b4f455c910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.005+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2376192555 >> 172.21.15.39:0/2376192555 conn(0x55b4f44c01b0 msgr2=0x55b4f4558e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.006+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2376192555 shutdown_connections 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.006+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/2376192555 wait complete. 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.006+0000 7f14f2bb4b80 1 -- start start 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.006+0000 7f14f2bb4b80 1 Processor -- start 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4562480 0x55b4f4581c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 0x55b4f4582140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14f2bb4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4582680 0x55b4f45e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f45843b0 con 0x55b4f4582680 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f4584520 con 0x55b4f4562480 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14f2bb4b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f45f32b0 con 0x55b4f4568270 2024-05-24T11:43:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14f2bb4b80 10 client.0 _ll_register_callbacks cb 0x55b4f4457ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:43:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14df7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 0x55b4f4582140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14df7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 0x55b4f4582140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:58060/0 (socket says 172.21.15.39:58060) 2024-05-24T11:43:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14df7fe700 1 -- 172.21.15.39:0/3196340983 learned_addr learned my addr 172.21.15.39:0/3196340983 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:43:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14e4f96700 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4562480 0x55b4f4581c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14df7fe700 1 -- 172.21.15.39:0/3196340983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4562480 msgr2=0x55b4f4581c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14df7fe700 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4562480 0x55b4f4581c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14df7fe700 1 -- 172.21.15.39:0/3196340983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4582680 msgr2=0x55b4f45e1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:43:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14df7fe700 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4582680 0x55b4f45e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14df7fe700 1 -- 172.21.15.39:0/3196340983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d4009d30 con 0x55b4f4568270 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.007+0000 7f14df7fe700 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 0x55b4f4582140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f14d40046e0 tx=0x7f14d4003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701b220eca474093 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.008+0000 7f14bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.008+0000 7f14bf7fe700 1 -- 172.21.15.39:0/3196340983 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4016070 con 0x55b4f4568270 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.008+0000 7f14bf7fe700 1 -- 172.21.15.39:0/3196340983 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f14d401f2f0 con 0x55b4f4568270 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.008+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/3196340983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14d800a0d0 con 0x55b4f4568270 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.008+0000 7f14bf7fe700 1 -- 172.21.15.39:0/3196340983 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d401f4b0 con 0x55b4f4568270 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.008+0000 7f14f2bb4b80 10 client.25435 fetch_fsmap learned FSMap version 27 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.008+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/3196340983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b4f4337c70 con 0x55b4f4568270 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.008+0000 7f14bf7fe700 1 -- 172.21.15.39:0/3196340983 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14d800a0d0 con 0x55b4f4568270 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.008+0000 7f14bf7fe700 1 -- 172.21.15.39:0/3196340983 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14d4029bc0 con 0x55b4f4568270 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.009+0000 7f14f2bb4b80 10 client.25435 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.009+0000 7f14f2bb4b80 10 client.25435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:43:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.009+0000 7f14f2bb4b80 20 client.25435 populate_metadata read hostname 'smithi039' 2024-05-24T11:43:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.009+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b4f45f4940 0x55b4f45f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:43:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.009+0000 7f14f2bb4b80 4 client.25435 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:43:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.009+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/3196340983 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b4f44aae60 con 0x55b4f45f4940 2024-05-24T11:43:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.009+0000 7f14e4f96700 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b4f45f4940 0x55b4f45f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:43:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.010+0000 7f14e4f96700 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b4f45f4940 0x55b4f45f6d60 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13a1b30feb5f486a server_cookie=b76843e75ee5e0bd in_seq=0 out_seq=0 2024-05-24T11:43:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.010+0000 7f14bf7fe700 10 client.25435 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.037+0000 7f14bf7fe700 1 -- 172.21.15.39:0/3196340983 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b4f44aae60 con 0x55b4f45f4940 2024-05-24T11:43:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.037+0000 7f14bf7fe700 10 client.25435 handle_command_reply: tid=0 2024-05-24T11:43:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 client.25435 shutdown 2024-05-24T11:43:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/3196340983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 msgr2=0x55b4f4582140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 0x55b4f4582140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f14d40046e0 tx=0x7f14d4003b20 comp rx=0 tx=0).stop 2024-05-24T11:43:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/3196340983 shutdown_connections 2024-05-24T11:43:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55b4f45f4940 0x55b4f45f6d60 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4f4582680 0x55b4f45e1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14bf7fe700 0 client.25435 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:43:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b4f4562480 0x55b4f4581c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 --2- 172.21.15.39:0/3196340983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4f4568270 0x55b4f4582140 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/3196340983 >> 172.21.15.39:0/3196340983 conn(0x55b4f44c01b0 msgr2=0x55b4f4565380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/3196340983 shutdown_connections 2024-05-24T11:43:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.038+0000 7f14f2bb4b80 1 -- 172.21.15.39:0/3196340983 wait complete. 2024-05-24T11:43:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.039+0000 7f14f2bb4b80 20 client.25435 trim_cache size 0 max 16384 2024-05-24T11:43:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:43:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:43:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:43:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:43:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.039+0000 7f14e69f9700 1 -- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f14c80a13d0 msgr2=0x7f14c80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.039+0000 7f14e69f9700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f14c80a13d0 0x7f14c80a3890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55b4f455d790 tx=0x7f14d0011f90 comp rx=0 tx=0).stop 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.039+0000 7f14e69f9700 1 -- 172.21.15.39:0/3835443315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00ff560 msgr2=0x7f14e01942f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.039+0000 7f14e69f9700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00ff560 0x7f14e01942f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14d000b530 tx=0x7f14d000b840 comp rx=0 tx=0).stop 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.040+0000 7f14e69f9700 1 -- 172.21.15.39:0/3835443315 shutdown_connections 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.040+0000 7f14e69f9700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14e0105620 0x7f14e0194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.040+0000 7f14e69f9700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f14c80a13d0 0x7f14c80a3890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.040+0000 7f14e69f9700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14e00ff560 0x7f14e01942f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.040+0000 7f14e69f9700 1 --2- 172.21.15.39:0/3835443315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14e00fff20 0x7f14e0194830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.040+0000 7f14e69f9700 1 -- 172.21.15.39:0/3835443315 >> 172.21.15.39:0/3835443315 conn(0x7f14e00773b0 msgr2=0x7f14e00fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.040+0000 7f14e69f9700 1 -- 172.21.15.39:0/3835443315 shutdown_connections 2024-05-24T11:43:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:43:50.040+0000 7f14e69f9700 1 -- 172.21.15.39:0/3835443315 wait complete. 2024-05-24T11:43:50.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:50 smithi039 ceph-mon[42919]: pgmap v171: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 18 MiB/s wr, 1.71k op/s 2024-05-24T11:43:50.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:50 smithi053 ceph-mon[47448]: pgmap v171: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 18 MiB/s wr, 1.71k op/s 2024-05-24T11:43:50.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:50 smithi137 ceph-mon[47907]: pgmap v171: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 18 MiB/s wr, 1.71k op/s 2024-05-24T11:43:52.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:52 smithi039 ceph-mon[42919]: pgmap v172: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-24T11:43:52.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:52 smithi053 ceph-mon[47448]: pgmap v172: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-24T11:43:52.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:52 smithi137 ceph-mon[47907]: pgmap v172: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-24T11:43:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 270.0 s, 1258.8 tps, lat 79.339 ms stddev 21.355 2024-05-24T11:43:54.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:54 smithi039 ceph-mon[42919]: pgmap v173: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-24T11:43:54.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:54 smithi053 ceph-mon[47448]: pgmap v173: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-24T11:43:54.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:54 smithi137 ceph-mon[47907]: pgmap v173: 97 pgs: 97 active+clean; 4.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-05-24T11:43:56.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:56 smithi039 ceph-mon[42919]: pgmap v174: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-05-24T11:43:56.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:56 smithi053 ceph-mon[47448]: pgmap v174: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-05-24T11:43:56.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:56 smithi137 ceph-mon[47907]: pgmap v174: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-05-24T11:43:58.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:43:58 smithi039 ceph-mon[42919]: pgmap v175: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-05-24T11:43:58.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:43:58 smithi053 ceph-mon[47448]: pgmap v175: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-05-24T11:43:58.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:43:58 smithi137 ceph-mon[47907]: pgmap v175: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-05-24T11:43:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 275.0 s, 1267.2 tps, lat 78.982 ms stddev 21.168 2024-05-24T11:44:00.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:00 smithi039 ceph-mon[42919]: pgmap v176: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-05-24T11:44:00.494 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:00 smithi053 ceph-mon[47448]: pgmap v176: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-05-24T11:44:00.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:00 smithi137 ceph-mon[47907]: pgmap v176: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-05-24T11:44:02.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:02 smithi039 ceph-mon[42919]: pgmap v177: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-05-24T11:44:02.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:02 smithi053 ceph-mon[47448]: pgmap v177: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-05-24T11:44:02.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:02 smithi137 ceph-mon[47907]: pgmap v177: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-05-24T11:44:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.211+0000 7f2496e87700 1 -- 172.21.15.39:0/2176296144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490073c60 msgr2=0x7f2490111960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.211+0000 7f2496e87700 1 --2- 172.21.15.39:0/2176296144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490073c60 0x7f2490111960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2480009990 tx=0x7f2480009ca0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.211+0000 7f2496e87700 1 -- 172.21.15.39:0/2176296144 shutdown_connections 2024-05-24T11:44:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.211+0000 7f2496e87700 1 --2- 172.21.15.39:0/2176296144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2490073340 0x7f2490073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.211+0000 7f2496e87700 1 --2- 172.21.15.39:0/2176296144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2490111ea0 0x7f2490114290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.211+0000 7f2496e87700 1 --2- 172.21.15.39:0/2176296144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490073c60 0x7f2490111960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.211+0000 7f2496e87700 1 -- 172.21.15.39:0/2176296144 >> 172.21.15.39:0/2176296144 conn(0x7f24900fc9a0 msgr2=0x7f24900fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.212+0000 7f2496e87700 1 -- 172.21.15.39:0/2176296144 shutdown_connections 2024-05-24T11:44:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.212+0000 7f2496e87700 1 -- 172.21.15.39:0/2176296144 wait complete. 2024-05-24T11:44:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.212+0000 7f2496e87700 1 Processor -- start 2024-05-24T11:44:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.212+0000 7f2496e87700 1 -- start start 2024-05-24T11:44:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f2496e87700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490073340 0x7f24901113c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f2496e87700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2490073c60 0x7f2490111940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f2496e87700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2490111ea0 0x7f249010a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f2496e87700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249010ae70 con 0x7f2490073c60 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f2496e87700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249010afe0 con 0x7f2490073340 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f2496e87700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249010b1a0 con 0x7f2490111ea0 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f248ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2490073c60 0x7f2490111940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f248ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2490073c60 0x7f2490111940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57194/0 (socket says 172.21.15.39:57194) 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f248ffff700 1 -- 172.21.15.39:0/1171015308 learned_addr learned my addr 172.21.15.39:0/1171015308 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f2494c23700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490073340 0x7f24901113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f248ffff700 1 -- 172.21.15.39:0/1171015308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2490111ea0 msgr2=0x7f249010a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f248ffff700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2490111ea0 0x7f249010a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f248ffff700 1 -- 172.21.15.39:0/1171015308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490073340 msgr2=0x7f24901113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f248ffff700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490073340 0x7f24901113c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.213+0000 7f248ffff700 1 -- 172.21.15.39:0/1171015308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24800095f0 con 0x7f2490073c60 2024-05-24T11:44:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.214+0000 7f248ffff700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2490073c60 0x7f2490111940 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2480009530 tx=0x7f24800045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.214+0000 7f248dffb700 1 -- 172.21.15.39:0/1171015308 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248001d070 con 0x7f2490073c60 2024-05-24T11:44:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.214+0000 7f248dffb700 1 -- 172.21.15.39:0/1171015308 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2480021410 con 0x7f2490073c60 2024-05-24T11:44:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.214+0000 7f248dffb700 1 -- 172.21.15.39:0/1171015308 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24800046d0 con 0x7f2490073c60 2024-05-24T11:44:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.215+0000 7f2496e87700 1 -- 172.21.15.39:0/1171015308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249010ef50 con 0x7f2490073c60 2024-05-24T11:44:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.216+0000 7f2496e87700 1 -- 172.21.15.39:0/1171015308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249010f380 con 0x7f2490073c60 2024-05-24T11:44:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.219+0000 7f248dffb700 1 -- 172.21.15.39:0/1171015308 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2480021580 con 0x7f2490073c60 2024-05-24T11:44:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.221+0000 7f248dffb700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f247c0a13d0 0x7f247c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.221+0000 7f248dffb700 1 -- 172.21.15.39:0/1171015308 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f24800cdba0 con 0x7f2490073c60 2024-05-24T11:44:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.222+0000 7f24a3042b80 1 Processor -- start 2024-05-24T11:44:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.222+0000 7f24a3042b80 1 -- start start 2024-05-24T11:44:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.222+0000 7f2494c23700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f247c0a13d0 0x7f247c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef5ffed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 0x557ef5ff8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ff98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f24a3042b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef5f500f0 con 0x557ef5fff270 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f24a3042b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef5f4dab0 con 0x557ef5ffe950 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f24a3042b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef5f493a0 con 0x557ef5ff9480 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f248ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 0x557ef5ff8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f2494c23700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef5ffed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f248ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 0x557ef5ff8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57208/0 (socket says 172.21.15.39:57208) 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f248ffff700 1 -- 172.21.15.39:0/2564583123 learned_addr learned my addr 172.21.15.39:0/2564583123 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f2494c23700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f247c0a13d0 0x7f247c0a3890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2488009380 tx=0x7f248800b040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f2495424700 1 --2- 172.21.15.39:0/2564583123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ff98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f2495424700 1 -- 172.21.15.39:0/2564583123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 msgr2=0x557ef5ffed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f2495424700 1 --2- 172.21.15.39:0/2564583123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef5ffed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f2495424700 1 -- 172.21.15.39:0/2564583123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 msgr2=0x557ef5ff8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f2495424700 1 --2- 172.21.15.39:0/2564583123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 0x557ef5ff8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.223+0000 7f2495424700 1 -- 172.21.15.39:0/2564583123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef5dca350 con 0x557ef5ff9480 2024-05-24T11:44:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.224+0000 7f2495424700 1 --2- 172.21.15.39:0/2564583123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ff98a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2478009b20 tx=0x7f2478009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bc9c961aa626f05 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.224+0000 7f2486ffd700 1 -- 172.21.15.39:0/2564583123 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478004230 con 0x557ef5ff9480 2024-05-24T11:44:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.224+0000 7f2486ffd700 1 -- 172.21.15.39:0/2564583123 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2478004cc0 con 0x557ef5ff9480 2024-05-24T11:44:03.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.224+0000 7f2486ffd700 1 -- 172.21.15.39:0/2564583123 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478005630 con 0x557ef5ff9480 2024-05-24T11:44:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.225+0000 7f24a3042b80 1 -- 172.21.15.39:0/2564583123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 msgr2=0x557ef5ff98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.225+0000 7f24a3042b80 1 --2- 172.21.15.39:0/2564583123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ff98a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2478009b20 tx=0x7f2478009e30 comp rx=0 tx=0).stop 2024-05-24T11:44:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.225+0000 7f24a3042b80 1 -- 172.21.15.39:0/2564583123 shutdown_connections 2024-05-24T11:44:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.225+0000 7f24a3042b80 1 --2- 172.21.15.39:0/2564583123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ff98a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.225+0000 7f24a3042b80 1 --2- 172.21.15.39:0/2564583123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef5ffed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.225+0000 7f24a3042b80 1 --2- 172.21.15.39:0/2564583123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 0x557ef5ff8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.225+0000 7f24a3042b80 1 -- 172.21.15.39:0/2564583123 >> 172.21.15.39:0/2564583123 conn(0x557ef5f571b0 msgr2=0x557ef5ff3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.226+0000 7f24a3042b80 1 -- 172.21.15.39:0/2564583123 shutdown_connections 2024-05-24T11:44:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.226+0000 7f24a3042b80 1 -- 172.21.15.39:0/2564583123 wait complete. 2024-05-24T11:44:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.226+0000 7f24a3042b80 1 -- start start 2024-05-24T11:44:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.226+0000 7f24a3042b80 1 Processor -- start 2024-05-24T11:44:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ff9480 0x557ef6002550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ffe950 0x557ef6002a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6002fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f24a3042b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef5ee5500 con 0x557ef5ffe950 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f24a3042b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef5ee51a0 con 0x557ef5ff9480 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f24a3042b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef5dcc640 con 0x557ef5fff270 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f24a3042b80 10 client.0 _ll_register_callbacks cb 0x557ef5eeead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f2495424700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6002fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f2495424700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6002fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:37456/0 (socket says 172.21.15.39:37456) 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f2495424700 1 -- 172.21.15.39:0/798616996 learned_addr learned my addr 172.21.15.39:0/798616996 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.227+0000 7f248ffff700 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ffe950 0x557ef6002a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f2495424700 1 -- 172.21.15.39:0/798616996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ff9480 msgr2=0x557ef6002550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f2495424700 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ff9480 0x557ef6002550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f2495424700 1 -- 172.21.15.39:0/798616996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ffe950 msgr2=0x557ef6002a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f2495424700 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ffe950 0x557ef6002a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f2495424700 1 -- 172.21.15.39:0/798616996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2478009800 con 0x557ef5fff270 2024-05-24T11:44:03.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f2495424700 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6002fd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2478000c00 tx=0x7f2478005510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61143c3f6adaa9a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f246ffff700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:44:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f246ffff700 1 -- 172.21.15.39:0/798616996 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478027070 con 0x557ef5fff270 2024-05-24T11:44:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f246ffff700 1 -- 172.21.15.39:0/798616996 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f2478014070 con 0x557ef5fff270 2024-05-24T11:44:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.228+0000 7f24a3042b80 1 -- 172.21.15.39:0/798616996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ef5ea2210 con 0x557ef5fff270 2024-05-24T11:44:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.229+0000 7f246ffff700 1 -- 172.21.15.39:0/798616996 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2478005020 con 0x557ef5fff270 2024-05-24T11:44:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.229+0000 7f24a3042b80 10 client.25444 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:03.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.229+0000 7f24a3042b80 1 -- 172.21.15.39:0/798616996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557ef5d76b00 con 0x557ef5fff270 2024-05-24T11:44:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.230+0000 7f246ffff700 1 -- 172.21.15.39:0/798616996 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557ef5ea2210 con 0x557ef5fff270 2024-05-24T11:44:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.230+0000 7f246ffff700 1 -- 172.21.15.39:0/798616996 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f247802a070 con 0x557ef5fff270 2024-05-24T11:44:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.230+0000 7f24a3042b80 10 client.25444 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.230+0000 7f24a3042b80 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.230+0000 7f24a3042b80 20 client.25444 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.230+0000 7f24a3042b80 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557ef6012ae0 0x557ef6014f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.230+0000 7f24a3042b80 4 client.25444 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:44:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.230+0000 7f24a3042b80 1 -- 172.21.15.39:0/798616996 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ef5fc87b0 con 0x557ef6012ae0 2024-05-24T11:44:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.230+0000 7f2494c23700 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557ef6012ae0 0x557ef6014f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.231+0000 7f246ffff700 10 client.25444 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.231+0000 7f2494c23700 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557ef6012ae0 0x557ef6014f00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa82d5a35d4daf9 server_cookie=acf1ae482a0df766 in_seq=0 out_seq=0 2024-05-24T11:44:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.232+0000 7f246ffff700 1 -- 172.21.15.39:0/798616996 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ef5fc87b0 con 0x557ef6012ae0 2024-05-24T11:44:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.232+0000 7f246ffff700 10 client.25444 handle_command_reply: tid=0 2024-05-24T11:44:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.232+0000 7f24a3042b80 1 client.25444 shutdown 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.232+0000 7f24a3042b80 1 -- 172.21.15.39:0/798616996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 msgr2=0x557ef6002fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.232+0000 7f24a3042b80 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6002fd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2478000c00 tx=0x7f2478005510 comp rx=0 tx=0).stop 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.232+0000 7f24a3042b80 1 -- 172.21.15.39:0/798616996 shutdown_connections 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.232+0000 7f24a3042b80 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6002fd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.233+0000 7f24a3042b80 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557ef6012ae0 0x557ef6014f00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.233+0000 7f24a3042b80 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ff9480 0x557ef6002550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.233+0000 7f24a3042b80 1 --2- 172.21.15.39:0/798616996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ffe950 0x557ef6002a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.233+0000 7f24a3042b80 1 -- 172.21.15.39:0/798616996 >> 172.21.15.39:0/798616996 conn(0x557ef5f571b0 msgr2=0x557ef5ffd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.233+0000 7f246ffff700 0 client.25444 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.233+0000 7f24a3042b80 1 -- 172.21.15.39:0/798616996 shutdown_connections 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.233+0000 7f24a3042b80 1 -- 172.21.15.39:0/798616996 wait complete. 2024-05-24T11:44:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.233+0000 7f24a3042b80 20 client.25444 trim_cache size 0 max 16384 2024-05-24T11:44:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.258+0000 7f24a3042b80 1 Processor -- start 2024-05-24T11:44:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.258+0000 7f24a3042b80 1 -- start start 2024-05-24T11:44:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.258+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ffed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef5ff3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 0x557ef5ff9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f24a3042b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef5ff9da0 con 0x557ef5fff270 2024-05-24T11:44:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f24a3042b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef6003ac0 con 0x557ef5ffe950 2024-05-24T11:44:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f24a3042b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef6000a30 con 0x557ef5ff9480 2024-05-24T11:44:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f2494c23700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ffed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f2494c23700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ffed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:37462/0 (socket says 172.21.15.39:37462) 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f2494c23700 1 -- 172.21.15.39:0/1755362240 learned_addr learned my addr 172.21.15.39:0/1755362240 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f2494c23700 1 -- 172.21.15.39:0/1755362240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 msgr2=0x557ef5ff3910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f2494c23700 1 --2- 172.21.15.39:0/1755362240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef5ff3910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f2494c23700 1 -- 172.21.15.39:0/1755362240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 msgr2=0x557ef5ff9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f2494c23700 1 --2- 172.21.15.39:0/1755362240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 0x557ef5ff9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.259+0000 7f2494c23700 1 -- 172.21.15.39:0/1755362240 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef5f0f500 con 0x557ef5ff9480 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.260+0000 7f2494c23700 1 --2- 172.21.15.39:0/1755362240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ffed30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2488007ff0 tx=0x7f2488004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2ed421861727d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.260+0000 7f24877fe700 1 -- 172.21.15.39:0/1755362240 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248802b070 con 0x557ef5ff9480 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.260+0000 7f24877fe700 1 -- 172.21.15.39:0/1755362240 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f248800f040 con 0x557ef5ff9480 2024-05-24T11:44:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.260+0000 7f24877fe700 1 -- 172.21.15.39:0/1755362240 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800e050 con 0x557ef5ff9480 2024-05-24T11:44:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.261+0000 7f24a3042b80 1 -- 172.21.15.39:0/1755362240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 msgr2=0x557ef5ffed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.261+0000 7f24a3042b80 1 --2- 172.21.15.39:0/1755362240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ffed30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2488007ff0 tx=0x7f2488004b40 comp rx=0 tx=0).stop 2024-05-24T11:44:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.261+0000 7f24a3042b80 1 -- 172.21.15.39:0/1755362240 shutdown_connections 2024-05-24T11:44:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.261+0000 7f24a3042b80 1 --2- 172.21.15.39:0/1755362240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5ff9480 0x557ef5ffed30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.261+0000 7f24a3042b80 1 --2- 172.21.15.39:0/1755362240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5fff270 0x557ef5ff9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.261+0000 7f24a3042b80 1 --2- 172.21.15.39:0/1755362240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef5ff3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.261+0000 7f24a3042b80 1 -- 172.21.15.39:0/1755362240 >> 172.21.15.39:0/1755362240 conn(0x557ef5f571b0 msgr2=0x557ef6001de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.261+0000 7f24a3042b80 1 -- 172.21.15.39:0/1755362240 shutdown_connections 2024-05-24T11:44:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.261+0000 7f24a3042b80 1 -- 172.21.15.39:0/1755362240 wait complete. 2024-05-24T11:44:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f24a3042b80 1 -- start start 2024-05-24T11:44:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f24a3042b80 1 Processor -- start 2024-05-24T11:44:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ff9480 0x557ef6009bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef600a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f24a3042b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f24a3042b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef6006140 con 0x557ef5ff9480 2024-05-24T11:44:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f24a3042b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef6006280 con 0x557ef5ffe950 2024-05-24T11:44:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f24a3042b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef6006440 con 0x557ef5fff270 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f24a3042b80 10 client.0 _ll_register_callbacks cb 0x557ef5eeead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f2494c23700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ff9480 0x557ef6009bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f2494c23700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ff9480 0x557ef6009bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57244/0 (socket says 172.21.15.39:57244) 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f2494c23700 1 -- 172.21.15.39:0/886877677 learned_addr learned my addr 172.21.15.39:0/886877677 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.262+0000 7f248ffff700 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef600a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f2495424700 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f2494c23700 1 -- 172.21.15.39:0/886877677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 msgr2=0x557ef6005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f2494c23700 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f2494c23700 1 -- 172.21.15.39:0/886877677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 msgr2=0x557ef600a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f2494c23700 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef600a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f2494c23700 1 -- 172.21.15.39:0/886877677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2488026e70 con 0x557ef5ff9480 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f2494c23700 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ff9480 0x557ef6009bf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2488026330 tx=0x7f2488026360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35eec5c6463a7980 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f246ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:44:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f246ffff700 1 -- 172.21.15.39:0/886877677 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248802b030 con 0x557ef5ff9480 2024-05-24T11:44:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f246ffff700 1 -- 172.21.15.39:0/886877677 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f248802c070 con 0x557ef5ff9480 2024-05-24T11:44:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f24a3042b80 1 -- 172.21.15.39:0/886877677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2478004cc0 con 0x557ef5ff9480 2024-05-24T11:44:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f246ffff700 1 -- 172.21.15.39:0/886877677 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f248800e050 con 0x557ef5ff9480 2024-05-24T11:44:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f24a3042b80 10 client.15843 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.263+0000 7f24a3042b80 1 -- 172.21.15.39:0/886877677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557ef5dcec70 con 0x557ef5ff9480 2024-05-24T11:44:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.264+0000 7f246ffff700 1 -- 172.21.15.39:0/886877677 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f248803d080 con 0x557ef5ff9480 2024-05-24T11:44:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f246ffff700 1 -- 172.21.15.39:0/886877677 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2488039410 con 0x557ef5ff9480 2024-05-24T11:44:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f24a3042b80 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f24a3042b80 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f24a3042b80 20 client.15843 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f24a3042b80 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557ef6068f90 0x557ef606b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f24a3042b80 4 client.15843 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:44:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f24a3042b80 1 -- 172.21.15.39:0/886877677 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557ef5f41e60 con 0x557ef6068f90 2024-05-24T11:44:03.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f248ffff700 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557ef6068f90 0x557ef606b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f246ffff700 10 client.15843 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.265+0000 7f248ffff700 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557ef6068f90 0x557ef606b3b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=279d720904e1516 server_cookie=94f93c3299dba273 in_seq=0 out_seq=0 2024-05-24T11:44:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.288+0000 7f246ffff700 1 -- 172.21.15.39:0/886877677 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557ef5f41e60 con 0x557ef6068f90 2024-05-24T11:44:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.288+0000 7f246ffff700 10 client.15843 handle_command_reply: tid=0 2024-05-24T11:44:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.288+0000 7f24a3042b80 1 client.15843 shutdown 2024-05-24T11:44:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 -- 172.21.15.39:0/886877677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ff9480 msgr2=0x557ef6009bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ff9480 0x557ef6009bf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2488026330 tx=0x7f2488026360 comp rx=0 tx=0).stop 2024-05-24T11:44:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 -- 172.21.15.39:0/886877677 shutdown_connections 2024-05-24T11:44:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef5fff270 0x557ef6005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x557ef6068f90 0x557ef606b3b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557ef5ff9480 0x557ef6009bf0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 --2- 172.21.15.39:0/886877677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ef5ffe950 0x557ef600a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 -- 172.21.15.39:0/886877677 >> 172.21.15.39:0/886877677 conn(0x557ef5f571b0 msgr2=0x557ef5ff75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f246ffff700 0 client.15843 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 -- 172.21.15.39:0/886877677 shutdown_connections 2024-05-24T11:44:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 1 -- 172.21.15.39:0/886877677 wait complete. 2024-05-24T11:44:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.289+0000 7f24a3042b80 20 client.15843 trim_cache size 0 max 16384 2024-05-24T11:44:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:44:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:44:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:44:03.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:44:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.290+0000 7f2496e87700 1 -- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f247c0a13d0 msgr2=0x7f247c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.290+0000 7f2496e87700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f247c0a13d0 0x7f247c0a3890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2488009380 tx=0x7f248800b040 comp rx=0 tx=0).stop 2024-05-24T11:44:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.290+0000 7f2496e87700 1 -- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2490073c60 msgr2=0x7f2490111940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.290+0000 7f2496e87700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2490073c60 0x7f2490111940 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2480009530 tx=0x7f24800045b0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.291+0000 7f2496e87700 1 -- 172.21.15.39:0/1171015308 shutdown_connections 2024-05-24T11:44:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.291+0000 7f2496e87700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2490111ea0 0x7f249010a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.291+0000 7f2496e87700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f247c0a13d0 0x7f247c0a3890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.291+0000 7f2496e87700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490073340 0x7f24901113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.291+0000 7f2496e87700 1 --2- 172.21.15.39:0/1171015308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2490073c60 0x7f2490111940 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.291+0000 7f2496e87700 1 -- 172.21.15.39:0/1171015308 >> 172.21.15.39:0/1171015308 conn(0x7f24900fc9a0 msgr2=0x7f24900fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.291+0000 7f2496e87700 1 -- 172.21.15.39:0/1171015308 shutdown_connections 2024-05-24T11:44:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:03.291+0000 7f2496e87700 1 -- 172.21.15.39:0/1171015308 wait complete. 2024-05-24T11:44:04.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 280.0 s, 1186.6 tps, lat 84.333 ms stddev 49.424 2024-05-24T11:44:04.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:04 smithi039 ceph-mon[42919]: pgmap v178: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-05-24T11:44:04.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:04 smithi053 ceph-mon[47448]: pgmap v178: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-05-24T11:44:04.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:04 smithi137 ceph-mon[47907]: pgmap v178: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 1.08k op/s 2024-05-24T11:44:06.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:06 smithi039 ceph-mon[42919]: pgmap v179: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 16 MiB/s wr, 1.68k op/s 2024-05-24T11:44:06.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:06 smithi053 ceph-mon[47448]: pgmap v179: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 16 MiB/s wr, 1.68k op/s 2024-05-24T11:44:06.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:06 smithi137 ceph-mon[47907]: pgmap v179: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 16 MiB/s wr, 1.68k op/s 2024-05-24T11:44:08.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:08 smithi039 ceph-mon[42919]: pgmap v180: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-05-24T11:44:08.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:08 smithi053 ceph-mon[47448]: pgmap v180: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-05-24T11:44:08.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:08 smithi137 ceph-mon[47907]: pgmap v180: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-05-24T11:44:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 285.0 s, 1324.2 tps, lat 75.514 ms stddev 22.639 2024-05-24T11:44:10.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:10 smithi039 ceph-mon[42919]: pgmap v181: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-05-24T11:44:10.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:10 smithi053 ceph-mon[47448]: pgmap v181: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-05-24T11:44:10.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:10 smithi137 ceph-mon[47907]: pgmap v181: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-05-24T11:44:12.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:12 smithi039 ceph-mon[42919]: pgmap v182: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-05-24T11:44:12.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:12 smithi053 ceph-mon[47448]: pgmap v182: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-05-24T11:44:12.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:12 smithi137 ceph-mon[47907]: pgmap v182: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-05-24T11:44:14.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 290.0 s, 1285.2 tps, lat 77.949 ms stddev 23.387 2024-05-24T11:44:14.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:14 smithi039 ceph-mon[42919]: pgmap v183: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-05-24T11:44:14.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:14 smithi053 ceph-mon[47448]: pgmap v183: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-05-24T11:44:14.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:14 smithi137 ceph-mon[47907]: pgmap v183: 97 pgs: 97 active+clean; 4.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-05-24T11:44:16.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:16 smithi039 ceph-mon[42919]: pgmap v184: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2024-05-24T11:44:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.463+0000 7f4671b93700 1 -- 172.21.15.39:0/4054477396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c105f20 msgr2=0x7f466c106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.463+0000 7f4671b93700 1 --2- 172.21.15.39:0/4054477396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c105f20 0x7f466c106300 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f465c009990 tx=0x7f465c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.463+0000 7f4671b93700 1 -- 172.21.15.39:0/4054477396 shutdown_connections 2024-05-24T11:44:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.463+0000 7f4671b93700 1 --2- 172.21.15.39:0/4054477396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f466c100c40 0x7f466c10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.463+0000 7f4671b93700 1 --2- 172.21.15.39:0/4054477396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c105f20 0x7f466c106300 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.463+0000 7f4671b93700 1 --2- 172.21.15.39:0/4054477396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c100280 0x7f466c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.463+0000 7f4671b93700 1 -- 172.21.15.39:0/4054477396 >> 172.21.15.39:0/4054477396 conn(0x7f466c076bc0 msgr2=0x7f466c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.463+0000 7f4671b93700 1 -- 172.21.15.39:0/4054477396 shutdown_connections 2024-05-24T11:44:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.463+0000 7f4671b93700 1 -- 172.21.15.39:0/4054477396 wait complete. 2024-05-24T11:44:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.464+0000 7f4671b93700 1 Processor -- start 2024-05-24T11:44:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.465+0000 7f4671b93700 1 -- start start 2024-05-24T11:44:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.465+0000 7f4671b93700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c100280 0x7f466c109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.465+0000 7f4671b93700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f466c100c40 0x7f466c10a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.465+0000 7f4671b93700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c105f20 0x7f466c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.465+0000 7f4671b93700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c19b0a0 con 0x7f466c100280 2024-05-24T11:44:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.465+0000 7f4671b93700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c19b210 con 0x7f466c105f20 2024-05-24T11:44:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.465+0000 7f4671b93700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c19b3d0 con 0x7f466c100c40 2024-05-24T11:44:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.465+0000 7f466bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c105f20 0x7f466c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.466+0000 7f466bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c105f20 0x7f466c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:41234/0 (socket says 172.21.15.39:41234) 2024-05-24T11:44:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.466+0000 7f466bfff700 1 -- 172.21.15.39:0/2345517606 learned_addr learned my addr 172.21.15.39:0/2345517606 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.466+0000 7f466b7fe700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c100280 0x7f466c109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.466+0000 7f466bfff700 1 -- 172.21.15.39:0/2345517606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f466c100c40 msgr2=0x7f466c10a1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:44:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.466+0000 7f466bfff700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f466c100c40 0x7f466c10a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.466+0000 7f466bfff700 1 -- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c100280 msgr2=0x7f466c109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.466+0000 7f466bfff700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c100280 0x7f466c109c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.466+0000 7f466bfff700 1 -- 172.21.15.39:0/2345517606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0095f0 con 0x7f466c105f20 2024-05-24T11:44:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.466+0000 7f466bfff700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c105f20 0x7f466c19aa10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4664007ee0 tx=0x7f466400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.467+0000 7f4668ff9700 1 -- 172.21.15.39:0/2345517606 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664017070 con 0x7f466c105f20 2024-05-24T11:44:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.467+0000 7f4668ff9700 1 -- 172.21.15.39:0/2345517606 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f466400f040 con 0x7f466c105f20 2024-05-24T11:44:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.467+0000 7f4668ff9700 1 -- 172.21.15.39:0/2345517606 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664005710 con 0x7f466c105f20 2024-05-24T11:44:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.467+0000 7f4671b93700 1 -- 172.21.15.39:0/2345517606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c19b6b0 con 0x7f466c105f20 2024-05-24T11:44:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.467+0000 7f4671b93700 1 -- 172.21.15.39:0/2345517606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c19dcc0 con 0x7f466c105f20 2024-05-24T11:44:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.471+0000 7f4668ff9700 1 -- 172.21.15.39:0/2345517606 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4664005870 con 0x7f466c105f20 2024-05-24T11:44:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f4668ff9700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46540a1350 0x7f46540a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f4668ff9700 1 -- 172.21.15.39:0/2345517606 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f46640cc3c0 con 0x7f466c105f20 2024-05-24T11:44:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f467dd4eb80 1 Processor -- start 2024-05-24T11:44:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f467dd4eb80 1 -- start start 2024-05-24T11:44:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f466b7fe700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46540a1350 0x7f46540a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957def40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc957360f0 con 0x55cc957df480 2024-05-24T11:44:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc95733ab0 con 0x55cc957e4950 2024-05-24T11:44:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc9572f3a0 con 0x55cc957e5270 2024-05-24T11:44:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.472+0000 7f466affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957def40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f466b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f466affd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957def40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:34296/0 (socket says 172.21.15.39:34296) 2024-05-24T11:44:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f466affd700 1 -- 172.21.15.39:0/2405462420 learned_addr learned my addr 172.21.15.39:0/2405462420 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f466affd700 1 -- 172.21.15.39:0/2405462420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 msgr2=0x55cc957e4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f466affd700 1 --2- 172.21.15.39:0/2405462420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f466affd700 1 -- 172.21.15.39:0/2405462420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 msgr2=0x55cc957df8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:44:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f466affd700 1 --2- 172.21.15.39:0/2405462420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957df8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f466affd700 1 -- 172.21.15.39:0/2405462420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc955b0350 con 0x55cc957e5270 2024-05-24T11:44:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f466affd700 1 --2- 172.21.15.39:0/2405462420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957def40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f466000bab0 tx=0x7f466000bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e7b8cb248b7cd15 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.473+0000 7f4651ffb700 1 -- 172.21.15.39:0/2405462420 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660004a30 con 0x55cc957e5270 2024-05-24T11:44:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.474+0000 7f4651ffb700 1 -- 172.21.15.39:0/2405462420 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f466000dc60 con 0x55cc957e5270 2024-05-24T11:44:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.474+0000 7f4651ffb700 1 -- 172.21.15.39:0/2405462420 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660005440 con 0x55cc957e5270 2024-05-24T11:44:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.474+0000 7f467dd4eb80 1 -- 172.21.15.39:0/2405462420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 msgr2=0x55cc957def40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.474+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/2405462420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957def40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f466000bab0 tx=0x7f466000bdc0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.475+0000 7f466b7fe700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46540a1350 0x7f46540a3810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f465c006010 tx=0x7f465c00b540 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.475+0000 7f467dd4eb80 1 -- 172.21.15.39:0/2405462420 shutdown_connections 2024-05-24T11:44:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.475+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/2405462420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957df8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.475+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/2405462420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.475+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/2405462420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957def40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.475+0000 7f467dd4eb80 1 -- 172.21.15.39:0/2405462420 >> 172.21.15.39:0/2405462420 conn(0x55cc9573d1b0 msgr2=0x55cc957d9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.475+0000 7f467dd4eb80 1 -- 172.21.15.39:0/2405462420 shutdown_connections 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.475+0000 7f467dd4eb80 1 -- 172.21.15.39:0/2405462420 wait complete. 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.475+0000 7f467dd4eb80 1 -- start start 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.476+0000 7f467dd4eb80 1 Processor -- start 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.476+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957eb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.476+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.476+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.476+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc956cb500 con 0x55cc957df480 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.476+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc956cb1a0 con 0x55cc957e4950 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.476+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc955b2640 con 0x55cc957e5270 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.476+0000 7f467dd4eb80 10 client.0 _ll_register_callbacks cb 0x55cc956d4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466affd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:41248/0 (socket says 172.21.15.39:41248) 2024-05-24T11:44:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466affd700 1 -- 172.21.15.39:0/3345918050 learned_addr learned my addr 172.21.15.39:0/3345918050 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466bfff700 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466affd700 1 -- 172.21.15.39:0/3345918050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 msgr2=0x55cc957e8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466affd700 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957e8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466affd700 1 -- 172.21.15.39:0/3345918050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 msgr2=0x55cc957eb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466affd700 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466affd700 1 -- 172.21.15.39:0/3345918050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466000b6e0 con 0x55cc957e4950 2024-05-24T11:44:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f466affd700 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e84f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4660004fc0 tx=0x7f466000d880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a47dc269e6f5df3 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f464affd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:44:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.477+0000 7f464affd700 1 -- 172.21.15.39:0/3345918050 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466001f070 con 0x55cc957e4950 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.478+0000 7f464affd700 1 -- 172.21.15.39:0/3345918050 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f46600052d0 con 0x55cc957e4950 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.478+0000 7f467dd4eb80 1 -- 172.21.15.39:0/3345918050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc95688210 con 0x55cc957e4950 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.478+0000 7f464affd700 1 -- 172.21.15.39:0/3345918050 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4660005490 con 0x55cc957e4950 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.478+0000 7f467dd4eb80 10 client.25556 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.478+0000 7f467dd4eb80 1 -- 172.21.15.39:0/3345918050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cc9555cb00 con 0x55cc957e4950 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.478+0000 7f464affd700 1 -- 172.21.15.39:0/3345918050 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cc95688210 con 0x55cc957e4950 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.479+0000 7f464affd700 1 -- 172.21.15.39:0/3345918050 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4660010460 con 0x55cc957e4950 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.479+0000 7f467dd4eb80 10 client.25556 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.479+0000 7f467dd4eb80 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.479+0000 7f467dd4eb80 20 client.25556 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.479+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55cc95809b40 0x55cc9580bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.479+0000 7f467dd4eb80 4 client.25556 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:44:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.479+0000 7f467dd4eb80 1 -- 172.21.15.39:0/3345918050 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc957ae7b0 con 0x55cc95809b40 2024-05-24T11:44:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.479+0000 7f466b7fe700 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55cc95809b40 0x55cc9580bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.480+0000 7f466b7fe700 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55cc95809b40 0x55cc9580bf60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce79fd3c1bdbc86 server_cookie=fe4f725e2a8141d0 in_seq=0 out_seq=0 2024-05-24T11:44:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.480+0000 7f464affd700 10 client.25556 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.480+0000 7f464affd700 1 -- 172.21.15.39:0/3345918050 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc957ae7b0 con 0x55cc95809b40 2024-05-24T11:44:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.480+0000 7f464affd700 10 client.25556 handle_command_reply: tid=0 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 client.25556 shutdown 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 -- 172.21.15.39:0/3345918050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 msgr2=0x55cc957e84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e84f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4660004fc0 tx=0x7f466000d880 comp rx=0 tx=0).stop 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 -- 172.21.15.39:0/3345918050 shutdown_connections 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55cc95809b40 0x55cc9580bf60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957eb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/3345918050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957e84f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 -- 172.21.15.39:0/3345918050 >> 172.21.15.39:0/3345918050 conn(0x55cc9573d1b0 msgr2=0x55cc957e34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f464affd700 0 client.25556 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 -- 172.21.15.39:0/3345918050 shutdown_connections 2024-05-24T11:44:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.481+0000 7f467dd4eb80 1 -- 172.21.15.39:0/3345918050 wait complete. 2024-05-24T11:44:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.482+0000 7f467dd4eb80 20 client.25556 trim_cache size 0 max 16384 2024-05-24T11:44:16.492 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:16 smithi053 ceph-mon[47448]: pgmap v184: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2024-05-24T11:44:16.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:16 smithi137 ceph-mon[47907]: pgmap v184: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2024-05-24T11:44:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f467dd4eb80 1 Processor -- start 2024-05-24T11:44:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f467dd4eb80 1 -- start start 2024-05-24T11:44:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957df480 0x55cc957d9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957e4950 0x55cc95800c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e5270 0x55cc95801160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc957d4330 con 0x55cc957e4950 2024-05-24T11:44:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc957e45e0 con 0x55cc957e5270 2024-05-24T11:44:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc957db0f0 con 0x55cc957df480 2024-05-24T11:44:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f466b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957df480 0x55cc957d9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f466b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957df480 0x55cc957d9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:34326/0 (socket says 172.21.15.39:34326) 2024-05-24T11:44:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f466b7fe700 1 -- 172.21.15.39:0/1090300165 learned_addr learned my addr 172.21.15.39:0/1090300165 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f466bfff700 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e5270 0x55cc95801160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.505+0000 7f466affd700 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957e4950 0x55cc95800c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f466b7fe700 1 -- 172.21.15.39:0/1090300165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e5270 msgr2=0x55cc95801160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f466b7fe700 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e5270 0x55cc95801160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f466b7fe700 1 -- 172.21.15.39:0/1090300165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957e4950 msgr2=0x55cc95800c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f466b7fe700 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957e4950 0x55cc95800c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f466b7fe700 1 -- 172.21.15.39:0/1090300165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc956f5500 con 0x55cc957df480 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f466affd700 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957e4950 0x55cc95800c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f466b7fe700 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957df480 0x55cc957d9910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f465c005c50 tx=0x7f465c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc0a42891b066652 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f46527fc700 1 -- 172.21.15.39:0/1090300165 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c02f040 con 0x55cc957df480 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f46527fc700 1 -- 172.21.15.39:0/1090300165 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f465c01e070 con 0x55cc957df480 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.506+0000 7f46527fc700 1 -- 172.21.15.39:0/1090300165 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c00f460 con 0x55cc957df480 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.507+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1090300165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957df480 msgr2=0x55cc957d9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.507+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957df480 0x55cc957d9910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f465c005c50 tx=0x7f465c020000 comp rx=0 tx=0).stop 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.507+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1090300165 shutdown_connections 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.507+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957df480 0x55cc957d9910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.507+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e5270 0x55cc95801160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.507+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1090300165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957e4950 0x55cc95800c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.507+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1090300165 >> 172.21.15.39:0/1090300165 conn(0x55cc9573d1b0 msgr2=0x55cc958108c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.507+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1090300165 shutdown_connections 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.507+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1090300165 wait complete. 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.508+0000 7f467dd4eb80 1 -- start start 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.508+0000 7f467dd4eb80 1 Processor -- start 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.508+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.508+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.508+0000 7f467dd4eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.508+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc9585f240 con 0x55cc957df480 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.508+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc9585f3b0 con 0x55cc957e4950 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.508+0000 7f467dd4eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc9585f570 con 0x55cc957e5270 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.508+0000 7f467dd4eb80 10 client.0 _ll_register_callbacks cb 0x7f465c009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:34336/0 (socket says 172.21.15.39:34336) 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466bfff700 1 -- 172.21.15.39:0/1377081852 learned_addr learned my addr 172.21.15.39:0/1377081852 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466affd700 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466b7fe700 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466bfff700 1 -- 172.21.15.39:0/1377081852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 msgr2=0x55cc957ff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466bfff700 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957ff140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466bfff700 1 -- 172.21.15.39:0/1377081852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 msgr2=0x55cc957fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466bfff700 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957fec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466bfff700 1 -- 172.21.15.39:0/1377081852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c0692b0 con 0x55cc957e5270 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f466bfff700 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957ff6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46640994a0 tx=0x7f466400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4362398cbe161dd server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f464affd700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f464affd700 1 -- 172.21.15.39:0/1377081852 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664099970 con 0x55cc957e5270 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f464affd700 1 -- 172.21.15.39:0/1377081852 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f466400dd60 con 0x55cc957e5270 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.509+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1377081852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f466000dc60 con 0x55cc957e5270 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f464affd700 1 -- 172.21.15.39:0/1377081852 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f466409d170 con 0x55cc957e5270 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f467dd4eb80 10 client.25465 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1377081852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cc955b4c70 con 0x55cc957e5270 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f464affd700 1 -- 172.21.15.39:0/1377081852 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f466407c080 con 0x55cc957e5270 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f464affd700 1 -- 172.21.15.39:0/1377081852 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4664098a00 con 0x55cc957e5270 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f467dd4eb80 10 client.25465 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f467dd4eb80 10 client.25465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f467dd4eb80 20 client.25465 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55cc95860d30 0x55cc95863150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f467dd4eb80 4 client.25465 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1377081852 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc95727e60 con 0x55cc95860d30 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.510+0000 7f466b7fe700 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55cc95860d30 0x55cc95863150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.511+0000 7f464affd700 10 client.25465 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.511+0000 7f466b7fe700 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55cc95860d30 0x55cc95863150 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa5826ca5994dbf server_cookie=f7d7a988801bab51 in_seq=0 out_seq=0 2024-05-24T11:44:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f464affd700 1 -- 172.21.15.39:0/1377081852 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc95727e60 con 0x55cc95860d30 2024-05-24T11:44:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f464affd700 10 client.25465 handle_command_reply: tid=0 2024-05-24T11:44:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f467dd4eb80 1 client.25465 shutdown 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1377081852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 msgr2=0x55cc957ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957ff6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46640994a0 tx=0x7f466400dcc0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1377081852 shutdown_connections 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc957e5270 0x55cc957ff6a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55cc95860d30 0x55cc95863150 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cc957df480 0x55cc957fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f467dd4eb80 1 --2- 172.21.15.39:0/1377081852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cc957e4950 0x55cc957ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1377081852 >> 172.21.15.39:0/1377081852 conn(0x55cc9573d1b0 msgr2=0x55cc957dd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.564+0000 7f464affd700 0 client.25465 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.565+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1377081852 shutdown_connections 2024-05-24T11:44:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.565+0000 7f467dd4eb80 1 -- 172.21.15.39:0/1377081852 wait complete. 2024-05-24T11:44:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.565+0000 7f467dd4eb80 20 client.25465 trim_cache size 0 max 16384 2024-05-24T11:44:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:44:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:44:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:44:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:44:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 -- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46540a1350 msgr2=0x7f46540a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46540a1350 0x7f46540a3810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f465c006010 tx=0x7f465c00b540 comp rx=0 tx=0).stop 2024-05-24T11:44:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 -- 172.21.15.39:0/2345517606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c105f20 msgr2=0x7f466c19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c105f20 0x7f466c19aa10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4664007ee0 tx=0x7f466400d3b0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 -- 172.21.15.39:0/2345517606 shutdown_connections 2024-05-24T11:44:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f46540a1350 0x7f46540a3810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c105f20 0x7f466c19aa10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c100280 0x7f466c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 --2- 172.21.15.39:0/2345517606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f466c100c40 0x7f466c10a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 -- 172.21.15.39:0/2345517606 >> 172.21.15.39:0/2345517606 conn(0x7f466c076bc0 msgr2=0x7f466c10b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 -- 172.21.15.39:0/2345517606 shutdown_connections 2024-05-24T11:44:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:16.566+0000 7f4671b93700 1 -- 172.21.15.39:0/2345517606 wait complete. 2024-05-24T11:44:18.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:18 smithi039 ceph-mon[42919]: pgmap v185: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 17 MiB/s wr, 1.06k op/s 2024-05-24T11:44:18.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:18 smithi053 ceph-mon[47448]: pgmap v185: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 17 MiB/s wr, 1.06k op/s 2024-05-24T11:44:18.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:18 smithi137 ceph-mon[47907]: pgmap v185: 97 pgs: 97 active+clean; 4.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 17 MiB/s wr, 1.06k op/s 2024-05-24T11:44:19.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 295.0 s, 1376.6 tps, lat 72.507 ms stddev 20.743 2024-05-24T11:44:20.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:20 smithi039 ceph-mon[42919]: pgmap v186: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2024-05-24T11:44:20.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:20 smithi053 ceph-mon[47448]: pgmap v186: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2024-05-24T11:44:20.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:20 smithi137 ceph-mon[47907]: pgmap v186: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2024-05-24T11:44:21.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:21 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:44:21.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:21 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:44:21.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:21 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:44:22.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:22 smithi053 ceph-mon[47448]: pgmap v187: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-05-24T11:44:22.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:22 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:44:22.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:22 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:44:22.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:22 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:44:22.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:22 smithi137 ceph-mon[47907]: pgmap v187: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-05-24T11:44:22.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:22 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:44:22.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:22 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:44:22.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:22 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:44:22.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:22 smithi039 ceph-mon[42919]: pgmap v187: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-05-24T11:44:22.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:22 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:44:22.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:22 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:44:22.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:22 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:44:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 300.0 s, 1282.2 tps, lat 77.880 ms stddev 24.201 2024-05-24T11:44:24.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:24 smithi053 ceph-mon[47448]: pgmap v188: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-05-24T11:44:24.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:24 smithi137 ceph-mon[47907]: pgmap v188: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-05-24T11:44:24.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:24 smithi039 ceph-mon[42919]: pgmap v188: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-05-24T11:44:26.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:26 smithi053 ceph-mon[47448]: pgmap v189: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 24 MiB/s wr, 1.65k op/s 2024-05-24T11:44:26.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:26 smithi137 ceph-mon[47907]: pgmap v189: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 24 MiB/s wr, 1.65k op/s 2024-05-24T11:44:26.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:26 smithi039 ceph-mon[42919]: pgmap v189: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 24 MiB/s wr, 1.65k op/s 2024-05-24T11:44:28.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:28 smithi053 ceph-mon[47448]: pgmap v190: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 16 MiB/s wr, 1.13k op/s 2024-05-24T11:44:28.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:28 smithi137 ceph-mon[47907]: pgmap v190: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 16 MiB/s wr, 1.13k op/s 2024-05-24T11:44:28.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:28 smithi039 ceph-mon[42919]: pgmap v190: 97 pgs: 97 active+clean; 4.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 16 MiB/s wr, 1.13k op/s 2024-05-24T11:44:29.118 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 305.0 s, 1261.8 tps, lat 79.172 ms stddev 22.773 2024-05-24T11:44:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.739+0000 7f9666bd0700 1 -- 172.21.15.39:0/2568059150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 msgr2=0x7f966010cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.739+0000 7f9666bd0700 1 --2- 172.21.15.39:0/2568059150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 0x7f966010cc30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f965800b0a0 tx=0x7f965800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.739+0000 7f9666bd0700 1 -- 172.21.15.39:0/2568059150 shutdown_connections 2024-05-24T11:44:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.739+0000 7f9666bd0700 1 --2- 172.21.15.39:0/2568059150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 0x7f966010cc30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.739+0000 7f9666bd0700 1 --2- 172.21.15.39:0/2568059150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9660105f20 0x7f9660106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.739+0000 7f9666bd0700 1 --2- 172.21.15.39:0/2568059150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9660100280 0x7f9660100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.739+0000 7f9666bd0700 1 -- 172.21.15.39:0/2568059150 >> 172.21.15.39:0/2568059150 conn(0x7f9660076bc0 msgr2=0x7f9660076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.740+0000 7f9666bd0700 1 -- 172.21.15.39:0/2568059150 shutdown_connections 2024-05-24T11:44:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.740+0000 7f9666bd0700 1 -- 172.21.15.39:0/2568059150 wait complete. 2024-05-24T11:44:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.741+0000 7f9666bd0700 1 Processor -- start 2024-05-24T11:44:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.741+0000 7f9666bd0700 1 -- start start 2024-05-24T11:44:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.742+0000 7f9666bd0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9660100280 0x7f9660197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.742+0000 7f9666bd0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 0x7f96601977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.742+0000 7f9666bd0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9660105f20 0x7f966019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.742+0000 7f9666bd0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019c090 con 0x7f9660100280 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.742+0000 7f9666bd0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019c200 con 0x7f9660105f20 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.742+0000 7f9666bd0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019c3c0 con 0x7f9660100c40 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.742+0000 7f965ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 0x7f96601977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.742+0000 7f965ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 0x7f96601977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:33436/0 (socket says 172.21.15.39:33436) 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.742+0000 7f965ffff700 1 -- 172.21.15.39:0/713539445 learned_addr learned my addr 172.21.15.39:0/713539445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f965ffff700 1 -- 172.21.15.39:0/713539445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9660105f20 msgr2=0x7f966019bb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f965ffff700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9660105f20 0x7f966019bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f965ffff700 1 -- 172.21.15.39:0/713539445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9660100280 msgr2=0x7f9660197270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f965ffff700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9660100280 0x7f9660197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f965ffff700 1 -- 172.21.15.39:0/713539445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9658009d00 con 0x7f9660100c40 2024-05-24T11:44:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f965ffff700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 0x7f96601977b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f964c00c9b0 tx=0x7f964c00cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f965dffb700 1 -- 172.21.15.39:0/713539445 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c007b00 con 0x7f9660100c40 2024-05-24T11:44:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f9666bd0700 1 -- 172.21.15.39:0/713539445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966019c5c0 con 0x7f9660100c40 2024-05-24T11:44:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f965dffb700 1 -- 172.21.15.39:0/713539445 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f964c007c60 con 0x7f9660100c40 2024-05-24T11:44:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f965dffb700 1 -- 172.21.15.39:0/713539445 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c018660 con 0x7f9660100c40 2024-05-24T11:44:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.743+0000 7f9666bd0700 1 -- 172.21.15.39:0/713539445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96601a1df0 con 0x7f9660100c40 2024-05-24T11:44:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.746+0000 7f965dffb700 1 -- 172.21.15.39:0/713539445 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f964c0187c0 con 0x7f9660100c40 2024-05-24T11:44:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.747+0000 7f965dffb700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f96500a1290 0x7f96500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.747+0000 7f965dffb700 1 -- 172.21.15.39:0/713539445 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f964c0cd130 con 0x7f9660100c40 2024-05-24T11:44:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.748+0000 7f9672d8bb80 1 Processor -- start 2024-05-24T11:44:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.748+0000 7f966496c700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f96500a1290 0x7f96500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.748+0000 7f9672d8bb80 1 -- start start 2024-05-24T11:44:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.748+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 0x562e71cbed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.748+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cb8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.748+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cb98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.748+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71c100f0 con 0x562e71cbf270 2024-05-24T11:44:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.748+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71c0dab0 con 0x562e71cb9480 2024-05-24T11:44:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.748+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71c093a0 con 0x562e71cbe950 2024-05-24T11:44:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f966516d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cb98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f966516d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cb98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:57476/0 (socket says 172.21.15.39:57476) 2024-05-24T11:44:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f966516d700 1 -- 172.21.15.39:0/1455294474 learned_addr learned my addr 172.21.15.39:0/1455294474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f965ffff700 1 --2- 172.21.15.39:0/1455294474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cb8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f966516d700 1 -- 172.21.15.39:0/1455294474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 msgr2=0x562e71cbed30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:44:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f966516d700 1 --2- 172.21.15.39:0/1455294474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 0x562e71cbed30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f966516d700 1 -- 172.21.15.39:0/1455294474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 msgr2=0x562e71cb8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f966516d700 1 --2- 172.21.15.39:0/1455294474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cb8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f966516d700 1 -- 172.21.15.39:0/1455294474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e71a8a350 con 0x562e71cb9480 2024-05-24T11:44:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f965ffff700 1 --2- 172.21.15.39:0/1455294474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cb8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:44:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f966516d700 1 --2- 172.21.15.39:0/1455294474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cb98a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f965800b070 tx=0x7f9658009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e75615476bc97c99 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.749+0000 7f964affd700 1 -- 172.21.15.39:0/1455294474 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800e040 con 0x562e71cb9480 2024-05-24T11:44:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.750+0000 7f966496c700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f96500a1290 0x7f96500a3750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9654005950 tx=0x7f96540058e0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.750+0000 7f964affd700 1 -- 172.21.15.39:0/1455294474 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9658007d00 con 0x562e71cb9480 2024-05-24T11:44:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.750+0000 7f964affd700 1 -- 172.21.15.39:0/1455294474 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658007700 con 0x562e71cb9480 2024-05-24T11:44:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.750+0000 7f9672d8bb80 1 -- 172.21.15.39:0/1455294474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 msgr2=0x562e71cb98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.750+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/1455294474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cb98a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f965800b070 tx=0x7f9658009bc0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.751+0000 7f9672d8bb80 1 -- 172.21.15.39:0/1455294474 shutdown_connections 2024-05-24T11:44:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.751+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/1455294474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 0x562e71cbed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.751+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/1455294474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cb98a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.751+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/1455294474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cb8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.751+0000 7f9672d8bb80 1 -- 172.21.15.39:0/1455294474 >> 172.21.15.39:0/1455294474 conn(0x562e71c171b0 msgr2=0x562e71cb3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.751+0000 7f9672d8bb80 1 -- 172.21.15.39:0/1455294474 shutdown_connections 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.751+0000 7f9672d8bb80 1 -- 172.21.15.39:0/1455294474 wait complete. 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.751+0000 7f9672d8bb80 1 -- start start 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.751+0000 7f9672d8bb80 1 Processor -- start 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.752+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cb9480 0x562e71cc51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.752+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbe950 0x562e71cc25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.752+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbf270 0x562e71cc2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.752+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71ba5500 con 0x562e71cbe950 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.752+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71ba51a0 con 0x562e71cbf270 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.752+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71a8c640 con 0x562e71cb9480 2024-05-24T11:44:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.752+0000 7f9672d8bb80 10 client.0 _ll_register_callbacks cb 0x562e71baead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.753+0000 7f966516d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbf270 0x562e71cc2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.753+0000 7f966516d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbf270 0x562e71cc2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:57482/0 (socket says 172.21.15.39:57482) 2024-05-24T11:44:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.753+0000 7f966516d700 1 -- 172.21.15.39:0/724931442 learned_addr learned my addr 172.21.15.39:0/724931442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.753+0000 7f965ffff700 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbe950 0x562e71cc25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f966516d700 1 -- 172.21.15.39:0/724931442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cb9480 msgr2=0x562e71cc51e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f966516d700 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cb9480 0x562e71cc51e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f966516d700 1 -- 172.21.15.39:0/724931442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbe950 msgr2=0x562e71cc25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f966516d700 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbe950 0x562e71cc25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f966516d700 1 -- 172.21.15.39:0/724931442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9660069470 con 0x562e71cbf270 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f966516d700 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbf270 0x562e71cc2be0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9658003d80 tx=0x7f96580093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d96e414845cdf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f963ffff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f963ffff700 1 -- 172.21.15.39:0/724931442 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800e040 con 0x562e71cbf270 2024-05-24T11:44:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f963ffff700 1 -- 172.21.15.39:0/724931442 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f96580076b0 con 0x562e71cbf270 2024-05-24T11:44:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.754+0000 7f9672d8bb80 1 -- 172.21.15.39:0/724931442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e71b62210 con 0x562e71cbf270 2024-05-24T11:44:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.755+0000 7f963ffff700 1 -- 172.21.15.39:0/724931442 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9658007870 con 0x562e71cbf270 2024-05-24T11:44:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.755+0000 7f9672d8bb80 10 client.25568 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.755+0000 7f9672d8bb80 1 -- 172.21.15.39:0/724931442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562e71a36b00 con 0x562e71cbf270 2024-05-24T11:44:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.755+0000 7f963ffff700 1 -- 172.21.15.39:0/724931442 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562e71b62210 con 0x562e71cbf270 2024-05-24T11:44:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.755+0000 7f963ffff700 1 -- 172.21.15.39:0/724931442 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9658024070 con 0x562e71cbf270 2024-05-24T11:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.756+0000 7f9672d8bb80 10 client.25568 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.756+0000 7f9672d8bb80 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.756+0000 7f9672d8bb80 20 client.25568 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.756+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e71cd2610 0x562e71cd4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.756+0000 7f9672d8bb80 4 client.25568 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.756+0000 7f9672d8bb80 1 -- 172.21.15.39:0/724931442 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e71c887b0 con 0x562e71cd2610 2024-05-24T11:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.756+0000 7f966496c700 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e71cd2610 0x562e71cd4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.756+0000 7f963ffff700 10 client.25568 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.756+0000 7f966496c700 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e71cd2610 0x562e71cd4a30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b890331f23dcff0 server_cookie=eb818584dac76b27 in_seq=0 out_seq=0 2024-05-24T11:44:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.757+0000 7f963ffff700 1 -- 172.21.15.39:0/724931442 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e71c887b0 con 0x562e71cd2610 2024-05-24T11:44:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.757+0000 7f963ffff700 10 client.25568 handle_command_reply: tid=0 2024-05-24T11:44:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.757+0000 7f9672d8bb80 1 client.25568 shutdown 2024-05-24T11:44:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.757+0000 7f9672d8bb80 1 -- 172.21.15.39:0/724931442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbf270 msgr2=0x562e71cc2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.757+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbf270 0x562e71cc2be0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9658003d80 tx=0x7f96580093b0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f9672d8bb80 1 -- 172.21.15.39:0/724931442 shutdown_connections 2024-05-24T11:44:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cb9480 0x562e71cc51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e71cd2610 0x562e71cd4a30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbf270 0x562e71cc2be0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f963ffff700 0 client.25568 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/724931442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbe950 0x562e71cc25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f9672d8bb80 1 -- 172.21.15.39:0/724931442 >> 172.21.15.39:0/724931442 conn(0x562e71c171b0 msgr2=0x562e71cbd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f9672d8bb80 1 -- 172.21.15.39:0/724931442 shutdown_connections 2024-05-24T11:44:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f9672d8bb80 1 -- 172.21.15.39:0/724931442 wait complete. 2024-05-24T11:44:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.758+0000 7f9672d8bb80 20 client.25568 trim_cache size 0 max 16384 2024-05-24T11:44:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.783+0000 7f9672d8bb80 1 Processor -- start 2024-05-24T11:44:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.783+0000 7f9672d8bb80 1 -- start start 2024-05-24T11:44:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.783+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cc25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.783+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 0x562e71cb3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.783+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cb9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.783+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71cb4330 con 0x562e71cbf270 2024-05-24T11:44:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.783+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71cc31c0 con 0x562e71cb9480 2024-05-24T11:44:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.783+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71cb4af0 con 0x562e71cbe950 2024-05-24T11:44:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f965ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 0x562e71cb3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f965ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 0x562e71cb3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:33466/0 (socket says 172.21.15.39:33466) 2024-05-24T11:44:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f965ffff700 1 -- 172.21.15.39:0/2102066208 learned_addr learned my addr 172.21.15.39:0/2102066208 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f966496c700 1 --2- 172.21.15.39:0/2102066208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cc25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f965ffff700 1 -- 172.21.15.39:0/2102066208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 msgr2=0x562e71cc25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f965ffff700 1 --2- 172.21.15.39:0/2102066208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cc25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f965ffff700 1 -- 172.21.15.39:0/2102066208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 msgr2=0x562e71cb9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f965ffff700 1 --2- 172.21.15.39:0/2102066208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cb9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f965ffff700 1 -- 172.21.15.39:0/2102066208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e71bcf500 con 0x562e71cbe950 2024-05-24T11:44:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f965ffff700 1 --2- 172.21.15.39:0/2102066208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 0x562e71cb3910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f964c0967c0 tx=0x7f964c099d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=684f487f3905a4f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f964b7fe700 1 -- 172.21.15.39:0/2102066208 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0b6070 con 0x562e71cbe950 2024-05-24T11:44:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f964b7fe700 1 -- 172.21.15.39:0/2102066208 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f964c09a2c0 con 0x562e71cbe950 2024-05-24T11:44:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.784+0000 7f964b7fe700 1 -- 172.21.15.39:0/2102066208 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0986f0 con 0x562e71cbe950 2024-05-24T11:44:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.785+0000 7f9672d8bb80 1 -- 172.21.15.39:0/2102066208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 msgr2=0x562e71cb3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.785+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/2102066208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 0x562e71cb3910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f964c0967c0 tx=0x7f964c099d10 comp rx=0 tx=0).stop 2024-05-24T11:44:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.786+0000 7f9672d8bb80 1 -- 172.21.15.39:0/2102066208 shutdown_connections 2024-05-24T11:44:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.786+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/2102066208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cb9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.786+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/2102066208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cb9480 0x562e71cc25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.786+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/2102066208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cbe950 0x562e71cb3910 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.786+0000 7f9672d8bb80 1 -- 172.21.15.39:0/2102066208 >> 172.21.15.39:0/2102066208 conn(0x562e71c171b0 msgr2=0x562e71cd9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.786+0000 7f9672d8bb80 1 -- 172.21.15.39:0/2102066208 shutdown_connections 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.786+0000 7f9672d8bb80 1 -- 172.21.15.39:0/2102066208 wait complete. 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.786+0000 7f9672d8bb80 1 -- start start 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.787+0000 7f9672d8bb80 1 Processor -- start 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.787+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cb9480 0x562e71cca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.787+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbe950 0x562e71cbd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.787+0000 7f9672d8bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cbc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.787+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71cbdff0 con 0x562e71cbf270 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.787+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71cbe160 con 0x562e71cbe950 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.787+0000 7f9672d8bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e71cbc8c0 con 0x562e71cb9480 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.787+0000 7f9672d8bb80 10 client.0 _ll_register_callbacks cb 0x562e71baead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.788+0000 7f966516d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cbc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.788+0000 7f966516d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cbc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55330/0 (socket says 172.21.15.39:55330) 2024-05-24T11:44:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.788+0000 7f966516d700 1 -- 172.21.15.39:0/3946000777 learned_addr learned my addr 172.21.15.39:0/3946000777 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.788+0000 7f965ffff700 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbe950 0x562e71cbd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.788+0000 7f966516d700 1 -- 172.21.15.39:0/3946000777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cb9480 msgr2=0x562e71cca110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:44:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.788+0000 7f966516d700 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cb9480 0x562e71cca110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.788+0000 7f966516d700 1 -- 172.21.15.39:0/3946000777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbe950 msgr2=0x562e71cbd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.789+0000 7f966516d700 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbe950 0x562e71cbd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.789+0000 7f966516d700 1 -- 172.21.15.39:0/3946000777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96601a1df0 con 0x562e71cbf270 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.789+0000 7f965ffff700 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbe950 0x562e71cbd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.789+0000 7f966516d700 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cbc380 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9658002230 tx=0x7f965800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00809f75f6ef7bd server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.789+0000 7f963ffff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.789+0000 7f963ffff700 1 -- 172.21.15.39:0/3946000777 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800e040 con 0x562e71cbf270 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f963ffff700 1 -- 172.21.15.39:0/3946000777 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f96580052b0 con 0x562e71cbf270 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f9672d8bb80 1 -- 172.21.15.39:0/3946000777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9658007d00 con 0x562e71cbf270 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f963ffff700 1 -- 172.21.15.39:0/3946000777 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f965800b830 con 0x562e71cbf270 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f9672d8bb80 10 client.15861 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f9672d8bb80 1 -- 172.21.15.39:0/3946000777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562e71a8ec70 con 0x562e71cbf270 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f963ffff700 1 -- 172.21.15.39:0/3946000777 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9658007d00 con 0x562e71cbf270 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f963ffff700 1 -- 172.21.15.39:0/3946000777 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f965800b0d0 con 0x562e71cbf270 2024-05-24T11:44:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f9672d8bb80 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f9672d8bb80 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f9672d8bb80 20 client.15861 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e71d25430 0x562e71d25810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f9672d8bb80 4 client.15861 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:44:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.790+0000 7f9672d8bb80 1 -- 172.21.15.39:0/3946000777 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562e71c01e60 con 0x562e71d25430 2024-05-24T11:44:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.791+0000 7f966496c700 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e71d25430 0x562e71d25810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.791+0000 7f966496c700 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e71d25430 0x562e71d25810 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b1bd8ef76456b0 server_cookie=c53699f87296061d in_seq=0 out_seq=0 2024-05-24T11:44:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:29.791+0000 7f963ffff700 10 client.15861 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:30.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:30 smithi053 ceph-mon[47448]: pgmap v191: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 23 MiB/s wr, 1.74k op/s 2024-05-24T11:44:30.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:30 smithi137 ceph-mon[47907]: pgmap v191: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 23 MiB/s wr, 1.74k op/s 2024-05-24T11:44:30.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:30 smithi039 ceph-mon[42919]: pgmap v191: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 23 MiB/s wr, 1.74k op/s 2024-05-24T11:44:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f963ffff700 1 -- 172.21.15.39:0/3946000777 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562e71c01e60 con 0x562e71d25430 2024-05-24T11:44:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f963ffff700 10 client.15861 handle_command_reply: tid=0 2024-05-24T11:44:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 client.15861 shutdown 2024-05-24T11:44:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 -- 172.21.15.39:0/3946000777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 msgr2=0x562e71cbc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cbc380 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9658002230 tx=0x7f965800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 -- 172.21.15.39:0/3946000777 shutdown_connections 2024-05-24T11:44:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e71cb9480 0x562e71cca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e71d25430 0x562e71d25810 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e71cbf270 0x562e71cbc380 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 --2- 172.21.15.39:0/3946000777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e71cbe950 0x562e71cbd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 -- 172.21.15.39:0/3946000777 >> 172.21.15.39:0/3946000777 conn(0x562e71c171b0 msgr2=0x562e71cb7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f963ffff700 0 client.15861 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.133+0000 7f9672d8bb80 1 -- 172.21.15.39:0/3946000777 shutdown_connections 2024-05-24T11:44:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.134+0000 7f9672d8bb80 1 -- 172.21.15.39:0/3946000777 wait complete. 2024-05-24T11:44:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.134+0000 7f9672d8bb80 20 client.15861 trim_cache size 0 max 16384 2024-05-24T11:44:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:44:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:44:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:44:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:44:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.134+0000 7f9666bd0700 1 -- 172.21.15.39:0/713539445 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f96500a1290 msgr2=0x7f96500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.134+0000 7f9666bd0700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f96500a1290 0x7f96500a3750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9654005950 tx=0x7f96540058e0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.134+0000 7f9666bd0700 1 -- 172.21.15.39:0/713539445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 msgr2=0x7f96601977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.134+0000 7f9666bd0700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 0x7f96601977b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f964c00c9b0 tx=0x7f964c00cd70 comp rx=0 tx=0).stop 2024-05-24T11:44:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.135+0000 7f9666bd0700 1 -- 172.21.15.39:0/713539445 shutdown_connections 2024-05-24T11:44:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.135+0000 7f9666bd0700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f96500a1290 0x7f96500a3750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.135+0000 7f9666bd0700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9660105f20 0x7f966019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.135+0000 7f9666bd0700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9660100280 0x7f9660197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.135+0000 7f9666bd0700 1 --2- 172.21.15.39:0/713539445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9660100c40 0x7f96601977b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.135+0000 7f9666bd0700 1 -- 172.21.15.39:0/713539445 >> 172.21.15.39:0/713539445 conn(0x7f9660076bc0 msgr2=0x7f96600feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.135+0000 7f9666bd0700 1 -- 172.21.15.39:0/713539445 shutdown_connections 2024-05-24T11:44:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:32.135+0000 7f9666bd0700 1 -- 172.21.15.39:0/713539445 wait complete. 2024-05-24T11:44:32.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:32 smithi053 ceph-mon[47448]: pgmap v192: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:44:32.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:32 smithi137 ceph-mon[47907]: pgmap v192: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:44:32.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:32 smithi039 ceph-mon[42919]: pgmap v192: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:44:34.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 310.0 s, 1239.2 tps, lat 80.763 ms stddev 23.926 2024-05-24T11:44:34.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:34 smithi053 ceph-mon[47448]: pgmap v193: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:44:34.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:34 smithi137 ceph-mon[47907]: pgmap v193: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:44:34.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:34 smithi039 ceph-mon[42919]: pgmap v193: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:44:36.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:36 smithi053 ceph-mon[47448]: pgmap v194: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 21 MiB/s wr, 1.64k op/s 2024-05-24T11:44:36.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:36 smithi137 ceph-mon[47907]: pgmap v194: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 21 MiB/s wr, 1.64k op/s 2024-05-24T11:44:36.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:36 smithi039 ceph-mon[42919]: pgmap v194: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 21 MiB/s wr, 1.64k op/s 2024-05-24T11:44:38.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:38 smithi053 ceph-mon[47448]: pgmap v195: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-05-24T11:44:38.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:38 smithi137 ceph-mon[47907]: pgmap v195: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-05-24T11:44:38.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:38 smithi039 ceph-mon[42919]: pgmap v195: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-05-24T11:44:39.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 315.0 s, 1291.2 tps, lat 77.449 ms stddev 23.021 2024-05-24T11:44:40.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:40 smithi053 ceph-mon[47448]: pgmap v196: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-24T11:44:40.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:40 smithi137 ceph-mon[47907]: pgmap v196: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-24T11:44:40.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:40 smithi039 ceph-mon[42919]: pgmap v196: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-05-24T11:44:42.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:42 smithi053 ceph-mon[47448]: pgmap v197: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-24T11:44:42.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:42 smithi137 ceph-mon[47907]: pgmap v197: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-24T11:44:42.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:42 smithi039 ceph-mon[42919]: pgmap v197: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-24T11:44:44.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 320.0 s, 1352.4 tps, lat 74.114 ms stddev 22.402 2024-05-24T11:44:44.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:44 smithi053 ceph-mon[47448]: pgmap v198: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-24T11:44:44.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:44 smithi137 ceph-mon[47907]: pgmap v198: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-24T11:44:44.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:44 smithi039 ceph-mon[42919]: pgmap v198: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-24T11:44:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.308+0000 7f1b97c72700 1 -- 172.21.15.39:0/2271224583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9006d290 msgr2=0x7f1b9011a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.308+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2271224583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9006d290 0x7f1b9011a100 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009990 tx=0x7f1b84009ca0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.308+0000 7f1b97c72700 1 -- 172.21.15.39:0/2271224583 shutdown_connections 2024-05-24T11:44:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.308+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2271224583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b9011a640 0x7f1b9011ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.308+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2271224583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9006c970 0x7f1b9006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.308+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2271224583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9006d290 0x7f1b9011a100 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.308+0000 7f1b97c72700 1 -- 172.21.15.39:0/2271224583 >> 172.21.15.39:0/2271224583 conn(0x7f1b9006c020 msgr2=0x7f1b90109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.308+0000 7f1b97c72700 1 -- 172.21.15.39:0/2271224583 shutdown_connections 2024-05-24T11:44:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.308+0000 7f1b97c72700 1 -- 172.21.15.39:0/2271224583 wait complete. 2024-05-24T11:44:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b97c72700 1 Processor -- start 2024-05-24T11:44:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b97c72700 1 -- start start 2024-05-24T11:44:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b97c72700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9006c970 0x7f1b901a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b97c72700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9006d290 0x7f1b901a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b97c72700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b9011a640 0x7f1b901aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b97c72700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b901ab060 con 0x7f1b9006d290 2024-05-24T11:44:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b97c72700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b901ab1d0 con 0x7f1b9006c970 2024-05-24T11:44:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b97c72700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b901ab390 con 0x7f1b9011a640 2024-05-24T11:44:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b9620f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b9011a640 0x7f1b901aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.309+0000 7f1b95a0e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9006c970 0x7f1b901a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b9620f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b9011a640 0x7f1b901aaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:50400/0 (socket says 172.21.15.39:50400) 2024-05-24T11:44:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b9620f700 1 -- 172.21.15.39:0/2655407507 learned_addr learned my addr 172.21.15.39:0/2655407507 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b9520d700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9006d290 0x7f1b901a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b95a0e700 1 -- 172.21.15.39:0/2655407507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b9011a640 msgr2=0x7f1b901aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b9011a640 0x7f1b901aaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b95a0e700 1 -- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9006d290 msgr2=0x7f1b901a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9006d290 0x7f1b901a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b95a0e700 1 -- 172.21.15.39:0/2655407507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840095f0 con 0x7f1b9006c970 2024-05-24T11:44:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b9620f700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b9011a640 0x7f1b901aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:44:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b9520d700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9006d290 0x7f1b901a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:44:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9006c970 0x7f1b901a6190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000ef10 tx=0x7f1b8000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b8effd700 1 -- 172.21.15.39:0/2655407507 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8000ca70 con 0x7f1b9006c970 2024-05-24T11:44:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b8effd700 1 -- 172.21.15.39:0/2655407507 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b8000f040 con 0x7f1b9006c970 2024-05-24T11:44:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b8effd700 1 -- 172.21.15.39:0/2655407507 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80014480 con 0x7f1b9006c970 2024-05-24T11:44:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.310+0000 7f1b97c72700 1 -- 172.21.15.39:0/2655407507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b901ab610 con 0x7f1b9006c970 2024-05-24T11:44:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.311+0000 7f1b97c72700 1 -- 172.21.15.39:0/2655407507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b901b2eb0 con 0x7f1b9006c970 2024-05-24T11:44:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.315+0000 7f1b8effd700 1 -- 172.21.15.39:0/2655407507 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1b800145e0 con 0x7f1b9006c970 2024-05-24T11:44:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.316+0000 7f1ba3e2db80 1 Processor -- start 2024-05-24T11:44:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.316+0000 7f1b8effd700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1b780a1350 0x7f1b780a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.316+0000 7f1b8effd700 1 -- 172.21.15.39:0/2655407507 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1b800ccfc0 con 0x7f1b9006c970 2024-05-24T11:44:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.317+0000 7f1b9620f700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1b780a1350 0x7f1b780a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.317+0000 7f1ba3e2db80 1 -- start start 2024-05-24T11:44:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.317+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816a950 0x55fe0816ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.317+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816b270 0x55fe08164f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.317+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 0x55fe081658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.317+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe080bc0f0 con 0x55fe08165480 2024-05-24T11:44:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.317+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe080b9ab0 con 0x55fe0816b270 2024-05-24T11:44:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.317+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe080b53a0 con 0x55fe0816a950 2024-05-24T11:44:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b95a0e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816a950 0x55fe0816ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b9620f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 0x55fe081658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b95a0e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816a950 0x55fe0816ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:50402/0 (socket says 172.21.15.39:50402) 2024-05-24T11:44:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b95a0e700 1 -- 172.21.15.39:0/3006645840 learned_addr learned my addr 172.21.15.39:0/3006645840 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b9520d700 1 --2- 172.21.15.39:0/3006645840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816b270 0x55fe08164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b9520d700 1 -- 172.21.15.39:0/3006645840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816a950 msgr2=0x55fe0816ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b9520d700 1 --2- 172.21.15.39:0/3006645840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816a950 0x55fe0816ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b9520d700 1 -- 172.21.15.39:0/3006645840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 msgr2=0x55fe081658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b9520d700 1 --2- 172.21.15.39:0/3006645840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 0x55fe081658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b9520d700 1 -- 172.21.15.39:0/3006645840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe07f36350 con 0x55fe0816b270 2024-05-24T11:44:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/3006645840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816a950 0x55fe0816ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:44:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.318+0000 7f1b9520d700 1 --2- 172.21.15.39:0/3006645840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816b270 0x55fe08164f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b84005910 tx=0x7f1b84005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256a184c1b1a93a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.319+0000 7f1b9620f700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1b780a1350 0x7f1b780a3810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b88005d90 tx=0x7f1b8800b3e0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.319+0000 7f1b77fff700 1 -- 172.21.15.39:0/3006645840 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840056a0 con 0x55fe0816b270 2024-05-24T11:44:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.319+0000 7f1b77fff700 1 -- 172.21.15.39:0/3006645840 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b8401e040 con 0x55fe0816b270 2024-05-24T11:44:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.319+0000 7f1b77fff700 1 -- 172.21.15.39:0/3006645840 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840049b0 con 0x55fe0816b270 2024-05-24T11:44:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3006645840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816b270 msgr2=0x55fe08164f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3006645840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816b270 0x55fe08164f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b84005910 tx=0x7f1b84005c40 comp rx=0 tx=0).stop 2024-05-24T11:44:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3006645840 shutdown_connections 2024-05-24T11:44:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3006645840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816a950 0x55fe0816ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3006645840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 0x55fe081658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3006645840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816b270 0x55fe08164f40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3006645840 >> 172.21.15.39:0/3006645840 conn(0x55fe080c31b0 msgr2=0x55fe0815f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3006645840 shutdown_connections 2024-05-24T11:44:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3006645840 wait complete. 2024-05-24T11:44:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 -- start start 2024-05-24T11:44:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.320+0000 7f1ba3e2db80 1 Processor -- start 2024-05-24T11:44:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe08165480 0x55fe0816e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 0x55fe0816eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816b270 0x55fe0816f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe08051500 con 0x55fe0816b270 2024-05-24T11:44:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe080511a0 con 0x55fe0816a950 2024-05-24T11:44:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe07f38640 con 0x55fe08165480 2024-05-24T11:44:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1ba3e2db80 10 client.0 _ll_register_callbacks cb 0x55fe0805aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b9620f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816b270 0x55fe0816f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b9520d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 0x55fe0816eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b9620f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816b270 0x55fe0816f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46454/0 (socket says 172.21.15.39:46454) 2024-05-24T11:44:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b9620f700 1 -- 172.21.15.39:0/3645809838 learned_addr learned my addr 172.21.15.39:0/3645809838 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe08165480 0x55fe0816e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b9620f700 1 -- 172.21.15.39:0/3645809838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe08165480 msgr2=0x55fe0816e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b9620f700 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe08165480 0x55fe0816e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b9620f700 1 -- 172.21.15.39:0/3645809838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 msgr2=0x55fe0816eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b9620f700 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 0x55fe0816eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.321+0000 7f1b9620f700 1 -- 172.21.15.39:0/3645809838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b901b2eb0 con 0x55fe0816b270 2024-05-24T11:44:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.322+0000 7f1b9620f700 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816b270 0x55fe0816f090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1b8801f960 tx=0x7f1b880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7253438fa7a5b37e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.322+0000 7f1b757fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:44:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.322+0000 7f1b757fa700 1 -- 172.21.15.39:0/3645809838 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b880115f0 con 0x55fe0816b270 2024-05-24T11:44:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.322+0000 7f1b757fa700 1 -- 172.21.15.39:0/3645809838 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1b88017070 con 0x55fe0816b270 2024-05-24T11:44:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.322+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3645809838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe0800e210 con 0x55fe0816b270 2024-05-24T11:44:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.322+0000 7f1b757fa700 1 -- 172.21.15.39:0/3645809838 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b88011750 con 0x55fe0816b270 2024-05-24T11:44:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.322+0000 7f1ba3e2db80 10 client.15873 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.322+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3645809838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f1b9011d3e0 con 0x55fe0816b270 2024-05-24T11:44:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.323+0000 7f1b757fa700 1 -- 172.21.15.39:0/3645809838 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe0800e210 con 0x55fe0816b270 2024-05-24T11:44:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.324+0000 7f1b757fa700 1 -- 172.21.15.39:0/3645809838 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1b880118b0 con 0x55fe0816b270 2024-05-24T11:44:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.327+0000 7f1ba3e2db80 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.327+0000 7f1ba3e2db80 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.327+0000 7f1ba3e2db80 20 client.15873 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.327+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0817ec60 0x55fe08181080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.327+0000 7f1ba3e2db80 4 client.15873 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:44:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.327+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3645809838 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe081347b0 con 0x55fe0817ec60 2024-05-24T11:44:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.327+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0817ec60 0x55fe08181080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.327+0000 7f1b757fa700 10 client.15873 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.327+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0817ec60 0x55fe08181080 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863e10589062d0 server_cookie=d3f94bb9d7d37bb0 in_seq=0 out_seq=0 2024-05-24T11:44:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1b757fa700 1 -- 172.21.15.39:0/3645809838 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe081347b0 con 0x55fe0817ec60 2024-05-24T11:44:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1b757fa700 10 client.15873 handle_command_reply: tid=0 2024-05-24T11:44:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1ba3e2db80 1 client.15873 shutdown 2024-05-24T11:44:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3645809838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816b270 msgr2=0x55fe0816f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816b270 0x55fe0816f090 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1b8801f960 tx=0x7f1b880047c0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3645809838 shutdown_connections 2024-05-24T11:44:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe08165480 0x55fe0816e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0817ec60 0x55fe08181080 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816b270 0x55fe0816f090 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/3645809838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 0x55fe0816eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3645809838 >> 172.21.15.39:0/3645809838 conn(0x55fe080c31b0 msgr2=0x55fe081694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.328+0000 7f1b757fa700 0 client.15873 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.329+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3645809838 shutdown_connections 2024-05-24T11:44:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.329+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/3645809838 wait complete. 2024-05-24T11:44:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.329+0000 7f1ba3e2db80 20 client.15873 trim_cache size 0 max 16384 2024-05-24T11:44:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.353+0000 7f1ba3e2db80 1 Processor -- start 2024-05-24T11:44:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1ba3e2db80 1 -- start start 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 0x55fe08181050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 0x55fe0815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe08165860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0816c660 con 0x55fe08165480 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe08165da0 con 0x55fe0816a950 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe08186290 con 0x55fe0816b270 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1b9620f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe08165860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1b95a0e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 0x55fe08181050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1b9620f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe08165860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:50424/0 (socket says 172.21.15.39:50424) 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1b9620f700 1 -- 172.21.15.39:0/2468020654 learned_addr learned my addr 172.21.15.39:0/2468020654 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.354+0000 7f1b9520d700 1 --2- 172.21.15.39:0/2468020654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 0x55fe0815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.355+0000 7f1b9620f700 1 -- 172.21.15.39:0/2468020654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 msgr2=0x55fe0815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.355+0000 7f1b9620f700 1 --2- 172.21.15.39:0/2468020654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 0x55fe0815f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.355+0000 7f1b9620f700 1 -- 172.21.15.39:0/2468020654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 msgr2=0x55fe08181050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.355+0000 7f1b9620f700 1 --2- 172.21.15.39:0/2468020654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 0x55fe08181050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.355+0000 7f1b9620f700 1 -- 172.21.15.39:0/2468020654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe07ee2b00 con 0x55fe0816b270 2024-05-24T11:44:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.355+0000 7f1b9620f700 1 --2- 172.21.15.39:0/2468020654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe08165860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1b880058c0 tx=0x7f1b88010bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f0b7426d5bfe9c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.355+0000 7f1b8cda5700 1 -- 172.21.15.39:0/2468020654 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88010ee0 con 0x55fe0816b270 2024-05-24T11:44:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.356+0000 7f1b8cda5700 1 -- 172.21.15.39:0/2468020654 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b88011e90 con 0x55fe0816b270 2024-05-24T11:44:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.356+0000 7f1b8cda5700 1 -- 172.21.15.39:0/2468020654 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88028a80 con 0x55fe0816b270 2024-05-24T11:44:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/2468020654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 msgr2=0x55fe08165860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/2468020654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe08165860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1b880058c0 tx=0x7f1b88010bc0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/2468020654 shutdown_connections 2024-05-24T11:44:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/2468020654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe08165860 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/2468020654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe08165480 0x55fe08181050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/2468020654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0816a950 0x55fe0815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/2468020654 >> 172.21.15.39:0/2468020654 conn(0x55fe080c31b0 msgr2=0x55fe0815be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/2468020654 shutdown_connections 2024-05-24T11:44:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/2468020654 wait complete. 2024-05-24T11:44:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 -- start start 2024-05-24T11:44:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.357+0000 7f1ba3e2db80 1 Processor -- start 2024-05-24T11:44:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe08165480 0x55fe08175de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816a950 0x55fe08171bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1ba3e2db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe081720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe08172630 con 0x55fe0816a950 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe08172770 con 0x55fe08165480 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1ba3e2db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe08172900 con 0x55fe0816b270 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1ba3e2db80 10 client.0 _ll_register_callbacks cb 0x55fe0805aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1b95a0e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe08165480 0x55fe08175de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1b95a0e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe08165480 0x55fe08175de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:33786/0 (socket says 172.21.15.39:33786) 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1b95a0e700 1 -- 172.21.15.39:0/392668792 learned_addr learned my addr 172.21.15.39:0/392668792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1b9520d700 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816a950 0x55fe08171bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.358+0000 7f1b9620f700 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe081720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b95a0e700 1 -- 172.21.15.39:0/392668792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 msgr2=0x55fe081720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe081720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b95a0e700 1 -- 172.21.15.39:0/392668792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816a950 msgr2=0x55fe08171bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816a950 0x55fe08171bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b95a0e700 1 -- 172.21.15.39:0/392668792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9011d3e0 con 0x55fe08165480 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b95a0e700 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe08165480 0x55fe08175de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b800cad50 tx=0x7f1b800cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ad213ade0fe296 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b757fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b757fa700 1 -- 172.21.15.39:0/392668792 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b800caea0 con 0x55fe08165480 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/392668792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b8401e040 con 0x55fe08165480 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b757fa700 1 -- 172.21.15.39:0/392668792 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1b80084070 con 0x55fe08165480 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b757fa700 1 -- 172.21.15.39:0/392668792 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b8000cd60 con 0x55fe08165480 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1ba3e2db80 10 client.25589 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/392668792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fe0807b500 con 0x55fe08165480 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.359+0000 7f1b757fa700 1 -- 172.21.15.39:0/392668792 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b800a6080 con 0x55fe08165480 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1b757fa700 1 -- 172.21.15.39:0/392668792 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1b80099600 con 0x55fe08165480 2024-05-24T11:44:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1ba3e2db80 10 client.25589 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1ba3e2db80 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1ba3e2db80 20 client.25589 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe081e6630 0x55fe081e8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1ba3e2db80 4 client.25589 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:44:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/392668792 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe080ade60 con 0x55fe081e6630 2024-05-24T11:44:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1b9520d700 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe081e6630 0x55fe081e8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1b9520d700 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe081e6630 0x55fe081e8a50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1993c46482368e70 server_cookie=92e9ae5b8bc79b30 in_seq=0 out_seq=0 2024-05-24T11:44:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.360+0000 7f1b757fa700 10 client.25589 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1b757fa700 1 -- 172.21.15.39:0/392668792 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe080ade60 con 0x55fe081e6630 2024-05-24T11:44:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1b757fa700 10 client.25589 handle_command_reply: tid=0 2024-05-24T11:44:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1ba3e2db80 1 client.25589 shutdown 2024-05-24T11:44:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/392668792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe08165480 msgr2=0x55fe08175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe08165480 0x55fe08175de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b800cad50 tx=0x7f1b800cc040 comp rx=0 tx=0).stop 2024-05-24T11:44:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/392668792 shutdown_connections 2024-05-24T11:44:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0816b270 0x55fe081720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe081e6630 0x55fe081e8a50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe08165480 0x55fe08175de0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1ba3e2db80 1 --2- 172.21.15.39:0/392668792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0816a950 0x55fe08171bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.417+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/392668792 >> 172.21.15.39:0/392668792 conn(0x55fe080c31b0 msgr2=0x55fe08164690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.418+0000 7f1b757fa700 0 client.25589 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.418+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/392668792 shutdown_connections 2024-05-24T11:44:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.418+0000 7f1ba3e2db80 1 -- 172.21.15.39:0/392668792 wait complete. 2024-05-24T11:44:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.418+0000 7f1ba3e2db80 20 client.25589 trim_cache size 0 max 16384 2024-05-24T11:44:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:44:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:44:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:44:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:44:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 -- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1b780a1350 msgr2=0x7f1b780a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1b780a1350 0x7f1b780a3810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b88005d90 tx=0x7f1b8800b3e0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 -- 172.21.15.39:0/2655407507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9006c970 msgr2=0x7f1b901a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9006c970 0x7f1b901a6190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000ef10 tx=0x7f1b8000c3a0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 -- 172.21.15.39:0/2655407507 shutdown_connections 2024-05-24T11:44:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b9011a640 0x7f1b901aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1b780a1350 0x7f1b780a3810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9006c970 0x7f1b901a6190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 --2- 172.21.15.39:0/2655407507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9006d290 0x7f1b901a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 -- 172.21.15.39:0/2655407507 >> 172.21.15.39:0/2655407507 conn(0x7f1b9006c020 msgr2=0x7f1b90109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 -- 172.21.15.39:0/2655407507 shutdown_connections 2024-05-24T11:44:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:45.419+0000 7f1b97c72700 1 -- 172.21.15.39:0/2655407507 wait complete. 2024-05-24T11:44:46.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:46 smithi053 ceph-mon[47448]: pgmap v199: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2024-05-24T11:44:46.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:46 smithi137 ceph-mon[47907]: pgmap v199: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2024-05-24T11:44:46.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:46 smithi039 ceph-mon[42919]: pgmap v199: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2024-05-24T11:44:48.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:48 smithi053 ceph-mon[47448]: pgmap v200: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-05-24T11:44:48.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:48 smithi137 ceph-mon[47907]: pgmap v200: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-05-24T11:44:48.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:48 smithi039 ceph-mon[42919]: pgmap v200: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-05-24T11:44:49.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 325.0 s, 1293.2 tps, lat 77.191 ms stddev 66.246 2024-05-24T11:44:50.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:50 smithi053 ceph-mon[47448]: pgmap v201: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:44:50.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:50 smithi137 ceph-mon[47907]: pgmap v201: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:44:50.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:50 smithi039 ceph-mon[42919]: pgmap v201: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:44:52.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:52 smithi053 ceph-mon[47448]: pgmap v202: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:44:52.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:52 smithi137 ceph-mon[47907]: pgmap v202: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:44:52.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:52 smithi039 ceph-mon[42919]: pgmap v202: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:44:54.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 330.0 s, 1246.4 tps, lat 80.216 ms stddev 24.392 2024-05-24T11:44:54.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:54 smithi053 ceph-mon[47448]: pgmap v203: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:44:54.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:54 smithi137 ceph-mon[47907]: pgmap v203: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:44:54.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:54 smithi039 ceph-mon[42919]: pgmap v203: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:44:56.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:56 smithi053 ceph-mon[47448]: pgmap v204: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-05-24T11:44:56.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:56 smithi137 ceph-mon[47907]: pgmap v204: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-05-24T11:44:56.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:56 smithi039 ceph-mon[42919]: pgmap v204: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-05-24T11:44:58.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:44:58 smithi053 ceph-mon[47448]: pgmap v205: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:44:58.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:44:58 smithi137 ceph-mon[47907]: pgmap v205: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:44:58.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:44:58 smithi039 ceph-mon[42919]: pgmap v205: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-05-24T11:44:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.590+0000 7f4866c1e700 1 -- 172.21.15.39:0/2757770180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48601140f0 msgr2=0x7f48601164e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.590+0000 7f4866c1e700 1 --2- 172.21.15.39:0/2757770180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48601140f0 0x7f48601164e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4850009990 tx=0x7f4850009ca0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.590+0000 7f4866c1e700 1 -- 172.21.15.39:0/2757770180 shutdown_connections 2024-05-24T11:44:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.590+0000 7f4866c1e700 1 --2- 172.21.15.39:0/2757770180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48601140f0 0x7f48601164e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.590+0000 7f4866c1e700 1 --2- 172.21.15.39:0/2757770180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4860107fa0 0x7f4860108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.590+0000 7f4866c1e700 1 --2- 172.21.15.39:0/2757770180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48601088c0 0x7f4860113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.590+0000 7f4866c1e700 1 -- 172.21.15.39:0/2757770180 >> 172.21.15.39:0/2757770180 conn(0x7f486006bf80 msgr2=0x7f486006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.590+0000 7f4866c1e700 1 -- 172.21.15.39:0/2757770180 shutdown_connections 2024-05-24T11:44:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.590+0000 7f4866c1e700 1 -- 172.21.15.39:0/2757770180 wait complete. 2024-05-24T11:44:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.591+0000 7f4866c1e700 1 Processor -- start 2024-05-24T11:44:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.591+0000 7f4866c1e700 1 -- start start 2024-05-24T11:44:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.591+0000 7f4866c1e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860107fa0 0x7f48601a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.591+0000 7f4866c1e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48601088c0 0x7f48601a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.591+0000 7f4866c1e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48601140f0 0x7f48601a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.591+0000 7f4866c1e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601a20a0 con 0x7f48601088c0 2024-05-24T11:44:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.591+0000 7f4866c1e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601a2210 con 0x7f4860107fa0 2024-05-24T11:44:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.591+0000 7f4866c1e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601a23d0 con 0x7f48601140f0 2024-05-24T11:44:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48651bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48601140f0 0x7f48601a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48651bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48601140f0 0x7f48601a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:52356/0 (socket says 172.21.15.39:52356) 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48651bb700 1 -- 172.21.15.39:0/4237286893 learned_addr learned my addr 172.21.15.39:0/4237286893 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f485ffff700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48601088c0 0x7f48601a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48651bb700 1 -- 172.21.15.39:0/4237286893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860107fa0 msgr2=0x7f48601a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48649ba700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860107fa0 0x7f48601a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48651bb700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860107fa0 0x7f48601a7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48651bb700 1 -- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48601088c0 msgr2=0x7f48601a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48651bb700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48601088c0 0x7f48601a7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48651bb700 1 -- 172.21.15.39:0/4237286893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48500095f0 con 0x7f48601140f0 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48649ba700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860107fa0 0x7f48601a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:44:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.592+0000 7f48651bb700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48601140f0 0x7f48601a1a40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4850005e50 tx=0x7f48500039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.593+0000 7f485dffb700 1 -- 172.21.15.39:0/4237286893 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485001d070 con 0x7f48601140f0 2024-05-24T11:44:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.593+0000 7f485dffb700 1 -- 172.21.15.39:0/4237286893 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f485000bcb0 con 0x7f48601140f0 2024-05-24T11:44:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.593+0000 7f4866c1e700 1 -- 172.21.15.39:0/4237286893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48601a2650 con 0x7f48601140f0 2024-05-24T11:44:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.593+0000 7f4866c1e700 1 -- 172.21.15.39:0/4237286893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48601ae900 con 0x7f48601140f0 2024-05-24T11:44:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.593+0000 7f485dffb700 1 -- 172.21.15.39:0/4237286893 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000fcb0 con 0x7f48601140f0 2024-05-24T11:44:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.596+0000 7f485dffb700 1 -- 172.21.15.39:0/4237286893 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f485000f460 con 0x7f48601140f0 2024-05-24T11:44:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.598+0000 7f4872dd9b80 1 Processor -- start 2024-05-24T11:44:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f4872dd9b80 1 -- start start 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505961950 0x557505961d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x55750595bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 0x55750595c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575058b30f0 con 0x557505961950 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575058b0ab0 con 0x557505962270 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575058ac3a0 con 0x55750595c480 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f48649ba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x55750595bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f48649ba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x55750595bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:38652/0 (socket says 172.21.15.39:38652) 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f48649ba700 1 -- 172.21.15.39:0/3456784620 learned_addr learned my addr 172.21.15.39:0/3456784620 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.599+0000 7f485ffff700 1 --2- 172.21.15.39:0/3456784620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 0x55750595c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f485ffff700 1 --2- 172.21.15.39:0/3456784620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505961950 0x557505961d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f485dffb700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f484c0a1420 0x7f484c0a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f48649ba700 1 -- 172.21.15.39:0/3456784620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 msgr2=0x55750595c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f48649ba700 1 --2- 172.21.15.39:0/3456784620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 0x55750595c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f48649ba700 1 -- 172.21.15.39:0/3456784620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505961950 msgr2=0x557505961d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f48649ba700 1 --2- 172.21.15.39:0/3456784620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505961950 0x557505961d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f48649ba700 1 -- 172.21.15.39:0/3456784620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55750572d350 con 0x557505962270 2024-05-24T11:44:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f48651bb700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f484c0a1420 0x7f484c0a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f48649ba700 1 --2- 172.21.15.39:0/3456784620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x55750595bf40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4858009400 tx=0x7f4858009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9665e9b9a29fce server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f485dffb700 1 -- 172.21.15.39:0/4237286893 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f48500cd950 con 0x7f48601140f0 2024-05-24T11:44:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.600+0000 7f4856ffd700 1 -- 172.21.15.39:0/3456784620 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858003c00 con 0x557505962270 2024-05-24T11:44:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f4856ffd700 1 -- 172.21.15.39:0/3456784620 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f485800f050 con 0x557505962270 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f4856ffd700 1 -- 172.21.15.39:0/3456784620 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858014670 con 0x557505962270 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3456784620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 msgr2=0x55750595bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f48651bb700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f484c0a1420 0x7f484c0a38e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4850097030 tx=0x7f4850027f90 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3456784620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x55750595bf40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4858009400 tx=0x7f4858009920 comp rx=0 tx=0).stop 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3456784620 shutdown_connections 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3456784620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 0x55750595c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3456784620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505961950 0x557505961d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3456784620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x55750595bf40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.601+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3456784620 >> 172.21.15.39:0/3456784620 conn(0x5575058ba1b0 msgr2=0x557505956840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.602+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3456784620 shutdown_connections 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.602+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3456784620 wait complete. 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.602+0000 7f4872dd9b80 1 -- start start 2024-05-24T11:44:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.602+0000 7f4872dd9b80 1 Processor -- start 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.603+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 0x557505968170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.603+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x5575059654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.603+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557505962270 0x557505965ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.603+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557505848500 con 0x55750595c480 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.603+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575058481a0 con 0x557505961950 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.603+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55750572f640 con 0x557505962270 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.603+0000 7f4872dd9b80 10 client.0 _ll_register_callbacks cb 0x557505851ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.604+0000 7f48649ba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x5575059654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.604+0000 7f48649ba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x5575059654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:38660/0 (socket says 172.21.15.39:38660) 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.604+0000 7f48649ba700 1 -- 172.21.15.39:0/1093268728 learned_addr learned my addr 172.21.15.39:0/1093268728 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.604+0000 7f48649ba700 1 -- 172.21.15.39:0/1093268728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557505962270 msgr2=0x557505965ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:44:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.604+0000 7f48649ba700 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557505962270 0x557505965ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.604+0000 7f48649ba700 1 -- 172.21.15.39:0/1093268728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 msgr2=0x557505968170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:44:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.604+0000 7f48649ba700 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 0x557505968170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.604+0000 7f48649ba700 1 -- 172.21.15.39:0/1093268728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48580090b0 con 0x557505961950 2024-05-24T11:44:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.605+0000 7f48649ba700 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x5575059654b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4858000f80 tx=0x7f4858015920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8262982a249877c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.605+0000 7f483ffff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:44:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.605+0000 7f483ffff700 1 -- 172.21.15.39:0/1093268728 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858015dc0 con 0x557505961950 2024-05-24T11:44:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.605+0000 7f4872dd9b80 1 -- 172.21.15.39:0/1093268728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557505805210 con 0x557505961950 2024-05-24T11:44:58.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.605+0000 7f483ffff700 1 -- 172.21.15.39:0/1093268728 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4858009600 con 0x557505961950 2024-05-24T11:44:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.605+0000 7f483ffff700 1 -- 172.21.15.39:0/1093268728 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4858028460 con 0x557505961950 2024-05-24T11:44:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.605+0000 7f4872dd9b80 10 client.25598 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.605+0000 7f4872dd9b80 1 -- 172.21.15.39:0/1093268728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5575056d9b00 con 0x557505961950 2024-05-24T11:44:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.606+0000 7f483ffff700 1 -- 172.21.15.39:0/1093268728 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557505805210 con 0x557505961950 2024-05-24T11:44:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.606+0000 7f483ffff700 1 -- 172.21.15.39:0/1093268728 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f485801b070 con 0x557505961950 2024-05-24T11:44:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.606+0000 7f4872dd9b80 10 client.25598 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.606+0000 7f4872dd9b80 10 client.25598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.606+0000 7f4872dd9b80 20 client.25598 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.606+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55750597e380 0x5575059807a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.606+0000 7f4872dd9b80 4 client.25598 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.606+0000 7f4872dd9b80 1 -- 172.21.15.39:0/1093268728 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55750592b7b0 con 0x55750597e380 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.606+0000 7f485ffff700 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55750597e380 0x5575059807a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.607+0000 7f483ffff700 10 client.25598 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.607+0000 7f485ffff700 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55750597e380 0x5575059807a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d725be8e816f6dde server_cookie=3e27b5c7c980a6de in_seq=0 out_seq=0 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.607+0000 7f483ffff700 1 -- 172.21.15.39:0/1093268728 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55750592b7b0 con 0x55750597e380 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.607+0000 7f483ffff700 10 client.25598 handle_command_reply: tid=0 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 client.25598 shutdown 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 -- 172.21.15.39:0/1093268728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 msgr2=0x5575059654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x5575059654b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4858000f80 tx=0x7f4858015920 comp rx=0 tx=0).stop 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 -- 172.21.15.39:0/1093268728 shutdown_connections 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557505962270 0x557505965ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55750597e380 0x5575059807a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 0x557505968170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/1093268728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x5575059654b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 -- 172.21.15.39:0/1093268728 >> 172.21.15.39:0/1093268728 conn(0x5575058ba1b0 msgr2=0x55750595af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f483ffff700 0 client.25598 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 -- 172.21.15.39:0/1093268728 shutdown_connections 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 1 -- 172.21.15.39:0/1093268728 wait complete. 2024-05-24T11:44:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.608+0000 7f4872dd9b80 20 client.25598 trim_cache size 0 max 16384 2024-05-24T11:44:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.629+0000 7f4872dd9b80 1 Processor -- start 2024-05-24T11:44:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.629+0000 7f4872dd9b80 1 -- start start 2024-05-24T11:44:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 0x557505956910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x55750595c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505962270 0x557505963d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55750595cdf0 con 0x557505962270 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557505985980 con 0x557505961950 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557505980e30 con 0x55750595c480 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f48649ba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x55750595c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f48649ba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x55750595c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:38676/0 (socket says 172.21.15.39:38676) 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f48649ba700 1 -- 172.21.15.39:0/4198295107 learned_addr learned my addr 172.21.15.39:0/4198295107 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f485ffff700 1 --2- 172.21.15.39:0/4198295107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505962270 0x557505963d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f485ffff700 1 --2- 172.21.15.39:0/4198295107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 0x557505956910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f48649ba700 1 -- 172.21.15.39:0/4198295107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 msgr2=0x557505956910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f48649ba700 1 --2- 172.21.15.39:0/4198295107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 0x557505956910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f48649ba700 1 -- 172.21.15.39:0/4198295107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505962270 msgr2=0x557505963d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f48649ba700 1 --2- 172.21.15.39:0/4198295107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505962270 0x557505963d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.630+0000 7f48649ba700 1 -- 172.21.15.39:0/4198295107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557505872500 con 0x557505961950 2024-05-24T11:44:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.631+0000 7f48649ba700 1 --2- 172.21.15.39:0/4198295107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x55750595c860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4858009550 tx=0x7f485801f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6194073b747ef463 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.631+0000 7f48577fe700 1 -- 172.21.15.39:0/4198295107 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858009660 con 0x557505961950 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.631+0000 7f48577fe700 1 -- 172.21.15.39:0/4198295107 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4858006000 con 0x557505961950 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.631+0000 7f48577fe700 1 -- 172.21.15.39:0/4198295107 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858015a40 con 0x557505961950 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.632+0000 7f4872dd9b80 1 -- 172.21.15.39:0/4198295107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 msgr2=0x55750595c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.632+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/4198295107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x55750595c860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4858009550 tx=0x7f485801f420 comp rx=0 tx=0).stop 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.632+0000 7f4872dd9b80 1 -- 172.21.15.39:0/4198295107 shutdown_connections 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.632+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/4198295107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55750595c480 0x557505956910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.632+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/4198295107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557505962270 0x557505963d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.632+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/4198295107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505961950 0x55750595c860 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.632+0000 7f4872dd9b80 1 -- 172.21.15.39:0/4198295107 >> 172.21.15.39:0/4198295107 conn(0x5575058ba1b0 msgr2=0x557505979f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.632+0000 7f4872dd9b80 1 -- 172.21.15.39:0/4198295107 shutdown_connections 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.632+0000 7f4872dd9b80 1 -- 172.21.15.39:0/4198295107 wait complete. 2024-05-24T11:44:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.633+0000 7f4872dd9b80 1 -- start start 2024-05-24T11:44:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.633+0000 7f4872dd9b80 1 Processor -- start 2024-05-24T11:44:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.633+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 0x5575059758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.633+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557505961950 0x557505973460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.633+0000 7f4872dd9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x557505973a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.633+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557505974110 con 0x55750595c480 2024-05-24T11:44:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.633+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557505974280 con 0x557505962270 2024-05-24T11:44:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.633+0000 7f4872dd9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575059d3840 con 0x557505961950 2024-05-24T11:44:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.633+0000 7f4872dd9b80 10 client.0 _ll_register_callbacks cb 0x557505851ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 0x5575059758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 0x5575059758d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42278/0 (socket says 172.21.15.39:42278) 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 -- 172.21.15.39:0/3614017316 learned_addr learned my addr 172.21.15.39:0/3614017316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x557505973a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f48649ba700 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557505961950 0x557505973460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 -- 172.21.15.39:0/3614017316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557505961950 msgr2=0x557505973460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557505961950 0x557505973460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 -- 172.21.15.39:0/3614017316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 msgr2=0x5575059758d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 0x5575059758d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 -- 172.21.15.39:0/3614017316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575056d9b00 con 0x557505962270 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f485ffff700 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x557505973a70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x557505956f20 tx=0x7f4848009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178eee35733695c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f483ffff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:44:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.634+0000 7f483ffff700 1 -- 172.21.15.39:0/3614017316 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848012050 con 0x557505962270 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f483ffff700 1 -- 172.21.15.39:0/3614017316 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f48480121b0 con 0x557505962270 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3614017316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f485800f050 con 0x557505962270 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f483ffff700 1 -- 172.21.15.39:0/3614017316 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4848012370 con 0x557505962270 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f4872dd9b80 10 client.25604 fetch_fsmap learned FSMap version 27 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3614017316 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557505731c70 con 0x557505962270 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f483ffff700 1 -- 172.21.15.39:0/3614017316 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f485800f050 con 0x557505962270 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f483ffff700 1 -- 172.21.15.39:0/3614017316 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f484802d820 con 0x557505962270 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f4872dd9b80 10 client.25604 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f4872dd9b80 10 client.25604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:44:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f4872dd9b80 20 client.25604 populate_metadata read hostname 'smithi039' 2024-05-24T11:44:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5575059d4fd0 0x5575059d73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:44:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f4872dd9b80 4 client.25604 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:44:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.635+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3614017316 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5575058a4e60 con 0x5575059d4fd0 2024-05-24T11:44:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.636+0000 7f48649ba700 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5575059d4fd0 0x5575059d73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:44:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.636+0000 7f483ffff700 10 client.25604 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.636+0000 7f48649ba700 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5575059d4fd0 0x5575059d73f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7208fff39195541a server_cookie=5f721c69fb87dd24 in_seq=0 out_seq=0 2024-05-24T11:44:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f483ffff700 1 -- 172.21.15.39:0/3614017316 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5575058a4e60 con 0x5575059d4fd0 2024-05-24T11:44:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f483ffff700 10 client.25604 handle_command_reply: tid=0 2024-05-24T11:44:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f4872dd9b80 1 client.25604 shutdown 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3614017316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 msgr2=0x557505973a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x557505973a70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x557505956f20 tx=0x7f4848009260 comp rx=0 tx=0).stop 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3614017316 shutdown_connections 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5575059d4fd0 0x5575059d73f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557505962270 0x557505973a70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f483ffff700 0 client.25604 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55750595c480 0x5575059758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f4872dd9b80 1 --2- 172.21.15.39:0/3614017316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557505961950 0x557505973460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.662+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3614017316 >> 172.21.15.39:0/3614017316 conn(0x5575058ba1b0 msgr2=0x55750595f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.663+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3614017316 shutdown_connections 2024-05-24T11:44:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.663+0000 7f4872dd9b80 1 -- 172.21.15.39:0/3614017316 wait complete. 2024-05-24T11:44:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.663+0000 7f4872dd9b80 20 client.25604 trim_cache size 0 max 16384 2024-05-24T11:44:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:44:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:44:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:44:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:44:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 -- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f484c0a1420 msgr2=0x7f484c0a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f484c0a1420 0x7f484c0a38e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4850097030 tx=0x7f4850027f90 comp rx=0 tx=0).stop 2024-05-24T11:44:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 -- 172.21.15.39:0/4237286893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48601140f0 msgr2=0x7f48601a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:44:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48601140f0 0x7f48601a1a40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4850005e50 tx=0x7f48500039d0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 -- 172.21.15.39:0/4237286893 shutdown_connections 2024-05-24T11:44:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48601140f0 0x7f48601a1a40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f484c0a1420 0x7f484c0a38e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4860107fa0 0x7f48601a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 --2- 172.21.15.39:0/4237286893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48601088c0 0x7f48601a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:44:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 -- 172.21.15.39:0/4237286893 >> 172.21.15.39:0/4237286893 conn(0x7f486006bf80 msgr2=0x7f486006fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:44:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 -- 172.21.15.39:0/4237286893 shutdown_connections 2024-05-24T11:44:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:44:58.664+0000 7f4866c1e700 1 -- 172.21.15.39:0/4237286893 wait complete. 2024-05-24T11:44:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 335.0 s, 1103.8 tps, lat 90.645 ms stddev 66.342 2024-05-24T11:45:00.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:00 smithi053 ceph-mon[47448]: pgmap v206: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 22 MiB/s wr, 1.70k op/s 2024-05-24T11:45:00.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:00 smithi137 ceph-mon[47907]: pgmap v206: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 22 MiB/s wr, 1.70k op/s 2024-05-24T11:45:00.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:00 smithi039 ceph-mon[42919]: pgmap v206: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 22 MiB/s wr, 1.70k op/s 2024-05-24T11:45:02.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:02 smithi053 ceph-mon[47448]: pgmap v207: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-05-24T11:45:02.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:02 smithi137 ceph-mon[47907]: pgmap v207: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-05-24T11:45:02.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:02 smithi039 ceph-mon[42919]: pgmap v207: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-05-24T11:45:04.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 340.0 s, 1151.4 tps, lat 67.729 ms stddev 33.293 2024-05-24T11:45:04.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:04 smithi053 ceph-mon[47448]: pgmap v208: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-05-24T11:45:04.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:04 smithi137 ceph-mon[47907]: pgmap v208: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-05-24T11:45:04.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:04 smithi039 ceph-mon[42919]: pgmap v208: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-05-24T11:45:06.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:06 smithi053 ceph-mon[47448]: pgmap v209: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 21 MiB/s wr, 1.65k op/s 2024-05-24T11:45:06.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:06 smithi137 ceph-mon[47907]: pgmap v209: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 21 MiB/s wr, 1.65k op/s 2024-05-24T11:45:06.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:06 smithi039 ceph-mon[42919]: pgmap v209: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 21 MiB/s wr, 1.65k op/s 2024-05-24T11:45:08.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:08 smithi053 ceph-mon[47448]: pgmap v210: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-05-24T11:45:08.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:08 smithi137 ceph-mon[47907]: pgmap v210: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-05-24T11:45:08.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:08 smithi039 ceph-mon[42919]: pgmap v210: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-05-24T11:45:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 345.0 s, 1073.8 tps, lat 41.820 ms stddev 15.519 2024-05-24T11:45:09.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:09 smithi053 ceph-mon[47448]: osdmap e81: 12 total, 12 up, 12 in 2024-05-24T11:45:09.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:09 smithi053 ceph-mon[47448]: pgmap v212: 97 pgs: 14 active+clean+snaptrim, 54 active+clean, 29 active+clean+snaptrim_wait; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:45:09.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:09 smithi137 ceph-mon[47907]: osdmap e81: 12 total, 12 up, 12 in 2024-05-24T11:45:09.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:09 smithi137 ceph-mon[47907]: pgmap v212: 97 pgs: 14 active+clean+snaptrim, 54 active+clean, 29 active+clean+snaptrim_wait; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:45:09.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:09 smithi039 ceph-mon[42919]: osdmap e81: 12 total, 12 up, 12 in 2024-05-24T11:45:09.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:09 smithi039 ceph-mon[42919]: pgmap v212: 97 pgs: 14 active+clean+snaptrim, 54 active+clean, 29 active+clean+snaptrim_wait; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:45:11.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:11 smithi053 ceph-mon[47448]: pgmap v213: 97 pgs: 14 active+clean+snaptrim, 54 active+clean, 29 active+clean+snaptrim_wait; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:45:11.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:11 smithi137 ceph-mon[47907]: pgmap v213: 97 pgs: 14 active+clean+snaptrim, 54 active+clean, 29 active+clean+snaptrim_wait; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:45:11.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:11 smithi039 ceph-mon[42919]: pgmap v213: 97 pgs: 14 active+clean+snaptrim, 54 active+clean, 29 active+clean+snaptrim_wait; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:45:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.838+0000 7fae356d9700 1 -- 172.21.15.39:0/388305703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffda0 msgr2=0x7fae30100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.838+0000 7fae356d9700 1 --2- 172.21.15.39:0/388305703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffda0 0x7fae30100220 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fae18009990 tx=0x7fae18009ca0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.838+0000 7fae356d9700 1 -- 172.21.15.39:0/388305703 shutdown_connections 2024-05-24T11:45:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.838+0000 7fae356d9700 1 --2- 172.21.15.39:0/388305703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30105e60 0x7fae30106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.838+0000 7fae356d9700 1 --2- 172.21.15.39:0/388305703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae30100760 0x7fae3010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.838+0000 7fae356d9700 1 --2- 172.21.15.39:0/388305703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffda0 0x7fae30100220 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.838+0000 7fae356d9700 1 -- 172.21.15.39:0/388305703 >> 172.21.15.39:0/388305703 conn(0x7fae30076f10 msgr2=0x7fae30077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.838+0000 7fae356d9700 1 -- 172.21.15.39:0/388305703 shutdown_connections 2024-05-24T11:45:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.839+0000 7fae356d9700 1 -- 172.21.15.39:0/388305703 wait complete. 2024-05-24T11:45:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.839+0000 7fae356d9700 1 Processor -- start 2024-05-24T11:45:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.839+0000 7fae356d9700 1 -- start start 2024-05-24T11:45:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae356d9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffda0 0x7fae30199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae356d9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30100760 0x7fae30199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae356d9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae30105e60 0x7fae3019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae356d9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3019e2e0 con 0x7fae300ffda0 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae356d9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3019e450 con 0x7fae30105e60 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae356d9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3019e610 con 0x7fae30100760 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae2f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae30105e60 0x7fae3019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae2f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae30105e60 0x7fae3019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:35822/0 (socket says 172.21.15.39:35822) 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae2f7fe700 1 -- 172.21.15.39:0/1151580863 learned_addr learned my addr 172.21.15.39:0/1151580863 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae27fff700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30100760 0x7fae30199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae2effd700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffda0 0x7fae30199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae2f7fe700 1 -- 172.21.15.39:0/1151580863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30100760 msgr2=0x7fae30199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30100760 0x7fae30199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae2f7fe700 1 -- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffda0 msgr2=0x7fae30199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffda0 0x7fae30199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.840+0000 7fae2f7fe700 1 -- 172.21.15.39:0/1151580863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180095f0 con 0x7fae30105e60 2024-05-24T11:45:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.841+0000 7fae27fff700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30100760 0x7fae30199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:45:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.841+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae30105e60 0x7fae3019dca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae2800ef30 tx=0x7fae2800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.841+0000 7fae2cff9700 1 -- 172.21.15.39:0/1151580863 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28018070 con 0x7fae30105e60 2024-05-24T11:45:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.841+0000 7fae356d9700 1 -- 172.21.15.39:0/1151580863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3019e890 con 0x7fae30105e60 2024-05-24T11:45:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.841+0000 7fae2cff9700 1 -- 172.21.15.39:0/1151580863 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae2800f040 con 0x7fae30105e60 2024-05-24T11:45:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.841+0000 7fae356d9700 1 -- 172.21.15.39:0/1151580863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae301a6130 con 0x7fae30105e60 2024-05-24T11:45:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.841+0000 7fae2cff9700 1 -- 172.21.15.39:0/1151580863 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2800cc70 con 0x7fae30105e60 2024-05-24T11:45:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.845+0000 7fae2cff9700 1 -- 172.21.15.39:0/1151580863 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fae28004000 con 0x7fae30105e60 2024-05-24T11:45:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.846+0000 7fae2cff9700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fae100a1390 0x7fae100a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.846+0000 7fae2cff9700 1 -- 172.21.15.39:0/1151580863 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fae280cc710 con 0x7fae30105e60 2024-05-24T11:45:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.846+0000 7fae2effd700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fae100a1390 0x7fae100a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.846+0000 7fae41894b80 1 Processor -- start 2024-05-24T11:45:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.847+0000 7fae41894b80 1 -- start start 2024-05-24T11:45:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.847+0000 7fae2effd700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fae100a1390 0x7fae100a3850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae20009c60 tx=0x7fae20009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.847+0000 7fae41894b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 0x5648d8679d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.847+0000 7fae41894b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.847+0000 7fae41894b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d867e240 0x5648d867e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.847+0000 7fae41894b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d85c80f0 con 0x5648d867e240 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.847+0000 7fae41894b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d85c5ab0 con 0x5648d867a260 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.847+0000 7fae41894b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d85c13a0 con 0x5648d8679940 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.847+0000 7fae27fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae27fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:35824/0 (socket says 172.21.15.39:35824) 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae27fff700 1 -- 172.21.15.39:0/3825690505 learned_addr learned my addr 172.21.15.39:0/3825690505 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae27fff700 1 -- 172.21.15.39:0/3825690505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 msgr2=0x5648d8679d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae27fff700 1 --2- 172.21.15.39:0/3825690505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 0x5648d8679d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae27fff700 1 -- 172.21.15.39:0/3825690505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d867e240 msgr2=0x5648d867e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae27fff700 1 --2- 172.21.15.39:0/3825690505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d867e240 0x5648d867e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae27fff700 1 -- 172.21.15.39:0/3825690505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648d8442350 con 0x5648d867a260 2024-05-24T11:45:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae27fff700 1 --2- 172.21.15.39:0/3825690505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867dd00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae18009990 tx=0x7fae18005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb63d43ae09fe5c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae257fa700 1 -- 172.21.15.39:0/3825690505 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1801d050 con 0x5648d867a260 2024-05-24T11:45:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae257fa700 1 -- 172.21.15.39:0/3825690505 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae1801e040 con 0x5648d867a260 2024-05-24T11:45:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.848+0000 7fae257fa700 1 -- 172.21.15.39:0/3825690505 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18005320 con 0x5648d867a260 2024-05-24T11:45:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.849+0000 7fae41894b80 1 -- 172.21.15.39:0/3825690505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 msgr2=0x5648d867dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.849+0000 7fae41894b80 1 --2- 172.21.15.39:0/3825690505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867dd00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae18009990 tx=0x7fae18005c00 comp rx=0 tx=0).stop 2024-05-24T11:45:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 -- 172.21.15.39:0/3825690505 shutdown_connections 2024-05-24T11:45:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 --2- 172.21.15.39:0/3825690505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 0x5648d8679d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 --2- 172.21.15.39:0/3825690505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d867e240 0x5648d867e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 --2- 172.21.15.39:0/3825690505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867dd00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 -- 172.21.15.39:0/3825690505 >> 172.21.15.39:0/3825690505 conn(0x5648d85cf1b0 msgr2=0x5648d866b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 -- 172.21.15.39:0/3825690505 shutdown_connections 2024-05-24T11:45:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 -- 172.21.15.39:0/3825690505 wait complete. 2024-05-24T11:45:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 -- start start 2024-05-24T11:45:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 Processor -- start 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 0x5648d8682d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d8683290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d86837d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d855d500 con 0x5648d8679940 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d855d1a0 con 0x5648d867a260 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8444640 con 0x5648d867e240 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.850+0000 7fae41894b80 10 client.0 _ll_register_callbacks cb 0x5648d8566ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae2effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 0x5648d8682d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae2effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 0x5648d8682d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51270/0 (socket says 172.21.15.39:51270) 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae27fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d8683290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae2effd700 1 -- 172.21.15.39:0/2328033854 learned_addr learned my addr 172.21.15.39:0/2328033854 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d86837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae27fff700 1 -- 172.21.15.39:0/2328033854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 msgr2=0x5648d86837d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae27fff700 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d86837d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae27fff700 1 -- 172.21.15.39:0/2328033854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 msgr2=0x5648d8682d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae27fff700 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 0x5648d8682d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae27fff700 1 -- 172.21.15.39:0/2328033854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3010d4f0 con 0x5648d867a260 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.851+0000 7fae27fff700 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d8683290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae18000c00 tx=0x7fae18004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=460ce000d65a5018 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae1e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2328033854 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18031070 con 0x5648d867a260 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2328033854 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fae180050a0 con 0x5648d867a260 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae41894b80 1 -- 172.21.15.39:0/2328033854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648d851a210 con 0x5648d867a260 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2328033854 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae1801d020 con 0x5648d867a260 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae41894b80 10 client.25616 fetch_fsmap learned FSMap version 27 2024-05-24T11:45:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae41894b80 1 -- 172.21.15.39:0/2328033854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648d83eeb00 con 0x5648d867a260 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2328033854 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5648d851a210 con 0x5648d867a260 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2328033854 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae18034020 con 0x5648d867a260 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae41894b80 10 client.25616 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae41894b80 10 client.25616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae41894b80 20 client.25616 populate_metadata read hostname 'smithi039' 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae41894b80 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5648d8688350 0x5648d868a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae41894b80 4 client.25616 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.852+0000 7fae41894b80 1 -- 172.21.15.39:0/2328033854 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648d86407b0 con 0x5648d8688350 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.853+0000 7fae2effd700 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5648d8688350 0x5648d868a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.853+0000 7fae1e7fc700 10 client.25616 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.853+0000 7fae2effd700 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5648d8688350 0x5648d868a770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe7c04f106a1e1d server_cookie=2c2532e44bcc127c in_seq=0 out_seq=0 2024-05-24T11:45:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2328033854 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648d86407b0 con 0x5648d8688350 2024-05-24T11:45:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae1e7fc700 10 client.25616 handle_command_reply: tid=0 2024-05-24T11:45:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 client.25616 shutdown 2024-05-24T11:45:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 -- 172.21.15.39:0/2328033854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 msgr2=0x5648d8683290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d8683290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae18000c00 tx=0x7fae18004a90 comp rx=0 tx=0).stop 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 -- 172.21.15.39:0/2328033854 shutdown_connections 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d86837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5648d8688350 0x5648d868a770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 0x5648d8682d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 --2- 172.21.15.39:0/2328033854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d8683290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae1e7fc700 0 client.25616 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 -- 172.21.15.39:0/2328033854 >> 172.21.15.39:0/2328033854 conn(0x5648d85cf1b0 msgr2=0x5648d867ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 -- 172.21.15.39:0/2328033854 shutdown_connections 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.854+0000 7fae41894b80 1 -- 172.21.15.39:0/2328033854 wait complete. 2024-05-24T11:45:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.855+0000 7fae41894b80 20 client.25616 trim_cache size 0 max 16384 2024-05-24T11:45:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.878+0000 7fae41894b80 1 Processor -- start 2024-05-24T11:45:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.878+0000 7fae41894b80 1 -- start start 2024-05-24T11:45:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.878+0000 7fae41894b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 0x5648d866b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae41894b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d867a260 0x5648d867f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae41894b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867e240 0x5648d86985d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae41894b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8666330 con 0x5648d867a260 2024-05-24T11:45:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae41894b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d86843c0 con 0x5648d867e240 2024-05-24T11:45:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae41894b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d866d0f0 con 0x5648d8679940 2024-05-24T11:45:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 0x5648d866b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 0x5648d866b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:42322/0 (socket says 172.21.15.39:42322) 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2effd700 1 -- 172.21.15.39:0/13049705 learned_addr learned my addr 172.21.15.39:0/13049705 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/13049705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867e240 0x5648d86985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae27fff700 1 --2- 172.21.15.39:0/13049705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d867a260 0x5648d867f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2effd700 1 -- 172.21.15.39:0/13049705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867e240 msgr2=0x5648d86985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2effd700 1 --2- 172.21.15.39:0/13049705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867e240 0x5648d86985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2effd700 1 -- 172.21.15.39:0/13049705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d867a260 msgr2=0x5648d867f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2effd700 1 --2- 172.21.15.39:0/13049705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d867a260 0x5648d867f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2effd700 1 -- 172.21.15.39:0/13049705 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648d8587500 con 0x5648d8679940 2024-05-24T11:45:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.879+0000 7fae2effd700 1 --2- 172.21.15.39:0/13049705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 0x5648d866b910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5648d868b040 tx=0x7fae20013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd0abdd464a3c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.880+0000 7fae25ffb700 1 -- 172.21.15.39:0/13049705 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20044040 con 0x5648d8679940 2024-05-24T11:45:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.880+0000 7fae25ffb700 1 -- 172.21.15.39:0/13049705 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae2004a610 con 0x5648d8679940 2024-05-24T11:45:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.880+0000 7fae25ffb700 1 -- 172.21.15.39:0/13049705 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae200523d0 con 0x5648d8679940 2024-05-24T11:45:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.880+0000 7fae41894b80 1 -- 172.21.15.39:0/13049705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 msgr2=0x5648d866b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.880+0000 7fae41894b80 1 --2- 172.21.15.39:0/13049705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 0x5648d866b910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5648d868b040 tx=0x7fae20013040 comp rx=0 tx=0).stop 2024-05-24T11:45:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.881+0000 7fae41894b80 1 -- 172.21.15.39:0/13049705 shutdown_connections 2024-05-24T11:45:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.881+0000 7fae41894b80 1 --2- 172.21.15.39:0/13049705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d8679940 0x5648d866b910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.881+0000 7fae41894b80 1 --2- 172.21.15.39:0/13049705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867e240 0x5648d86985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.881+0000 7fae41894b80 1 --2- 172.21.15.39:0/13049705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d867a260 0x5648d867f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.881+0000 7fae41894b80 1 -- 172.21.15.39:0/13049705 >> 172.21.15.39:0/13049705 conn(0x5648d85cf1b0 msgr2=0x5648d866d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.881+0000 7fae41894b80 1 -- 172.21.15.39:0/13049705 shutdown_connections 2024-05-24T11:45:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.881+0000 7fae41894b80 1 -- 172.21.15.39:0/13049705 wait complete. 2024-05-24T11:45:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.881+0000 7fae41894b80 1 -- start start 2024-05-24T11:45:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.881+0000 7fae41894b80 1 Processor -- start 2024-05-24T11:45:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae41894b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 0x5648d8675590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae41894b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae41894b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d867ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae41894b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8680440 con 0x5648d8679940 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae41894b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d86805b0 con 0x5648d867a260 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae41894b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648d8680770 con 0x5648d867e240 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae41894b80 10 client.0 _ll_register_callbacks cb 0x5648d8566ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae2f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d867ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae2f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d867ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:42330/0 (socket says 172.21.15.39:42330) 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae2f7fe700 1 -- 172.21.15.39:0/2088643949 learned_addr learned my addr 172.21.15.39:0/2088643949 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae27fff700 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae2f7fe700 1 -- 172.21.15.39:0/2088643949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 msgr2=0x5648d867f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae2f7fe700 1 -- 172.21.15.39:0/2088643949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 msgr2=0x5648d8675590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 0x5648d8675590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.882+0000 7fae2f7fe700 1 -- 172.21.15.39:0/2088643949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae20017070 con 0x5648d867e240 2024-05-24T11:45:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.883+0000 7fae2f7fe700 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d867ff00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae28003fd0 tx=0x7fae2809a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db21bde79c44a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.883+0000 7fae1e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:45:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.883+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2088643949 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae280a1c30 con 0x5648d867e240 2024-05-24T11:45:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.883+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2088643949 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fae280ca070 con 0x5648d867e240 2024-05-24T11:45:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.883+0000 7fae41894b80 1 -- 172.21.15.39:0/2088643949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae1801e040 con 0x5648d867e240 2024-05-24T11:45:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.888+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2088643949 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae280ca230 con 0x5648d867e240 2024-05-24T11:45:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.888+0000 7fae41894b80 10 client.25522 fetch_fsmap learned FSMap version 27 2024-05-24T11:45:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.888+0000 7fae41894b80 1 -- 172.21.15.39:0/2088643949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648d8446c70 con 0x5648d867e240 2024-05-24T11:45:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.888+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2088643949 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae1801e040 con 0x5648d867e240 2024-05-24T11:45:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.889+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2088643949 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae28098dc0 con 0x5648d867e240 2024-05-24T11:45:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.889+0000 7fae41894b80 10 client.25522 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:45:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.889+0000 7fae41894b80 10 client.25522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:45:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.889+0000 7fae41894b80 20 client.25522 populate_metadata read hostname 'smithi039' 2024-05-24T11:45:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.889+0000 7fae41894b80 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5648d86e1ac0 0x5648d86e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.889+0000 7fae41894b80 4 client.25522 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:45:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.889+0000 7fae41894b80 1 -- 172.21.15.39:0/2088643949 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648d85b9e60 con 0x5648d86e1ac0 2024-05-24T11:45:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.889+0000 7fae2effd700 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5648d86e1ac0 0x5648d86e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.890+0000 7fae1e7fc700 10 client.25522 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.890+0000 7fae2effd700 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5648d86e1ac0 0x5648d86e3ee0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=527bd6eab6c7bc23 server_cookie=8cc83f102a29c693 in_seq=0 out_seq=0 2024-05-24T11:45:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.938+0000 7fae1e7fc700 1 -- 172.21.15.39:0/2088643949 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648d85b9e60 con 0x5648d86e1ac0 2024-05-24T11:45:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.938+0000 7fae1e7fc700 10 client.25522 handle_command_reply: tid=0 2024-05-24T11:45:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.938+0000 7fae41894b80 1 client.25522 shutdown 2024-05-24T11:45:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.938+0000 7fae41894b80 1 -- 172.21.15.39:0/2088643949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 msgr2=0x5648d867ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.938+0000 7fae41894b80 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d867ff00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae28003fd0 tx=0x7fae2809a0b0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae41894b80 1 -- 172.21.15.39:0/2088643949 shutdown_connections 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae41894b80 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5648d867e240 0x5648d867ff00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae41894b80 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5648d86e1ac0 0x5648d86e3ee0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae41894b80 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5648d8679940 0x5648d8675590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae41894b80 1 --2- 172.21.15.39:0/2088643949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648d867a260 0x5648d867f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae1e7fc700 0 client.25522 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae41894b80 1 -- 172.21.15.39:0/2088643949 >> 172.21.15.39:0/2088643949 conn(0x5648d85cf1b0 msgr2=0x5648d86985d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae41894b80 1 -- 172.21.15.39:0/2088643949 shutdown_connections 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae41894b80 1 -- 172.21.15.39:0/2088643949 wait complete. 2024-05-24T11:45:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.941+0000 7fae41894b80 20 client.25522 trim_cache size 0 max 16384 2024-05-24T11:45:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:45:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:45:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:45:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:45:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 -- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fae100a1390 msgr2=0x7fae100a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fae100a1390 0x7fae100a3850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae20009c60 tx=0x7fae20009260 comp rx=0 tx=0).stop 2024-05-24T11:45:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 -- 172.21.15.39:0/1151580863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae30105e60 msgr2=0x7fae3019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae30105e60 0x7fae3019dca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae2800ef30 tx=0x7fae2800c3a0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 -- 172.21.15.39:0/1151580863 shutdown_connections 2024-05-24T11:45:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fae100a1390 0x7fae100a3850 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae30105e60 0x7fae3019dca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae300ffda0 0x7fae30199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 --2- 172.21.15.39:0/1151580863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30100760 0x7fae30199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 -- 172.21.15.39:0/1151580863 >> 172.21.15.39:0/1151580863 conn(0x7fae30076f10 msgr2=0x7fae30068350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 -- 172.21.15.39:0/1151580863 shutdown_connections 2024-05-24T11:45:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:11.942+0000 7fae356d9700 1 -- 172.21.15.39:0/1151580863 wait complete. 2024-05-24T11:45:13.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:13 smithi053 ceph-mon[47448]: pgmap v214: 97 pgs: 14 active+clean+snaptrim, 54 active+clean, 29 active+clean+snaptrim_wait; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:45:13.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:13 smithi137 ceph-mon[47907]: pgmap v214: 97 pgs: 14 active+clean+snaptrim, 54 active+clean, 29 active+clean+snaptrim_wait; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:45:13.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:13 smithi039 ceph-mon[42919]: pgmap v214: 97 pgs: 14 active+clean+snaptrim, 54 active+clean, 29 active+clean+snaptrim_wait; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:45:14.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 350.0 s, 1191.6 tps, lat 148.578 ms stddev 799.713 2024-05-24T11:45:15.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:15 smithi053 ceph-mon[47448]: pgmap v215: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 20 MiB/s wr, 1.35k op/s 2024-05-24T11:45:15.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:15 smithi053 ceph-mon[47448]: osdmap e82: 12 total, 12 up, 12 in 2024-05-24T11:45:15.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:15 smithi137 ceph-mon[47907]: pgmap v215: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 20 MiB/s wr, 1.35k op/s 2024-05-24T11:45:15.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:15 smithi137 ceph-mon[47907]: osdmap e82: 12 total, 12 up, 12 in 2024-05-24T11:45:16.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:15 smithi039 ceph-mon[42919]: pgmap v215: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 20 MiB/s wr, 1.35k op/s 2024-05-24T11:45:16.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:15 smithi039 ceph-mon[42919]: osdmap e82: 12 total, 12 up, 12 in 2024-05-24T11:45:17.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:17 smithi053 ceph-mon[47448]: pgmap v217: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 25 MiB/s wr, 1.66k op/s 2024-05-24T11:45:17.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:17 smithi137 ceph-mon[47907]: pgmap v217: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 25 MiB/s wr, 1.66k op/s 2024-05-24T11:45:18.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:17 smithi039 ceph-mon[42919]: pgmap v217: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 25 MiB/s wr, 1.66k op/s 2024-05-24T11:45:19.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 355.0 s, 1240.0 tps, lat 80.284 ms stddev 23.942 2024-05-24T11:45:19.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:19 smithi053 ceph-mon[47448]: pgmap v218: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-05-24T11:45:19.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:19 smithi137 ceph-mon[47907]: pgmap v218: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-05-24T11:45:20.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:19 smithi039 ceph-mon[42919]: pgmap v218: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-05-24T11:45:21.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:21 smithi053 ceph-mon[47448]: pgmap v219: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-05-24T11:45:21.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:21 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:45:21.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:21 smithi137 ceph-mon[47907]: pgmap v219: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-05-24T11:45:21.998 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:21 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:45:22.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:21 smithi039 ceph-mon[42919]: pgmap v219: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-05-24T11:45:22.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:21 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:45:22.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:22 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:45:22.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:22 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:45:22.993 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:22 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:45:22.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:22 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:45:22.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:22 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:45:22.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:22 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:45:23.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:22 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:45:23.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:22 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:45:23.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:22 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:45:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 360.0 s, 1111.2 tps, lat 90.311 ms stddev 31.201 2024-05-24T11:45:24.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:24 smithi053 ceph-mon[47448]: pgmap v220: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-05-24T11:45:24.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:24 smithi137 ceph-mon[47907]: pgmap v220: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-05-24T11:45:24.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:24 smithi039 ceph-mon[42919]: pgmap v220: 97 pgs: 97 active+clean; 4.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 25 MiB/s wr, 1.40k op/s 2024-05-24T11:45:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.125+0000 7f841659c700 1 -- 172.21.15.39:0/1107230567 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800ba10 con 0x7f8418105f20 2024-05-24T11:45:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.125+0000 7f841d0ee700 1 -- 172.21.15.39:0/1107230567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418105f20 msgr2=0x7f8418106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.125+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1107230567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418105f20 0x7f8418106300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8408009990 tx=0x7f8408009ca0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.125+0000 7f841d0ee700 1 -- 172.21.15.39:0/1107230567 shutdown_connections 2024-05-24T11:45:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.125+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1107230567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8418100c40 0x7f841810cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.125+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1107230567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418105f20 0x7f8418106300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.125+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1107230567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418100280 0x7f8418100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.125+0000 7f841d0ee700 1 -- 172.21.15.39:0/1107230567 >> 172.21.15.39:0/1107230567 conn(0x7f8418076bc0 msgr2=0x7f8418076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.130+0000 7f841d0ee700 1 -- 172.21.15.39:0/1107230567 shutdown_connections 2024-05-24T11:45:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.130+0000 7f841d0ee700 1 -- 172.21.15.39:0/1107230567 wait complete. 2024-05-24T11:45:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.131+0000 7f841d0ee700 1 Processor -- start 2024-05-24T11:45:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.131+0000 7f841d0ee700 1 -- start start 2024-05-24T11:45:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.131+0000 7f841d0ee700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418100280 0x7f8418113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.131+0000 7f841d0ee700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8418100c40 0x7f8418110580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f841d0ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418105f20 0x7f8418110ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f841d0ee700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84181110c0 con 0x7f8418100c40 2024-05-24T11:45:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f841d0ee700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418111200 con 0x7f8418100280 2024-05-24T11:45:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f841d0ee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418111390 con 0x7f8418105f20 2024-05-24T11:45:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f8416d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418100280 0x7f8418113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f8416d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418100280 0x7f8418113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:45056/0 (socket says 172.21.15.39:45056) 2024-05-24T11:45:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f8416d9d700 1 -- 172.21.15.39:0/1072356995 learned_addr learned my addr 172.21.15.39:0/1072356995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f840ffff700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8418100c40 0x7f8418110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f841759e700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418105f20 0x7f8418110ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f8416d9d700 1 -- 172.21.15.39:0/1072356995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418105f20 msgr2=0x7f8418110ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f8416d9d700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418105f20 0x7f8418110ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f8416d9d700 1 -- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8418100c40 msgr2=0x7f8418110580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f8416d9d700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8418100c40 0x7f8418110580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.132+0000 7f8416d9d700 1 -- 172.21.15.39:0/1072356995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400009520 con 0x7f8418100280 2024-05-24T11:45:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.133+0000 7f8416d9d700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418100280 0x7f8418113ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8408004760 tx=0x7f8408004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.133+0000 7f8414d99700 1 -- 172.21.15.39:0/1072356995 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840801d070 con 0x7f8418100280 2024-05-24T11:45:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.133+0000 7f8414d99700 1 -- 172.21.15.39:0/1072356995 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8408005a00 con 0x7f8418100280 2024-05-24T11:45:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.133+0000 7f8414d99700 1 -- 172.21.15.39:0/1072356995 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408021e00 con 0x7f8418100280 2024-05-24T11:45:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.134+0000 7f841d0ee700 1 -- 172.21.15.39:0/1072356995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84080095f0 con 0x7f8418100280 2024-05-24T11:45:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.134+0000 7f841d0ee700 1 -- 172.21.15.39:0/1072356995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841806acc0 con 0x7f8418100280 2024-05-24T11:45:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.137+0000 7f8414d99700 1 -- 172.21.15.39:0/1072356995 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8408021410 con 0x7f8418100280 2024-05-24T11:45:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.138+0000 7f8414d99700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f84040a1390 0x7f84040a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.138+0000 7f8414d99700 1 -- 172.21.15.39:0/1072356995 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f84080cda60 con 0x7f8418100280 2024-05-24T11:45:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.138+0000 7f840ffff700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f84040a1390 0x7f84040a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.139+0000 7f840ffff700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f84040a1390 0x7f84040a3850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84181117e0 tx=0x7f8400009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.139+0000 7f84292a9b80 1 Processor -- start 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.139+0000 7f84292a9b80 1 -- start start 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 0x5616192a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f84292a9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616191fc0f0 con 0x5616192aa950 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f84292a9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616191f9ab0 con 0x5616192a5480 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f84292a9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616191f53a0 con 0x5616192ab270 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f840ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f840ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192a4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:36498/0 (socket says 172.21.15.39:36498) 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f840ffff700 1 -- 172.21.15.39:0/520713351 learned_addr learned my addr 172.21.15.39:0/520713351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f8416d9d700 1 --2- 172.21.15.39:0/520713351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.140+0000 7f841759e700 1 --2- 172.21.15.39:0/520713351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 0x5616192a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.141+0000 7f8416d9d700 1 -- 172.21.15.39:0/520713351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 msgr2=0x5616192a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.141+0000 7f8416d9d700 1 --2- 172.21.15.39:0/520713351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.141+0000 7f8416d9d700 1 -- 172.21.15.39:0/520713351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 msgr2=0x5616192a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.141+0000 7f8416d9d700 1 --2- 172.21.15.39:0/520713351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 0x5616192a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.141+0000 7f8416d9d700 1 -- 172.21.15.39:0/520713351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561619076350 con 0x5616192aa950 2024-05-24T11:45:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.141+0000 7f8416d9d700 1 --2- 172.21.15.39:0/520713351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192aad30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f84080cb160 tx=0x7f840809a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb73a21afb7a9f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.141+0000 7f840d7fa700 1 -- 172.21.15.39:0/520713351 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840809abb0 con 0x5616192aa950 2024-05-24T11:45:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.141+0000 7f840d7fa700 1 -- 172.21.15.39:0/520713351 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f840809e9e0 con 0x5616192aa950 2024-05-24T11:45:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.141+0000 7f840d7fa700 1 -- 172.21.15.39:0/520713351 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840809a300 con 0x5616192aa950 2024-05-24T11:45:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.142+0000 7f84292a9b80 1 -- 172.21.15.39:0/520713351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 msgr2=0x5616192aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.142+0000 7f84292a9b80 1 --2- 172.21.15.39:0/520713351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192aad30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f84080cb160 tx=0x7f840809a770 comp rx=0 tx=0).stop 2024-05-24T11:45:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.142+0000 7f84292a9b80 1 -- 172.21.15.39:0/520713351 shutdown_connections 2024-05-24T11:45:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.142+0000 7f84292a9b80 1 --2- 172.21.15.39:0/520713351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 0x5616192a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.142+0000 7f84292a9b80 1 --2- 172.21.15.39:0/520713351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192aad30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.142+0000 7f84292a9b80 1 --2- 172.21.15.39:0/520713351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.142+0000 7f84292a9b80 1 -- 172.21.15.39:0/520713351 >> 172.21.15.39:0/520713351 conn(0x5616192031b0 msgr2=0x56161929f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.143+0000 7f84292a9b80 1 -- 172.21.15.39:0/520713351 shutdown_connections 2024-05-24T11:45:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.143+0000 7f84292a9b80 1 -- 172.21.15.39:0/520713351 wait complete. 2024-05-24T11:45:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.143+0000 7f84292a9b80 1 -- start start 2024-05-24T11:45:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.143+0000 7f84292a9b80 1 Processor -- start 2024-05-24T11:45:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x5616192b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192aa950 0x5616192ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192ab270 0x5616192aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f84292a9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619191500 con 0x5616192a5480 2024-05-24T11:45:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f84292a9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616191911a0 con 0x5616192ab270 2024-05-24T11:45:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f84292a9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619078640 con 0x5616192aa950 2024-05-24T11:45:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f84292a9b80 10 client.0 _ll_register_callbacks cb 0x56161919aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:45:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f840ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192aa950 0x5616192ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f840ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192aa950 0x5616192ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:36500/0 (socket says 172.21.15.39:36500) 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f840ffff700 1 -- 172.21.15.39:0/186389694 learned_addr learned my addr 172.21.15.39:0/186389694 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f841759e700 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192ab270 0x5616192aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f8416d9d700 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x5616192b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f840ffff700 1 -- 172.21.15.39:0/186389694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192ab270 msgr2=0x5616192aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f840ffff700 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192ab270 0x5616192aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f840ffff700 1 -- 172.21.15.39:0/186389694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 msgr2=0x5616192b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f840ffff700 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x5616192b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.144+0000 7f840ffff700 1 -- 172.21.15.39:0/186389694 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841806acc0 con 0x5616192aa950 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.145+0000 7f840ffff700 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192aa950 0x5616192ae450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8400005d90 tx=0x7f84000047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94b4aeb371a2be77 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.145+0000 7f83fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:45:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.145+0000 7f83fe7fc700 1 -- 172.21.15.39:0/186389694 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400004230 con 0x5616192aa950 2024-05-24T11:45:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.145+0000 7f84292a9b80 1 -- 172.21.15.39:0/186389694 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56161914e210 con 0x5616192aa950 2024-05-24T11:45:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.145+0000 7f83fe7fc700 1 -- 172.21.15.39:0/186389694 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f840001e320 con 0x5616192aa950 2024-05-24T11:45:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.145+0000 7f83fe7fc700 1 -- 172.21.15.39:0/186389694 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8400004390 con 0x5616192aa950 2024-05-24T11:45:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.145+0000 7f84292a9b80 10 client.25531 fetch_fsmap learned FSMap version 27 2024-05-24T11:45:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.145+0000 7f84292a9b80 1 -- 172.21.15.39:0/186389694 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561619022b00 con 0x5616192aa950 2024-05-24T11:45:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.146+0000 7f83fe7fc700 1 -- 172.21.15.39:0/186389694 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56161914e210 con 0x5616192aa950 2024-05-24T11:45:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.147+0000 7f83fe7fc700 1 -- 172.21.15.39:0/186389694 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8400003b70 con 0x5616192aa950 2024-05-24T11:45:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.147+0000 7f84292a9b80 10 client.25531 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:45:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.147+0000 7f84292a9b80 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:45:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.147+0000 7f84292a9b80 20 client.25531 populate_metadata read hostname 'smithi039' 2024-05-24T11:45:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.147+0000 7f84292a9b80 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5616192be750 0x5616192c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.147+0000 7f84292a9b80 4 client.25531 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:45:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.147+0000 7f84292a9b80 1 -- 172.21.15.39:0/186389694 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616192747b0 con 0x5616192be750 2024-05-24T11:45:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.148+0000 7f8416d9d700 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5616192be750 0x5616192c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.148+0000 7f8416d9d700 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5616192be750 0x5616192c0b70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2441e4e2332c77d1 server_cookie=f16f45353a982bc1 in_seq=0 out_seq=0 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.148+0000 7f83fe7fc700 10 client.25531 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f83fe7fc700 1 -- 172.21.15.39:0/186389694 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616192747b0 con 0x5616192be750 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f83fe7fc700 10 client.25531 handle_command_reply: tid=0 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f84292a9b80 1 client.25531 shutdown 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f84292a9b80 1 -- 172.21.15.39:0/186389694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192aa950 msgr2=0x5616192ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f84292a9b80 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192aa950 0x5616192ae450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8400005d90 tx=0x7f84000047c0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f84292a9b80 1 -- 172.21.15.39:0/186389694 shutdown_connections 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f84292a9b80 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5616192be750 0x5616192c0b70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f84292a9b80 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192ab270 0x5616192aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f84292a9b80 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x5616192b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f84292a9b80 1 --2- 172.21.15.39:0/186389694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192aa950 0x5616192ae450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.149+0000 7f84292a9b80 1 -- 172.21.15.39:0/186389694 >> 172.21.15.39:0/186389694 conn(0x5616192031b0 msgr2=0x5616192a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.150+0000 7f83fe7fc700 0 client.25531 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.150+0000 7f84292a9b80 1 -- 172.21.15.39:0/186389694 shutdown_connections 2024-05-24T11:45:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.150+0000 7f84292a9b80 1 -- 172.21.15.39:0/186389694 wait complete. 2024-05-24T11:45:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.150+0000 7f84292a9b80 20 client.25531 trim_cache size 0 max 16384 2024-05-24T11:45:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.174+0000 7f84292a9b80 1 Processor -- start 2024-05-24T11:45:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.174+0000 7f84292a9b80 1 -- start start 2024-05-24T11:45:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.174+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x56161929f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.174+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192aa950 0x5616192a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.174+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.174+0000 7f84292a9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616192bb370 con 0x5616192a5480 2024-05-24T11:45:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.174+0000 7f84292a9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619297a40 con 0x5616192aa950 2024-05-24T11:45:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.174+0000 7f84292a9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616192a0760 con 0x5616192ab270 2024-05-24T11:45:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f8416d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x56161929f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f8416d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x56161929f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36424/0 (socket says 172.21.15.39:36424) 2024-05-24T11:45:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f8416d9d700 1 -- 172.21.15.39:0/2122263068 learned_addr learned my addr 172.21.15.39:0/2122263068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f840ffff700 1 --2- 172.21.15.39:0/2122263068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192aa950 0x5616192a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f841759e700 1 --2- 172.21.15.39:0/2122263068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f8416d9d700 1 -- 172.21.15.39:0/2122263068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 msgr2=0x5616192a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f8416d9d700 1 --2- 172.21.15.39:0/2122263068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f8416d9d700 1 -- 172.21.15.39:0/2122263068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192aa950 msgr2=0x5616192a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f8416d9d700 1 --2- 172.21.15.39:0/2122263068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192aa950 0x5616192a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f8416d9d700 1 -- 172.21.15.39:0/2122263068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616191bb500 con 0x5616192a5480 2024-05-24T11:45:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.175+0000 7f8416d9d700 1 --2- 172.21.15.39:0/2122263068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x56161929f910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f84080970b0 tx=0x7f8408009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a67a723acb7cc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.176+0000 7f840dffb700 1 -- 172.21.15.39:0/2122263068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080b7070 con 0x5616192a5480 2024-05-24T11:45:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.176+0000 7f840dffb700 1 -- 172.21.15.39:0/2122263068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84080a2fa0 con 0x5616192a5480 2024-05-24T11:45:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.176+0000 7f840dffb700 1 -- 172.21.15.39:0/2122263068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840809a0a0 con 0x5616192a5480 2024-05-24T11:45:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.176+0000 7f84292a9b80 1 -- 172.21.15.39:0/2122263068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 msgr2=0x56161929f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.176+0000 7f84292a9b80 1 --2- 172.21.15.39:0/2122263068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x56161929f910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f84080970b0 tx=0x7f8408009310 comp rx=0 tx=0).stop 2024-05-24T11:45:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.177+0000 7f84292a9b80 1 -- 172.21.15.39:0/2122263068 shutdown_connections 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.177+0000 7f84292a9b80 1 --2- 172.21.15.39:0/2122263068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.177+0000 7f84292a9b80 1 --2- 172.21.15.39:0/2122263068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192a5480 0x56161929f910 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.177+0000 7f84292a9b80 1 --2- 172.21.15.39:0/2122263068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192aa950 0x5616192a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.177+0000 7f84292a9b80 1 -- 172.21.15.39:0/2122263068 >> 172.21.15.39:0/2122263068 conn(0x5616192031b0 msgr2=0x5616192a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.177+0000 7f84292a9b80 1 -- 172.21.15.39:0/2122263068 shutdown_connections 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.177+0000 7f84292a9b80 1 -- 172.21.15.39:0/2122263068 wait complete. 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.177+0000 7f84292a9b80 1 -- start start 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.177+0000 7f84292a9b80 1 Processor -- start 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 0x5616192b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f84292a9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f84292a9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616192b4780 con 0x5616192aa950 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f84292a9b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619313c50 con 0x5616192a5480 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f84292a9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619313d90 con 0x5616192ab270 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f84292a9b80 10 client.0 _ll_register_callbacks cb 0x56161919aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f8416d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 0x5616192b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f840ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f840ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36428/0 (socket says 172.21.15.39:36428) 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f840ffff700 1 -- 172.21.15.39:0/3795782732 learned_addr learned my addr 172.21.15.39:0/3795782732 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f841759e700 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f841759e700 1 -- 172.21.15.39:0/3795782732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 msgr2=0x5616192b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f841759e700 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 0x5616192b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f841759e700 1 -- 172.21.15.39:0/3795782732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 msgr2=0x5616192b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f841759e700 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.178+0000 7f841759e700 1 -- 172.21.15.39:0/3795782732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561619076350 con 0x5616192ab270 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.179+0000 7f841759e700 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192b4070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5616192acc30 tx=0x7f8410007ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c21abdd3ce5fa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.179+0000 7f83fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.179+0000 7f83fe7fc700 1 -- 172.21.15.39:0/3795782732 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410003c00 con 0x5616192ab270 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.179+0000 7f83fe7fc700 1 -- 172.21.15.39:0/3795782732 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8410003d60 con 0x5616192ab270 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.179+0000 7f84292a9b80 1 -- 172.21.15.39:0/3795782732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f840809e9e0 con 0x5616192ab270 2024-05-24T11:45:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.179+0000 7f83fe7fc700 1 -- 172.21.15.39:0/3795782732 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84100177f0 con 0x5616192ab270 2024-05-24T11:45:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.182+0000 7f84292a9b80 10 client.25537 fetch_fsmap learned FSMap version 27 2024-05-24T11:45:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.182+0000 7f84292a9b80 1 -- 172.21.15.39:0/3795782732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56161907ac70 con 0x5616192ab270 2024-05-24T11:45:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.184+0000 7f83fe7fc700 1 -- 172.21.15.39:0/3795782732 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f840809e9e0 con 0x5616192ab270 2024-05-24T11:45:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.184+0000 7f83fe7fc700 1 -- 172.21.15.39:0/3795782732 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8410027070 con 0x5616192ab270 2024-05-24T11:45:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.184+0000 7f84292a9b80 10 client.25537 fetch_fsmap finished waiting for FSMap version 27 2024-05-24T11:45:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.184+0000 7f84292a9b80 10 client.25537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:45:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.184+0000 7f84292a9b80 20 client.25537 populate_metadata read hostname 'smithi039' 2024-05-24T11:45:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.184+0000 7f84292a9b80 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561619315650 0x561619317a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.184+0000 7f84292a9b80 4 client.25537 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:45:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.184+0000 7f84292a9b80 1 -- 172.21.15.39:0/3795782732 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616191ede60 con 0x561619315650 2024-05-24T11:45:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.184+0000 7f8416d9d700 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561619315650 0x561619317a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.185+0000 7f8416d9d700 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561619315650 0x561619317a70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724f5e4da834a866 server_cookie=fd703a04b0252a8c in_seq=0 out_seq=0 2024-05-24T11:45:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.185+0000 7f83fe7fc700 10 client.25537 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.211+0000 7f83fe7fc700 1 -- 172.21.15.39:0/3795782732 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616191ede60 con 0x561619315650 2024-05-24T11:45:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.211+0000 7f83fe7fc700 10 client.25537 handle_command_reply: tid=0 2024-05-24T11:45:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.211+0000 7f84292a9b80 1 client.25537 shutdown 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.212+0000 7f84292a9b80 1 -- 172.21.15.39:0/3795782732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 msgr2=0x5616192b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.212+0000 7f84292a9b80 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192b4070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5616192acc30 tx=0x7f8410007ed0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f84292a9b80 1 -- 172.21.15.39:0/3795782732 shutdown_connections 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f84292a9b80 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616192ab270 0x5616192b4070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f84292a9b80 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561619315650 0x561619317a70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f84292a9b80 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5616192a5480 0x5616192b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f84292a9b80 1 --2- 172.21.15.39:0/3795782732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616192aa950 0x5616192b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f83fe7fc700 0 client.25537 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f84292a9b80 1 -- 172.21.15.39:0/3795782732 >> 172.21.15.39:0/3795782732 conn(0x5616192031b0 msgr2=0x5616192ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f84292a9b80 1 -- 172.21.15.39:0/3795782732 shutdown_connections 2024-05-24T11:45:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f84292a9b80 1 -- 172.21.15.39:0/3795782732 wait complete. 2024-05-24T11:45:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.213+0000 7f84292a9b80 20 client.25537 trim_cache size 0 max 16384 2024-05-24T11:45:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:45:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:45:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:45:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:45:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 -- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f84040a1390 msgr2=0x7f84040a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f84040a1390 0x7f84040a3850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84181117e0 tx=0x7f8400009260 comp rx=0 tx=0).stop 2024-05-24T11:45:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 -- 172.21.15.39:0/1072356995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418100280 msgr2=0x7f8418113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418100280 0x7f8418113ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8408004760 tx=0x7f8408004790 comp rx=0 tx=0).stop 2024-05-24T11:45:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 -- 172.21.15.39:0/1072356995 shutdown_connections 2024-05-24T11:45:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418105f20 0x7f8418110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f84040a1390 0x7f84040a3850 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8418100280 0x7f8418113ff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 --2- 172.21.15.39:0/1072356995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8418100c40 0x7f8418110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.216+0000 7f841d0ee700 1 -- 172.21.15.39:0/1072356995 >> 172.21.15.39:0/1072356995 conn(0x7f8418076bc0 msgr2=0x7f84180fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.217+0000 7f841d0ee700 1 -- 172.21.15.39:0/1072356995 shutdown_connections 2024-05-24T11:45:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:25.217+0000 7f841d0ee700 1 -- 172.21.15.39:0/1072356995 wait complete. 2024-05-24T11:45:25.998 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:45:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setuser ceph since I am not root 2024-05-24T11:45:25.998 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:45:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setgroup ceph since I am not root 2024-05-24T11:45:25.998 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:45:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: starting mds.f at 2024-05-24T11:45:26.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:26 smithi053 ceph-mon[47448]: pgmap v221: 97 pgs: 97 active+clean; 5.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 23 MiB/s wr, 1.45k op/s 2024-05-24T11:45:26.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:26 smithi053 ceph-mon[47448]: daemon mds.f restarted 2024-05-24T11:45:26.492 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:26 smithi053 ceph-mon[47448]: osdmap e83: 12 total, 12 up, 12 in 2024-05-24T11:45:26.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:26 smithi137 ceph-mon[47907]: pgmap v221: 97 pgs: 97 active+clean; 5.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 23 MiB/s wr, 1.45k op/s 2024-05-24T11:45:26.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:26 smithi137 ceph-mon[47907]: daemon mds.f restarted 2024-05-24T11:45:26.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:26 smithi137 ceph-mon[47907]: osdmap e83: 12 total, 12 up, 12 in 2024-05-24T11:45:26.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:26 smithi039 ceph-mon[42919]: pgmap v221: 97 pgs: 97 active+clean; 5.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 23 MiB/s wr, 1.45k op/s 2024-05-24T11:45:26.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:26 smithi039 ceph-mon[42919]: daemon mds.f restarted 2024-05-24T11:45:26.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:26 smithi039 ceph-mon[42919]: osdmap e83: 12 total, 12 up, 12 in 2024-05-24T11:45:27.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:27 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6834/1371926690,v1:172.21.15.137:6835/1371926690] up:boot 2024-05-24T11:45:27.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:27 smithi053 ceph-mon[47448]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:45:27.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:45:27.492 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:27 smithi053 ceph-mon[47448]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:45:27.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:27 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6834/1371926690,v1:172.21.15.137:6835/1371926690] up:boot 2024-05-24T11:45:27.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:27 smithi137 ceph-mon[47907]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:45:27.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:45:27.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:27 smithi137 ceph-mon[47907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:45:27.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:27 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6834/1371926690,v1:172.21.15.137:6835/1371926690] up:boot 2024-05-24T11:45:27.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:27 smithi039 ceph-mon[42919]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:45:27.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:45:27.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:27 smithi039 ceph-mon[42919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:45:28.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:28 smithi053 ceph-mon[47448]: pgmap v223: 97 pgs: 97 active+clean; 5.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 23 MiB/s wr, 1.45k op/s 2024-05-24T11:45:28.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:28 smithi137 ceph-mon[47907]: pgmap v223: 97 pgs: 97 active+clean; 5.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 23 MiB/s wr, 1.45k op/s 2024-05-24T11:45:28.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:28 smithi039 ceph-mon[42919]: pgmap v223: 97 pgs: 97 active+clean; 5.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 23 MiB/s wr, 1.45k op/s 2024-05-24T11:45:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 365.0 s, 1118.8 tps, lat 89.544 ms stddev 31.254 2024-05-24T11:45:30.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:30 smithi053 ceph-mon[47448]: pgmap v224: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2024-05-24T11:45:30.498 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:30 smithi137 ceph-mon[47907]: pgmap v224: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2024-05-24T11:45:30.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:30 smithi039 ceph-mon[42919]: pgmap v224: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2024-05-24T11:45:32.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:32 smithi053 ceph-mon[47448]: pgmap v225: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2024-05-24T11:45:32.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:32 smithi137 ceph-mon[47907]: pgmap v225: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2024-05-24T11:45:32.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:32 smithi039 ceph-mon[42919]: pgmap v225: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2024-05-24T11:45:33.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:33 smithi053 ceph-mon[47448]: pgmap v226: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2024-05-24T11:45:33.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:33 smithi137 ceph-mon[47907]: pgmap v226: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2024-05-24T11:45:33.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:33 smithi039 ceph-mon[42919]: pgmap v226: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2024-05-24T11:45:34.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 370.0 s, 1235.0 tps, lat 80.816 ms stddev 24.852 2024-05-24T11:45:35.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:35 smithi053 ceph-mon[47448]: pgmap v227: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 20 MiB/s wr, 1.45k op/s 2024-05-24T11:45:35.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:35 smithi137 ceph-mon[47907]: pgmap v227: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 20 MiB/s wr, 1.45k op/s 2024-05-24T11:45:35.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:35 smithi039 ceph-mon[42919]: pgmap v227: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 20 MiB/s wr, 1.45k op/s 2024-05-24T11:45:37.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:37 smithi053 ceph-mon[47448]: pgmap v228: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-05-24T11:45:37.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:37 smithi137 ceph-mon[47907]: pgmap v228: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-05-24T11:45:37.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:37 smithi039 ceph-mon[42919]: pgmap v228: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-05-24T11:45:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.393+0000 7fd5d3fec700 1 -- 172.21.15.39:0/725975547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc104de0 msgr2=0x7fd5cc1051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.393+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/725975547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc104de0 0x7fd5cc1051c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80099c0 tx=0x7fd5c8009cd0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.393+0000 7fd5d3fec700 1 -- 172.21.15.39:0/725975547 shutdown_connections 2024-05-24T11:45:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.393+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/725975547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5cc100a40 0x7fd5cc10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.393+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/725975547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc104de0 0x7fd5cc1051c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.393+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/725975547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5cc100080 0x7fd5cc100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.393+0000 7fd5d3fec700 1 -- 172.21.15.39:0/725975547 >> 172.21.15.39:0/725975547 conn(0x7fd5cc076c10 msgr2=0x7fd5cc077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.393+0000 7fd5d3fec700 1 -- 172.21.15.39:0/725975547 shutdown_connections 2024-05-24T11:45:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.393+0000 7fd5d3fec700 1 -- 172.21.15.39:0/725975547 wait complete. 2024-05-24T11:45:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.394+0000 7fd5d3fec700 1 Processor -- start 2024-05-24T11:45:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.394+0000 7fd5d3fec700 1 -- start start 2024-05-24T11:45:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.394+0000 7fd5d3fec700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5cc100080 0x7fd5cc111e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.394+0000 7fd5d3fec700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc100a40 0x7fd5cc1105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.394+0000 7fd5d3fec700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5cc104de0 0x7fd5cc110af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.394+0000 7fd5d3fec700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc111030 con 0x7fd5cc100a40 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.394+0000 7fd5d3fec700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc1111a0 con 0x7fd5cc104de0 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.394+0000 7fd5d3fec700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc111310 con 0x7fd5cc100080 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1587700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc100a40 0x7fd5cc1105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1d88700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5cc100080 0x7fd5cc111e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1587700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc100a40 0x7fd5cc1105b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35234/0 (socket says 172.21.15.39:35234) 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1587700 1 -- 172.21.15.39:0/3356556798 learned_addr learned my addr 172.21.15.39:0/3356556798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1d88700 1 -- 172.21.15.39:0/3356556798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5cc104de0 msgr2=0x7fd5cc110af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5cc104de0 0x7fd5cc110af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1d88700 1 -- 172.21.15.39:0/3356556798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc100a40 msgr2=0x7fd5cc1105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc100a40 0x7fd5cc1105b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1d88700 1 -- 172.21.15.39:0/3356556798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80095f0 con 0x7fd5cc100080 2024-05-24T11:45:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.395+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5cc100080 0x7fd5cc111e80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800b580 tx=0x7fd5c8005460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.396+0000 7fd5c2ffd700 1 -- 172.21.15.39:0/3356556798 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c801d070 con 0x7fd5cc100080 2024-05-24T11:45:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.396+0000 7fd5c2ffd700 1 -- 172.21.15.39:0/3356556798 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5c8004ba0 con 0x7fd5cc100080 2024-05-24T11:45:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.396+0000 7fd5c2ffd700 1 -- 172.21.15.39:0/3356556798 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800fab0 con 0x7fd5cc100080 2024-05-24T11:45:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.396+0000 7fd5d3fec700 1 -- 172.21.15.39:0/3356556798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5cc1aa120 con 0x7fd5cc100080 2024-05-24T11:45:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.396+0000 7fd5d3fec700 1 -- 172.21.15.39:0/3356556798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5cc1aa4f0 con 0x7fd5cc100080 2024-05-24T11:45:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.399+0000 7fd5c2ffd700 1 -- 172.21.15.39:0/3356556798 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd5c80046b0 con 0x7fd5cc100080 2024-05-24T11:45:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.400+0000 7fd5c2ffd700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5b80a1350 0x7fd5b80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.400+0000 7fd5c2ffd700 1 -- 172.21.15.39:0/3356556798 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd5c80cdb70 con 0x7fd5cc100080 2024-05-24T11:45:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.400+0000 7fd5e01a7b80 1 Processor -- start 2024-05-24T11:45:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.400+0000 7fd5d1587700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5b80a1350 0x7fd5b80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.400+0000 7fd5e01a7b80 1 -- start start 2024-05-24T11:45:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3a950 0x563e16e3ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3b270 0x563e16e34f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e358a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16d8c0f0 con 0x563e16e3a950 2024-05-24T11:45:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16d89ab0 con 0x563e16e3b270 2024-05-24T11:45:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16d853a0 con 0x563e16e35480 2024-05-24T11:45:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d2589700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d2589700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e358a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:48362/0 (socket says 172.21.15.39:48362) 2024-05-24T11:45:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d2589700 1 -- 172.21.15.39:0/410277530 learned_addr learned my addr 172.21.15.39:0/410277530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d1587700 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3b270 0x563e16e34f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3a950 0x563e16e3ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d2589700 1 -- 172.21.15.39:0/410277530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3b270 msgr2=0x563e16e34f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d2589700 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3b270 0x563e16e34f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d2589700 1 -- 172.21.15.39:0/410277530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3a950 msgr2=0x563e16e3ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d2589700 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3a950 0x563e16e3ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.401+0000 7fd5d2589700 1 -- 172.21.15.39:0/410277530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e16c06350 con 0x563e16e35480 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.402+0000 7fd5d1587700 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3b270 0x563e16e34f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.402+0000 7fd5d1587700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5b80a1350 0x7fd5b80a3810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc111940 tx=0x7fd5bc004080 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.402+0000 7fd5d2589700 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e358a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e380 tx=0x7fd5c400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dfbcd9744ffbdc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.402+0000 7fd5b7fff700 1 -- 172.21.15.39:0/410277530 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4018070 con 0x563e16e35480 2024-05-24T11:45:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.402+0000 7fd5b7fff700 1 -- 172.21.15.39:0/410277530 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5c4004bb0 con 0x563e16e35480 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.402+0000 7fd5b7fff700 1 -- 172.21.15.39:0/410277530 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4010040 con 0x563e16e35480 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.403+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/410277530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 msgr2=0x563e16e358a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.403+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e358a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e380 tx=0x7fd5c400b040 comp rx=0 tx=0).stop 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.403+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/410277530 shutdown_connections 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.403+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e358a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.403+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3a950 0x563e16e3ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.403+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/410277530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3b270 0x563e16e34f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.403+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/410277530 >> 172.21.15.39:0/410277530 conn(0x563e16d931b0 msgr2=0x563e16e2f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.403+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/410277530 shutdown_connections 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.403+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/410277530 wait complete. 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.404+0000 7fd5e01a7b80 1 -- start start 2024-05-24T11:45:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.404+0000 7fd5e01a7b80 1 Processor -- start 2024-05-24T11:45:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.404+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e35480 0x563e16e41160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.404+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e3e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.404+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e3b270 0x563e16e3ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.404+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16d21500 con 0x563e16e35480 2024-05-24T11:45:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.404+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16d211a0 con 0x563e16e3a950 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.404+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16c08640 con 0x563e16e3b270 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.404+0000 7fd5e01a7b80 10 client.0 _ll_register_callbacks cb 0x563e16d2aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d2589700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e3b270 0x563e16e3ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1d88700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e35480 0x563e16e41160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1587700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e3e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1d88700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e35480 0x563e16e41160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35258/0 (socket says 172.21.15.39:35258) 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1587700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e3e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:58952/0 (socket says 172.21.15.39:58952) 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1d88700 1 -- 172.21.15.39:0/1575870248 learned_addr learned my addr 172.21.15.39:0/1575870248 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1d88700 1 -- 172.21.15.39:0/1575870248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e3b270 msgr2=0x563e16e3ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e3b270 0x563e16e3ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1d88700 1 -- 172.21.15.39:0/1575870248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 msgr2=0x563e16e3e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e3e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.405+0000 7fd5d1d88700 1 -- 172.21.15.39:0/1575870248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c400e0c0 con 0x563e16e35480 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.406+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e35480 0x563e16e41160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80cb3b0 tx=0x7fd5c809a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1f2079ae9f46c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.406+0000 7fd5b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.406+0000 7fd5b57fa700 1 -- 172.21.15.39:0/1575870248 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80b2070 con 0x563e16e35480 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.406+0000 7fd5b57fa700 1 -- 172.21.15.39:0/1575870248 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd5c809ad50 con 0x563e16e35480 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.406+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1575870248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e16cde210 con 0x563e16e35480 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5b57fa700 1 -- 172.21.15.39:0/1575870248 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5c809eb80 con 0x563e16e35480 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5e01a7b80 10 client.15942 fetch_fsmap learned FSMap version 29 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1575870248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563e16bb2b00 con 0x563e16e35480 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5b57fa700 1 -- 172.21.15.39:0/1575870248 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5c807d080 con 0x563e16e35480 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5b57fa700 1 -- 172.21.15.39:0/1575870248 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5c809afb0 con 0x563e16e35480 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5e01a7b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5e01a7b80 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5e01a7b80 20 client.15942 populate_metadata read hostname 'smithi039' 2024-05-24T11:45:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563e16e5f7e0 0x563e16e61c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5e01a7b80 4 client.15942 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:45:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1575870248 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e16e047b0 con 0x563e16e5f7e0 2024-05-24T11:45:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.407+0000 7fd5d1587700 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563e16e5f7e0 0x563e16e61c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.408+0000 7fd5b57fa700 10 client.15942 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.408+0000 7fd5d1587700 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563e16e5f7e0 0x563e16e61c00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=490c6d13f96a154f server_cookie=6821991c4b114926 in_seq=0 out_seq=0 2024-05-24T11:45:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5b57fa700 1 -- 172.21.15.39:0/1575870248 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e16e047b0 con 0x563e16e5f7e0 2024-05-24T11:45:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5b57fa700 10 client.15942 handle_command_reply: tid=0 2024-05-24T11:45:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5e01a7b80 1 client.15942 shutdown 2024-05-24T11:45:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1575870248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e35480 msgr2=0x563e16e41160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e35480 0x563e16e41160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80cb3b0 tx=0x7fd5c809a7a0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1575870248 shutdown_connections 2024-05-24T11:45:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e3b270 0x563e16e3ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563e16e5f7e0 0x563e16e61c00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e35480 0x563e16e41160 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5b57fa700 0 client.15942 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1575870248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e3e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.409+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1575870248 >> 172.21.15.39:0/1575870248 conn(0x563e16d931b0 msgr2=0x563e16e394a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.410+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1575870248 shutdown_connections 2024-05-24T11:45:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.410+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1575870248 wait complete. 2024-05-24T11:45:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.410+0000 7fd5e01a7b80 20 client.15942 trim_cache size 0 max 16384 2024-05-24T11:45:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.433+0000 7fd5e01a7b80 1 Processor -- start 2024-05-24T11:45:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.433+0000 7fd5e01a7b80 1 -- start start 2024-05-24T11:45:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.433+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e61bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.433+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e2f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.433+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3b270 0x563e16e2be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.433+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16e66e10 con 0x563e16e3b270 2024-05-24T11:45:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.433+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16e2c340 con 0x563e16e3a950 2024-05-24T11:45:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.433+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16e34080 con 0x563e16e35480 2024-05-24T11:45:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.433+0000 7fd5d1d88700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e61bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.434+0000 7fd5d1d88700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e61bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:48380/0 (socket says 172.21.15.39:48380) 2024-05-24T11:45:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.434+0000 7fd5d1d88700 1 -- 172.21.15.39:0/1502337272 learned_addr learned my addr 172.21.15.39:0/1502337272 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.434+0000 7fd5d1d88700 1 -- 172.21.15.39:0/1502337272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 msgr2=0x563e16e2f910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.434+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/1502337272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e2f910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.434+0000 7fd5d1d88700 1 -- 172.21.15.39:0/1502337272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3b270 msgr2=0x563e16e2be00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.434+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/1502337272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3b270 0x563e16e2be00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.434+0000 7fd5d1d88700 1 -- 172.21.15.39:0/1502337272 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e16d4b500 con 0x563e16e35480 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.434+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/1502337272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e61bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009990 tx=0x7fd5c80b9170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=248f923d89566714 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.434+0000 7fd5c0ff9700 1 -- 172.21.15.39:0/1502337272 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80b2070 con 0x563e16e35480 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.435+0000 7fd5c0ff9700 1 -- 172.21.15.39:0/1502337272 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563e16cde210 con 0x563e16e35480 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.435+0000 7fd5c0ff9700 1 -- 172.21.15.39:0/1502337272 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c809eb80 con 0x563e16e35480 2024-05-24T11:45:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.435+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1502337272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 msgr2=0x563e16e61bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.435+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1502337272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e61bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009990 tx=0x7fd5c80b9170 comp rx=0 tx=0).stop 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.436+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1502337272 shutdown_connections 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.436+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1502337272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e61bd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.436+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1502337272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3b270 0x563e16e2be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.436+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/1502337272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e2f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.436+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1502337272 >> 172.21.15.39:0/1502337272 conn(0x563e16d931b0 msgr2=0x563e16e33510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.436+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1502337272 shutdown_connections 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.436+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/1502337272 wait complete. 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.436+0000 7fd5e01a7b80 1 -- start start 2024-05-24T11:45:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.436+0000 7fd5e01a7b80 1 Processor -- start 2024-05-24T11:45:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e55dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e56310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5e01a7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3b270 0x563e16e56850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16eb4ce0 con 0x563e16e3b270 2024-05-24T11:45:38.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16eb4e20 con 0x563e16e3a950 2024-05-24T11:45:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5e01a7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e16eb4f60 con 0x563e16e35480 2024-05-24T11:45:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5e01a7b80 10 client.0 _ll_register_callbacks cb 0x563e16d2aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:45:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1d88700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e55dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1d88700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e55dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:48386/0 (socket says 172.21.15.39:48386) 2024-05-24T11:45:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1d88700 1 -- 172.21.15.39:0/2503815643 learned_addr learned my addr 172.21.15.39:0/2503815643 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1587700 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e56310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d2589700 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3b270 0x563e16e56850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1d88700 1 -- 172.21.15.39:0/2503815643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 msgr2=0x563e16e56310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e56310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1d88700 1 -- 172.21.15.39:0/2503815643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3b270 msgr2=0x563e16e56850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3b270 0x563e16e56850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1d88700 1 -- 172.21.15.39:0/2503815643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e16bb2b00 con 0x563e16e35480 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5d1d88700 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e55dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd5c809aa50 tx=0x7fd5c809b350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2bd70527b665e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.437+0000 7fd5b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5b57fa700 1 -- 172.21.15.39:0/2503815643 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80b2070 con 0x563e16e35480 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5b57fa700 1 -- 172.21.15.39:0/2503815643 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fd5c8097460 con 0x563e16e35480 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/2503815643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5c4004bb0 con 0x563e16e35480 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5b57fa700 1 -- 172.21.15.39:0/2503815643 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5c809a6b0 con 0x563e16e35480 2024-05-24T11:45:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5e01a7b80 10 client.25561 fetch_fsmap learned FSMap version 29 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/2503815643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563e16c0ac70 con 0x563e16e35480 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5b57fa700 1 -- 172.21.15.39:0/2503815643 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5c4004bb0 con 0x563e16e35480 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5b57fa700 1 -- 172.21.15.39:0/2503815643 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5c8080080 con 0x563e16e35480 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5e01a7b80 10 client.25561 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5e01a7b80 10 client.25561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5e01a7b80 20 client.25561 populate_metadata read hostname 'smithi039' 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563e16eb61e0 0x563e16eb85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5e01a7b80 4 client.25561 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.438+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/2503815643 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e16d7de60 con 0x563e16eb61e0 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.439+0000 7fd5d1587700 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563e16eb61e0 0x563e16eb85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.439+0000 7fd5d1587700 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563e16eb61e0 0x563e16eb85d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6bf63d7c36e33ad server_cookie=4ad76855dca97e3a in_seq=0 out_seq=0 2024-05-24T11:45:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:38.439+0000 7fd5b57fa700 10 client.25561 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:39.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 375.0 s, 1153.2 tps, lat 86.819 ms stddev 26.122 2024-05-24T11:45:39.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:39 smithi053 ceph-mon[47448]: pgmap v229: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2024-05-24T11:45:39.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:39 smithi137 ceph-mon[47907]: pgmap v229: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2024-05-24T11:45:39.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:39 smithi039 ceph-mon[42919]: pgmap v229: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 24 MiB/s wr, 1.80k op/s 2024-05-24T11:45:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5b57fa700 1 -- 172.21.15.39:0/2503815643 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e16d7de60 con 0x563e16eb61e0 2024-05-24T11:45:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5b57fa700 10 client.25561 handle_command_reply: tid=0 2024-05-24T11:45:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5e01a7b80 1 client.25561 shutdown 2024-05-24T11:45:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/2503815643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 msgr2=0x563e16e55dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e55dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd5c809aa50 tx=0x7fd5c809b350 comp rx=0 tx=0).stop 2024-05-24T11:45:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/2503815643 shutdown_connections 2024-05-24T11:45:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e16e35480 0x563e16e55dd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563e16eb61e0 0x563e16eb85d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563e16e3b270 0x563e16e56850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5e01a7b80 1 --2- 172.21.15.39:0/2503815643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563e16e3a950 0x563e16e56310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/2503815643 >> 172.21.15.39:0/2503815643 conn(0x563e16d931b0 msgr2=0x563e16e3cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.144+0000 7fd5b57fa700 0 client.25561 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.145+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/2503815643 shutdown_connections 2024-05-24T11:45:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.145+0000 7fd5e01a7b80 1 -- 172.21.15.39:0/2503815643 wait complete. 2024-05-24T11:45:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.145+0000 7fd5e01a7b80 20 client.25561 trim_cache size 0 max 16384 2024-05-24T11:45:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:45:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:45:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:45:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:45:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 -- 172.21.15.39:0/3356556798 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5b80a1350 msgr2=0x7fd5b80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5b80a1350 0x7fd5b80a3810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc111940 tx=0x7fd5bc004080 comp rx=0 tx=0).stop 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 -- 172.21.15.39:0/3356556798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5cc100080 msgr2=0x7fd5cc111e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5cc100080 0x7fd5cc111e80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800b580 tx=0x7fd5c8005460 comp rx=0 tx=0).stop 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 -- 172.21.15.39:0/3356556798 shutdown_connections 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5cc100080 0x7fd5cc111e80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd5b80a1350 0x7fd5b80a3810 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5cc104de0 0x7fd5cc110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 --2- 172.21.15.39:0/3356556798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc100a40 0x7fd5cc1105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 -- 172.21.15.39:0/3356556798 >> 172.21.15.39:0/3356556798 conn(0x7fd5cc076c10 msgr2=0x7fd5cc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 -- 172.21.15.39:0/3356556798 shutdown_connections 2024-05-24T11:45:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:41.146+0000 7fd5d3fec700 1 -- 172.21.15.39:0/3356556798 wait complete. 2024-05-24T11:45:41.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:41 smithi053 ceph-mon[47448]: pgmap v230: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-05-24T11:45:41.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:41 smithi137 ceph-mon[47907]: pgmap v230: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-05-24T11:45:41.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:41 smithi039 ceph-mon[42919]: pgmap v230: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-05-24T11:45:43.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:43 smithi053 ceph-mon[47448]: pgmap v231: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-05-24T11:45:43.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:43 smithi137 ceph-mon[47907]: pgmap v231: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-05-24T11:45:43.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:43 smithi039 ceph-mon[42919]: pgmap v231: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-05-24T11:45:44.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 380.0 s, 1178.4 tps, lat 84.711 ms stddev 27.189 2024-05-24T11:45:45.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:45 smithi053 ceph-mon[47448]: pgmap v232: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2024-05-24T11:45:45.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:45 smithi137 ceph-mon[47907]: pgmap v232: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2024-05-24T11:45:45.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:45 smithi039 ceph-mon[42919]: pgmap v232: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2024-05-24T11:45:47.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:47 smithi053 ceph-mon[47448]: pgmap v233: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-05-24T11:45:47.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:47 smithi137 ceph-mon[47907]: pgmap v233: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-05-24T11:45:47.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:47 smithi039 ceph-mon[42919]: pgmap v233: 97 pgs: 97 active+clean; 5.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-05-24T11:45:49.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 385.0 s, 1125.2 tps, lat 88.936 ms stddev 202.314 2024-05-24T11:45:49.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:49 smithi053 ceph-mon[47448]: pgmap v234: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 23 MiB/s wr, 1.83k op/s 2024-05-24T11:45:49.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:49 smithi137 ceph-mon[47907]: pgmap v234: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 23 MiB/s wr, 1.83k op/s 2024-05-24T11:45:49.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:49 smithi039 ceph-mon[42919]: pgmap v234: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 23 MiB/s wr, 1.83k op/s 2024-05-24T11:45:51.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:51 smithi053 ceph-mon[47448]: pgmap v235: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:45:51.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:51 smithi137 ceph-mon[47907]: pgmap v235: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:45:51.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:51 smithi039 ceph-mon[42919]: pgmap v235: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:45:53.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:53 smithi053 ceph-mon[47448]: pgmap v236: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:45:53.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:53 smithi137 ceph-mon[47907]: pgmap v236: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:45:53.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:53 smithi039 ceph-mon[42919]: pgmap v236: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:45:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 390.0 s, 1234.4 tps, lat 81.192 ms stddev 26.272 2024-05-24T11:45:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.322+0000 7fea6a8a2700 1 -- 172.21.15.39:0/1587648259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 msgr2=0x7fea6410c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.322+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/1587648259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 0x7fea6410c490 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00c180 tx=0x7fea5c00c490 comp rx=0 tx=0).stop 2024-05-24T11:45:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.322+0000 7fea6a8a2700 1 -- 172.21.15.39:0/1587648259 shutdown_connections 2024-05-24T11:45:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.322+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/1587648259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 0x7fea6410c490 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.322+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/1587648259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea640ffa50 0x7fea640ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.322+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/1587648259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea64100400 0x7fea641041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.322+0000 7fea6a8a2700 1 -- 172.21.15.39:0/1587648259 >> 172.21.15.39:0/1587648259 conn(0x7fea64075130 msgr2=0x7fea64075540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.322+0000 7fea6a8a2700 1 -- 172.21.15.39:0/1587648259 shutdown_connections 2024-05-24T11:45:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.322+0000 7fea6a8a2700 1 -- 172.21.15.39:0/1587648259 wait complete. 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.323+0000 7fea6a8a2700 1 Processor -- start 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.323+0000 7fea6a8a2700 1 -- start start 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea6a8a2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea640ffa50 0x7fea641995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea6a8a2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 0x7fea64199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea6a8a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6419de60 0x7fea6419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea6a8a2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6419e8e0 con 0x7fea6419de60 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea6a8a2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6419ea50 con 0x7fea640ffa50 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea6a8a2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6419ec10 con 0x7fea64104910 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea63fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea640ffa50 0x7fea641995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea637fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 0x7fea64199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea637fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 0x7fea64199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38792/0 (socket says 172.21.15.39:38792) 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea637fe700 1 -- 172.21.15.39:0/2066781263 learned_addr learned my addr 172.21.15.39:0/2066781263 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea68e3f700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6419de60 0x7fea6419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea637fe700 1 -- 172.21.15.39:0/2066781263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea640ffa50 msgr2=0x7fea641995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea637fe700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea640ffa50 0x7fea641995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea637fe700 1 -- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6419de60 msgr2=0x7fea6419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea637fe700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6419de60 0x7fea6419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea637fe700 1 -- 172.21.15.39:0/2066781263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea540095f0 con 0x7fea64104910 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.324+0000 7fea63fff700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea640ffa50 0x7fea641995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.325+0000 7fea637fe700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 0x7fea64199b10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e970 tx=0x7fea5800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.325+0000 7fea617fa700 1 -- 172.21.15.39:0/2066781263 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58009790 con 0x7fea64104910 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.325+0000 7fea617fa700 1 -- 172.21.15.39:0/2066781263 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea5800c7e0 con 0x7fea64104910 2024-05-24T11:45:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.325+0000 7fea617fa700 1 -- 172.21.15.39:0/2066781263 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58005440 con 0x7fea64104910 2024-05-24T11:45:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.325+0000 7fea6a8a2700 1 -- 172.21.15.39:0/2066781263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c00ad50 con 0x7fea64104910 2024-05-24T11:45:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.325+0000 7fea6a8a2700 1 -- 172.21.15.39:0/2066781263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea641a6470 con 0x7fea64104910 2024-05-24T11:45:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.328+0000 7fea617fa700 1 -- 172.21.15.39:0/2066781263 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fea5800c950 con 0x7fea64104910 2024-05-24T11:45:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.329+0000 7fea617fa700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea4c0a1310 0x7fea4c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.329+0000 7fea617fa700 1 -- 172.21.15.39:0/2066781263 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fea58014070 con 0x7fea64104910 2024-05-24T11:45:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.329+0000 7fea63fff700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea4c0a1310 0x7fea4c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.330+0000 7fea76a5db80 1 Processor -- start 2024-05-24T11:45:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.330+0000 7fea76a5db80 1 -- start start 2024-05-24T11:45:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.330+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400b3100 0x55e7400b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.330+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400a5f40 0x55e7400a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.330+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a6900 0x55e7400bae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.330+0000 7fea76a5db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73fffd0f0 con 0x55e7400b3100 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.330+0000 7fea76a5db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73fffaab0 con 0x55e7400a5f40 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.330+0000 7fea76a5db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73fff63a0 con 0x55e7400a6900 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea63fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400b3100 0x55e7400b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea68e3f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a6900 0x55e7400bae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea68e3f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a6900 0x55e7400bae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38796/0 (socket says 172.21.15.39:38796) 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea68e3f700 1 -- 172.21.15.39:0/3674147689 learned_addr learned my addr 172.21.15.39:0/3674147689 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea63fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400b3100 0x55e7400b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48182/0 (socket says 172.21.15.39:48182) 2024-05-24T11:45:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea68e3f700 1 -- 172.21.15.39:0/3674147689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400a5f40 msgr2=0x55e7400a63c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:45:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea68e3f700 1 --2- 172.21.15.39:0/3674147689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400a5f40 0x55e7400a63c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea68e3f700 1 -- 172.21.15.39:0/3674147689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400b3100 msgr2=0x55e7400b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea68e3f700 1 --2- 172.21.15.39:0/3674147689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400b3100 0x55e7400b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea68e3f700 1 -- 172.21.15.39:0/3674147689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e73fe77350 con 0x55e7400a6900 2024-05-24T11:45:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.331+0000 7fea68e3f700 1 --2- 172.21.15.39:0/3674147689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a6900 0x55e7400bae40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00ac60 tx=0x7fea5c00ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=443fa4034b543e78 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.332+0000 7fea4a7fc700 1 -- 172.21.15.39:0/3674147689 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00f050 con 0x55e7400a6900 2024-05-24T11:45:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.332+0000 7fea4a7fc700 1 -- 172.21.15.39:0/3674147689 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea5c008730 con 0x55e7400a6900 2024-05-24T11:45:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.332+0000 7fea4a7fc700 1 -- 172.21.15.39:0/3674147689 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c003c60 con 0x55e7400a6900 2024-05-24T11:45:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 -- 172.21.15.39:0/3674147689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a6900 msgr2=0x55e7400bae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3674147689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a6900 0x55e7400bae40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00ac60 tx=0x7fea5c00ac90 comp rx=0 tx=0).stop 2024-05-24T11:45:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea63fff700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea4c0a1310 0x7fea4c0a37d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea54005fd0 tx=0x7fea5400f5f0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 -- 172.21.15.39:0/3674147689 shutdown_connections 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3674147689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a6900 0x55e7400bae40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3674147689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400b3100 0x55e7400b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3674147689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400a5f40 0x55e7400a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 -- 172.21.15.39:0/3674147689 >> 172.21.15.39:0/3674147689 conn(0x55e7400041b0 msgr2=0x55e7400a08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 -- 172.21.15.39:0/3674147689 shutdown_connections 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 -- 172.21.15.39:0/3674147689 wait complete. 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.333+0000 7fea76a5db80 1 -- start start 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea76a5db80 1 Processor -- start 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400a5f40 0x55e7400badf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400b3100 0x55e7400b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea76a5db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73ff92500 con 0x55e7400a6900 2024-05-24T11:45:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea76a5db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73ff921a0 con 0x55e7400a5f40 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea76a5db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73fe79640 con 0x55e7400b3100 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea76a5db80 10 client.0 _ll_register_callbacks cb 0x55e73ff9bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea68e3f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400b3100 0x55e7400b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea68e3f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400b3100 0x55e7400b8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38804/0 (socket says 172.21.15.39:38804) 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea68e3f700 1 -- 172.21.15.39:0/2682843987 learned_addr learned my addr 172.21.15.39:0/2682843987 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.334+0000 7fea637fe700 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea68e3f700 1 -- 172.21.15.39:0/2682843987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400a5f40 msgr2=0x55e7400badf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea68e3f700 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400a5f40 0x55e7400badf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea68e3f700 1 -- 172.21.15.39:0/2682843987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 msgr2=0x55e7400b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea68e3f700 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea68e3f700 1 -- 172.21.15.39:0/2682843987 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6410cca0 con 0x55e7400b3100 2024-05-24T11:45:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea68e3f700 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400b3100 0x55e7400b8760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea5c008d30 tx=0x7fea5c00df50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3673f96a96285370 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea437fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:45:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea437fe700 1 -- 172.21.15.39:0/2682843987 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c003e10 con 0x55e7400b3100 2024-05-24T11:45:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea437fe700 1 -- 172.21.15.39:0/2682843987 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fea5c0050b0 con 0x55e7400b3100 2024-05-24T11:45:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea76a5db80 1 -- 172.21.15.39:0/2682843987 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e73ff4f210 con 0x55e7400b3100 2024-05-24T11:45:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.335+0000 7fea437fe700 1 -- 172.21.15.39:0/2682843987 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea5c00f020 con 0x55e7400b3100 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea76a5db80 10 client.25573 fetch_fsmap learned FSMap version 29 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea76a5db80 1 -- 172.21.15.39:0/2682843987 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e73fe23b00 con 0x55e7400b3100 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea437fe700 1 -- 172.21.15.39:0/2682843987 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e73ff4f210 con 0x55e7400b3100 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea437fe700 1 -- 172.21.15.39:0/2682843987 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fea5c00c1b0 con 0x55e7400b3100 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea76a5db80 10 client.25573 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea76a5db80 10 client.25573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea76a5db80 20 client.25573 populate_metadata read hostname 'smithi039' 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea76a5db80 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e7400d8f70 0x55e7400db360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea76a5db80 4 client.25573 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea76a5db80 1 -- 172.21.15.39:0/2682843987 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7400757b0 con 0x55e7400d8f70 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.336+0000 7fea63fff700 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e7400d8f70 0x55e7400db360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.337+0000 7fea437fe700 10 client.25573 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.337+0000 7fea63fff700 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e7400d8f70 0x55e7400db360 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4945b35fd8f109dc server_cookie=3a2785e088ef5358 in_seq=0 out_seq=0 2024-05-24T11:45:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea437fe700 1 -- 172.21.15.39:0/2682843987 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e7400757b0 con 0x55e7400d8f70 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea437fe700 10 client.25573 handle_command_reply: tid=0 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 client.25573 shutdown 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 -- 172.21.15.39:0/2682843987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400b3100 msgr2=0x55e7400b8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400b3100 0x55e7400b8760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea5c008d30 tx=0x7fea5c00df50 comp rx=0 tx=0).stop 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 -- 172.21.15.39:0/2682843987 shutdown_connections 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400b3100 0x55e7400b8760 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e7400d8f70 0x55e7400db360 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400a5f40 0x55e7400badf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 --2- 172.21.15.39:0/2682843987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea437fe700 0 client.25573 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 -- 172.21.15.39:0/2682843987 >> 172.21.15.39:0/2682843987 conn(0x55e7400041b0 msgr2=0x55e7400a4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 -- 172.21.15.39:0/2682843987 shutdown_connections 2024-05-24T11:45:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.338+0000 7fea76a5db80 1 -- 172.21.15.39:0/2682843987 wait complete. 2024-05-24T11:45:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.339+0000 7fea76a5db80 20 client.25573 trim_cache size 0 max 16384 2024-05-24T11:45:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.366+0000 7fea76a5db80 1 Processor -- start 2024-05-24T11:45:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.367+0000 7fea76a5db80 1 -- start start 2024-05-24T11:45:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.367+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a5f40 0x55e7400b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.367+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400ba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.367+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.367+0000 7fea76a5db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7400b9040 con 0x55e7400a6900 2024-05-24T11:45:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.367+0000 7fea76a5db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7400b9500 con 0x55e7400b3100 2024-05-24T11:45:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.367+0000 7fea76a5db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7400a20f0 con 0x55e7400a5f40 2024-05-24T11:45:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.368+0000 7fea68e3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.368+0000 7fea68e3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400d04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:60682/0 (socket says 172.21.15.39:60682) 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.368+0000 7fea68e3f700 1 -- 172.21.15.39:0/4143197261 learned_addr learned my addr 172.21.15.39:0/4143197261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.368+0000 7fea68e3f700 1 -- 172.21.15.39:0/4143197261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a5f40 msgr2=0x55e7400b34e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.368+0000 7fea68e3f700 1 --2- 172.21.15.39:0/4143197261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a5f40 0x55e7400b34e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.368+0000 7fea68e3f700 1 -- 172.21.15.39:0/4143197261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 msgr2=0x55e7400ba920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.368+0000 7fea68e3f700 1 --2- 172.21.15.39:0/4143197261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400ba920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.368+0000 7fea68e3f700 1 -- 172.21.15.39:0/4143197261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e73ffbc500 con 0x55e7400b3100 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.369+0000 7fea68e3f700 1 --2- 172.21.15.39:0/4143197261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400d04c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea5c002230 tx=0x7fea5c006160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f71b72347d5923 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.369+0000 7fea4affd700 1 -- 172.21.15.39:0/4143197261 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00f070 con 0x55e7400b3100 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.369+0000 7fea4affd700 1 -- 172.21.15.39:0/4143197261 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea5c031770 con 0x55e7400b3100 2024-05-24T11:45:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.369+0000 7fea4affd700 1 -- 172.21.15.39:0/4143197261 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0127a0 con 0x55e7400b3100 2024-05-24T11:45:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.370+0000 7fea76a5db80 1 -- 172.21.15.39:0/4143197261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 msgr2=0x55e7400d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.370+0000 7fea76a5db80 1 --2- 172.21.15.39:0/4143197261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400d04c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea5c002230 tx=0x7fea5c006160 comp rx=0 tx=0).stop 2024-05-24T11:45:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.370+0000 7fea76a5db80 1 -- 172.21.15.39:0/4143197261 shutdown_connections 2024-05-24T11:45:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.370+0000 7fea76a5db80 1 --2- 172.21.15.39:0/4143197261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a5f40 0x55e7400b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.370+0000 7fea76a5db80 1 --2- 172.21.15.39:0/4143197261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400d04c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.370+0000 7fea76a5db80 1 --2- 172.21.15.39:0/4143197261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400ba920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.370+0000 7fea76a5db80 1 -- 172.21.15.39:0/4143197261 >> 172.21.15.39:0/4143197261 conn(0x55e7400041b0 msgr2=0x55e7400d4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.370+0000 7fea76a5db80 1 -- 172.21.15.39:0/4143197261 shutdown_connections 2024-05-24T11:45:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.370+0000 7fea76a5db80 1 -- 172.21.15.39:0/4143197261 wait complete. 2024-05-24T11:45:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.371+0000 7fea76a5db80 1 -- start start 2024-05-24T11:45:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.371+0000 7fea76a5db80 1 Processor -- start 2024-05-24T11:45:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a5f40 0x55e7400d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea76a5db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea76a5db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7400cf160 con 0x55e7400a6900 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea76a5db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7400cf2d0 con 0x55e7400b3100 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea76a5db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74012e350 con 0x55e7400a5f40 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea76a5db80 10 client.0 _ll_register_callbacks cb 0x55e73ff9bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48188/0 (socket says 172.21.15.39:48188) 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea637fe700 1 -- 172.21.15.39:0/3826498552 learned_addr learned my addr 172.21.15.39:0/3826498552 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea68e3f700 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea68e3f700 1 -- 172.21.15.39:0/3826498552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a5f40 msgr2=0x55e7400d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea68e3f700 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a5f40 0x55e7400d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea68e3f700 1 -- 172.21.15.39:0/3826498552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 msgr2=0x55e7400ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea68e3f700 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.372+0000 7fea68e3f700 1 -- 172.21.15.39:0/3826498552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e73fe23b00 con 0x55e7400b3100 2024-05-24T11:45:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.373+0000 7fea68e3f700 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400cea00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00aa70 tx=0x7fea5c031670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf2dc71ab0213019 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:45:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.373+0000 7fea437fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:45:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.373+0000 7fea437fe700 1 -- 172.21.15.39:0/3826498552 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00f040 con 0x55e7400b3100 2024-05-24T11:45:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.373+0000 7fea437fe700 1 -- 172.21.15.39:0/3826498552 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fea5c031b60 con 0x55e7400b3100 2024-05-24T11:45:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.373+0000 7fea76a5db80 1 -- 172.21.15.39:0/3826498552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea5c008730 con 0x55e7400b3100 2024-05-24T11:45:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.373+0000 7fea437fe700 1 -- 172.21.15.39:0/3826498552 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea5c031d20 con 0x55e7400b3100 2024-05-24T11:45:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.373+0000 7fea76a5db80 10 client.25670 fetch_fsmap learned FSMap version 29 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.373+0000 7fea76a5db80 1 -- 172.21.15.39:0/3826498552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e73fe7bc70 con 0x55e7400b3100 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.373+0000 7fea437fe700 1 -- 172.21.15.39:0/3826498552 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea5c008730 con 0x55e7400b3100 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.374+0000 7fea437fe700 1 -- 172.21.15.39:0/3826498552 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fea5c038030 con 0x55e7400b3100 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.374+0000 7fea76a5db80 10 client.25670 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.374+0000 7fea76a5db80 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.374+0000 7fea76a5db80 20 client.25670 populate_metadata read hostname 'smithi039' 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.374+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e74012f5d0 0x55e7401319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.374+0000 7fea76a5db80 4 client.25670 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.374+0000 7fea76a5db80 1 -- 172.21.15.39:0/3826498552 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e73ffeee60 con 0x55e74012f5d0 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.374+0000 7fea63fff700 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e74012f5d0 0x55e7401319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.375+0000 7fea63fff700 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e74012f5d0 0x55e7401319c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2aa7ac1e21b8689 server_cookie=e6bfebf4693c1101 in_seq=0 out_seq=0 2024-05-24T11:45:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.375+0000 7fea437fe700 10 client.25670 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.406+0000 7fea437fe700 1 -- 172.21.15.39:0/3826498552 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e73ffeee60 con 0x55e74012f5d0 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.406+0000 7fea437fe700 10 client.25670 handle_command_reply: tid=0 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.406+0000 7fea76a5db80 1 client.25670 shutdown 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.406+0000 7fea76a5db80 1 -- 172.21.15.39:0/3826498552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 msgr2=0x55e7400cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.406+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400cea00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00aa70 tx=0x7fea5c031670 comp rx=0 tx=0).stop 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea76a5db80 1 -- 172.21.15.39:0/3826498552 shutdown_connections 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7400a5f40 0x55e7400d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e74012f5d0 0x55e7401319c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e7400b3100 0x55e7400cea00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea76a5db80 1 --2- 172.21.15.39:0/3826498552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e7400a6900 0x55e7400ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea437fe700 0 client.25670 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea76a5db80 1 -- 172.21.15.39:0/3826498552 >> 172.21.15.39:0/3826498552 conn(0x55e7400041b0 msgr2=0x55e7400d4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea76a5db80 1 -- 172.21.15.39:0/3826498552 shutdown_connections 2024-05-24T11:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea76a5db80 1 -- 172.21.15.39:0/3826498552 wait complete. 2024-05-24T11:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.407+0000 7fea76a5db80 20 client.25670 trim_cache size 0 max 16384 2024-05-24T11:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.410+0000 7fea6a8a2700 1 -- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea4c0a1310 msgr2=0x7fea4c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.410+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea4c0a1310 0x7fea4c0a37d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea54005fd0 tx=0x7fea5400f5f0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.410+0000 7fea6a8a2700 1 -- 172.21.15.39:0/2066781263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 msgr2=0x7fea64199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.410+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 0x7fea64199b10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e970 tx=0x7fea5800ec80 comp rx=0 tx=0).stop 2024-05-24T11:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.411+0000 7fea6a8a2700 1 -- 172.21.15.39:0/2066781263 shutdown_connections 2024-05-24T11:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.411+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea4c0a1310 0x7fea4c0a37d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.411+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6419de60 0x7fea6419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.411+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea640ffa50 0x7fea641995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.411+0000 7fea6a8a2700 1 --2- 172.21.15.39:0/2066781263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104910 0x7fea64199b10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.411+0000 7fea6a8a2700 1 -- 172.21.15.39:0/2066781263 >> 172.21.15.39:0/2066781263 conn(0x7fea64075130 msgr2=0x7fea640fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.411+0000 7fea6a8a2700 1 -- 172.21.15.39:0/2066781263 shutdown_connections 2024-05-24T11:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:45:54.411+0000 7fea6a8a2700 1 -- 172.21.15.39:0/2066781263 wait complete. 2024-05-24T11:45:55.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:55 smithi053 ceph-mon[47448]: pgmap v237: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2024-05-24T11:45:55.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:55 smithi137 ceph-mon[47907]: pgmap v237: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2024-05-24T11:45:55.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:55 smithi039 ceph-mon[42919]: pgmap v237: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 22 MiB/s wr, 1.82k op/s 2024-05-24T11:45:57.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:57 smithi053 ceph-mon[47448]: pgmap v238: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2024-05-24T11:45:57.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:57 smithi137 ceph-mon[47907]: pgmap v238: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2024-05-24T11:45:57.831 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:57 smithi039 ceph-mon[42919]: pgmap v238: 97 pgs: 97 active+clean; 5.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2024-05-24T11:45:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 395.0 s, 1022.7 tps, lat 97.438 ms stddev 95.230 2024-05-24T11:45:59.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:45:59 smithi053 ceph-mon[47448]: pgmap v239: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 23 MiB/s wr, 1.91k op/s 2024-05-24T11:45:59.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:45:59 smithi137 ceph-mon[47907]: pgmap v239: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 23 MiB/s wr, 1.91k op/s 2024-05-24T11:45:59.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:45:59 smithi039 ceph-mon[42919]: pgmap v239: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 23 MiB/s wr, 1.91k op/s 2024-05-24T11:46:01.742 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:01 smithi053 ceph-mon[47448]: pgmap v240: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:46:01.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:01 smithi137 ceph-mon[47907]: pgmap v240: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:46:01.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:01 smithi039 ceph-mon[42919]: pgmap v240: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:46:03.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:03 smithi053 ceph-mon[47448]: pgmap v241: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:46:03.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:03 smithi137 ceph-mon[47907]: pgmap v241: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:46:03.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:03 smithi039 ceph-mon[42919]: pgmap v241: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2024-05-24T11:46:04.118 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 400.0 s, 1116.4 tps, lat 89.853 ms stddev 34.865 2024-05-24T11:46:05.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:05 smithi053 ceph-mon[47448]: pgmap v242: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 21 MiB/s wr, 1.80k op/s 2024-05-24T11:46:05.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:05 smithi137 ceph-mon[47907]: pgmap v242: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 21 MiB/s wr, 1.80k op/s 2024-05-24T11:46:05.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:05 smithi039 ceph-mon[42919]: pgmap v242: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 21 MiB/s wr, 1.80k op/s 2024-05-24T11:46:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.579+0000 7f426b05f700 1 -- 172.21.15.39:0/3667425069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4264101110 msgr2=0x7f42641014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.579+0000 7f426b05f700 1 --2- 172.21.15.39:0/3667425069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4264101110 0x7f42641014f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4258009990 tx=0x7f4258009ca0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.580+0000 7f426b05f700 1 -- 172.21.15.39:0/3667425069 shutdown_connections 2024-05-24T11:46:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.580+0000 7f426b05f700 1 --2- 172.21.15.39:0/3667425069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4264106230 0x7f426410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.580+0000 7f426b05f700 1 --2- 172.21.15.39:0/3667425069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4264101110 0x7f42641014f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.580+0000 7f426b05f700 1 --2- 172.21.15.39:0/3667425069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4264101ac0 0x7f4264105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.580+0000 7f426b05f700 1 -- 172.21.15.39:0/3667425069 >> 172.21.15.39:0/3667425069 conn(0x7f42640fc9b0 msgr2=0x7f42640fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.580+0000 7f426b05f700 1 -- 172.21.15.39:0/3667425069 shutdown_connections 2024-05-24T11:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.580+0000 7f426b05f700 1 -- 172.21.15.39:0/3667425069 wait complete. 2024-05-24T11:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f426b05f700 1 Processor -- start 2024-05-24T11:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f426b05f700 1 -- start start 2024-05-24T11:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f426b05f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4264101110 0x7f4264072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f426b05f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4264106230 0x7f4264072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f426b05f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426406b7b0 0x7f426406bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f426b05f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426406c3d0 con 0x7f4264106230 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f426b05f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426406c540 con 0x7f4264101110 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f426b05f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4264070180 con 0x7f426406b7b0 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f42695fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426406b7b0 0x7f426406bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f42695fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426406b7b0 0x7f426406bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47618/0 (socket says 172.21.15.39:47618) 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f42695fc700 1 -- 172.21.15.39:0/3336775418 learned_addr learned my addr 172.21.15.39:0/3336775418 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f4268dfb700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4264101110 0x7f4264072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.581+0000 7f4263fff700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4264106230 0x7f4264072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4268dfb700 1 -- 172.21.15.39:0/3336775418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426406b7b0 msgr2=0x7f426406bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4268dfb700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426406b7b0 0x7f426406bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4268dfb700 1 -- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4264106230 msgr2=0x7f4264072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4268dfb700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4264106230 0x7f4264072ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4268dfb700 1 -- 172.21.15.39:0/3336775418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42580095f0 con 0x7f4264101110 2024-05-24T11:46:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4263fff700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4264106230 0x7f4264072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:46:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4268dfb700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4264101110 0x7f4264072660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4258009990 tx=0x7f425800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4261ffb700 1 -- 172.21.15.39:0/3336775418 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258021070 con 0x7f4264101110 2024-05-24T11:46:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f426b05f700 1 -- 172.21.15.39:0/3336775418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4264070400 con 0x7f4264101110 2024-05-24T11:46:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f426b05f700 1 -- 172.21.15.39:0/3336775418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4264070960 con 0x7f4264101110 2024-05-24T11:46:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4261ffb700 1 -- 172.21.15.39:0/3336775418 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f425801ce00 con 0x7f4264101110 2024-05-24T11:46:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.582+0000 7f4261ffb700 1 -- 172.21.15.39:0/3336775418 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258027420 con 0x7f4264101110 2024-05-24T11:46:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.588+0000 7f427721ab80 1 Processor -- start 2024-05-24T11:46:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.588+0000 7f4261ffb700 1 -- 172.21.15.39:0/3336775418 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4258025070 con 0x7f4264101110 2024-05-24T11:46:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.588+0000 7f427721ab80 1 -- start start 2024-05-24T11:46:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.588+0000 7f427721ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024160 0x558f55024540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.588+0000 7f427721ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024a80 0x558f5501e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.588+0000 7f427721ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f5501ec90 0x558f5501f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.588+0000 7f427721ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f54f6d0f0 con 0x558f55024160 2024-05-24T11:46:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.588+0000 7f427721ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f54f6aab0 con 0x558f5501ec90 2024-05-24T11:46:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.588+0000 7f427721ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f54f663a0 con 0x558f55024a80 2024-05-24T11:46:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4268dfb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024a80 0x558f5501e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4268dfb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024a80 0x558f5501e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47624/0 (socket says 172.21.15.39:47624) 2024-05-24T11:46:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4268dfb700 1 -- 172.21.15.39:0/931924105 learned_addr learned my addr 172.21.15.39:0/931924105 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4263fff700 1 --2- 172.21.15.39:0/931924105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f5501ec90 0x558f5501f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4268dfb700 1 -- 172.21.15.39:0/931924105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f5501ec90 msgr2=0x558f5501f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4268dfb700 1 --2- 172.21.15.39:0/931924105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f5501ec90 0x558f5501f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4268dfb700 1 -- 172.21.15.39:0/931924105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024160 msgr2=0x558f55024540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4268dfb700 1 --2- 172.21.15.39:0/931924105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024160 0x558f55024540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4268dfb700 1 -- 172.21.15.39:0/931924105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f54de7350 con 0x558f55024a80 2024-05-24T11:46:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f4261ffb700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f42540a1310 0x7f42540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.589+0000 7f42695fc700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f42540a1310 0x7f42540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.590+0000 7f4268dfb700 1 --2- 172.21.15.39:0/931924105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024a80 0x558f5501e750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f42580cf640 tx=0x7f425801ac30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d0ba160d8f2af14 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.590+0000 7f4261ffb700 1 -- 172.21.15.39:0/3336775418 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f42580d1f10 con 0x7f4264101110 2024-05-24T11:46:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.590+0000 7f42695fc700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f42540a1310 0x7f42540a37d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f425c007800 tx=0x7f425c007750 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.591+0000 7f424effd700 1 -- 172.21.15.39:0/931924105 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425801ae70 con 0x558f55024a80 2024-05-24T11:46:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.591+0000 7f424effd700 1 -- 172.21.15.39:0/931924105 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f425809b430 con 0x558f55024a80 2024-05-24T11:46:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.591+0000 7f424effd700 1 -- 172.21.15.39:0/931924105 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425809ea50 con 0x558f55024a80 2024-05-24T11:46:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.591+0000 7f427721ab80 1 -- 172.21.15.39:0/931924105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024a80 msgr2=0x558f5501e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.591+0000 7f427721ab80 1 --2- 172.21.15.39:0/931924105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024a80 0x558f5501e750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f42580cf640 tx=0x7f425801ac30 comp rx=0 tx=0).stop 2024-05-24T11:46:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.594+0000 7f427721ab80 1 -- 172.21.15.39:0/931924105 shutdown_connections 2024-05-24T11:46:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.594+0000 7f427721ab80 1 --2- 172.21.15.39:0/931924105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f5501ec90 0x558f5501f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.594+0000 7f427721ab80 1 --2- 172.21.15.39:0/931924105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024160 0x558f55024540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.594+0000 7f427721ab80 1 --2- 172.21.15.39:0/931924105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024a80 0x558f5501e750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.594+0000 7f427721ab80 1 -- 172.21.15.39:0/931924105 >> 172.21.15.39:0/931924105 conn(0x558f54f741b0 msgr2=0x558f55010840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.596+0000 7f427721ab80 1 -- 172.21.15.39:0/931924105 shutdown_connections 2024-05-24T11:46:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.596+0000 7f427721ab80 1 -- 172.21.15.39:0/931924105 wait complete. 2024-05-24T11:46:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.597+0000 7f427721ab80 1 -- start start 2024-05-24T11:46:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.597+0000 7f427721ab80 1 Processor -- start 2024-05-24T11:46:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.597+0000 7f427721ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f5501ec90 0x558f55017f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.597+0000 7f427721ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f55018460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.597+0000 7f427721ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024a80 0x558f5501dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.597+0000 7f427721ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f54f02500 con 0x558f55024a80 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.597+0000 7f427721ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f54f021a0 con 0x558f5501ec90 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.597+0000 7f427721ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f54de9640 con 0x558f55024160 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f427721ab80 10 client.0 _ll_register_callbacks cb 0x558f54f0bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4263fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f55018460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4263fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f55018460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47634/0 (socket says 172.21.15.39:47634) 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4263fff700 1 -- 172.21.15.39:0/1711488181 learned_addr learned my addr 172.21.15.39:0/1711488181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4263fff700 1 -- 172.21.15.39:0/1711488181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f5501ec90 msgr2=0x558f55017f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4263fff700 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f5501ec90 0x558f55017f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4263fff700 1 -- 172.21.15.39:0/1711488181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024a80 msgr2=0x558f5501dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4263fff700 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024a80 0x558f5501dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4263fff700 1 -- 172.21.15.39:0/1711488181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4264070960 con 0x558f55024160 2024-05-24T11:46:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4263fff700 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f55018460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4250012550 tx=0x7f4250012910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e9c9a761c50c1d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.598+0000 7f4243fff700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:46:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.599+0000 7f4243fff700 1 -- 172.21.15.39:0/1711488181 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500108a0 con 0x558f55024160 2024-05-24T11:46:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.599+0000 7f4243fff700 1 -- 172.21.15.39:0/1711488181 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f4250010a00 con 0x558f55024160 2024-05-24T11:46:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.599+0000 7f427721ab80 1 -- 172.21.15.39:0/1711488181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f54ebf210 con 0x558f55024160 2024-05-24T11:46:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.599+0000 7f4243fff700 1 -- 172.21.15.39:0/1711488181 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4250010bc0 con 0x558f55024160 2024-05-24T11:46:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.599+0000 7f427721ab80 10 client.25585 fetch_fsmap learned FSMap version 29 2024-05-24T11:46:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.599+0000 7f427721ab80 1 -- 172.21.15.39:0/1711488181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558f54d93b00 con 0x558f55024160 2024-05-24T11:46:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.600+0000 7f4243fff700 1 -- 172.21.15.39:0/1711488181 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558f54ebf210 con 0x558f55024160 2024-05-24T11:46:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.600+0000 7f4243fff700 1 -- 172.21.15.39:0/1711488181 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4250022070 con 0x558f55024160 2024-05-24T11:46:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.600+0000 7f427721ab80 10 client.25585 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:46:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.600+0000 7f427721ab80 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:46:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.600+0000 7f427721ab80 20 client.25585 populate_metadata read hostname 'smithi039' 2024-05-24T11:46:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.600+0000 7f427721ab80 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558f550408f0 0x558f55042d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.601+0000 7f427721ab80 4 client.25585 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:46:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.601+0000 7f427721ab80 1 -- 172.21.15.39:0/1711488181 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f54fe57b0 con 0x558f550408f0 2024-05-24T11:46:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.601+0000 7f4268dfb700 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558f550408f0 0x558f55042d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.601+0000 7f4268dfb700 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558f550408f0 0x558f55042d10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35d8e3638d3c46e server_cookie=2cef20995aceaa43 in_seq=0 out_seq=0 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.601+0000 7f4243fff700 10 client.25585 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f4243fff700 1 -- 172.21.15.39:0/1711488181 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f54fe57b0 con 0x558f550408f0 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f4243fff700 10 client.25585 handle_command_reply: tid=0 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f427721ab80 1 client.25585 shutdown 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f427721ab80 1 -- 172.21.15.39:0/1711488181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 msgr2=0x558f55018460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f427721ab80 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f55018460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4250012550 tx=0x7f4250012910 comp rx=0 tx=0).stop 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f427721ab80 1 -- 172.21.15.39:0/1711488181 shutdown_connections 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f427721ab80 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558f550408f0 0x558f55042d10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f427721ab80 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024a80 0x558f5501dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f427721ab80 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f5501ec90 0x558f55017f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f427721ab80 1 --2- 172.21.15.39:0/1711488181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f55018460 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f427721ab80 1 -- 172.21.15.39:0/1711488181 >> 172.21.15.39:0/1711488181 conn(0x558f54f741b0 msgr2=0x558f55021c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.602+0000 7f4243fff700 0 client.25585 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.603+0000 7f427721ab80 1 -- 172.21.15.39:0/1711488181 shutdown_connections 2024-05-24T11:46:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.603+0000 7f427721ab80 1 -- 172.21.15.39:0/1711488181 wait complete. 2024-05-24T11:46:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.603+0000 7f427721ab80 20 client.25585 trim_cache size 0 max 16384 2024-05-24T11:46:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.633+0000 7f427721ab80 1 Processor -- start 2024-05-24T11:46:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.633+0000 7f427721ab80 1 -- start start 2024-05-24T11:46:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f427721ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f5501ec90 0x558f55010910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f427721ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024160 0x558f550264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f427721ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f55026a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f427721ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5500d9b0 con 0x558f55024160 2024-05-24T11:46:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f427721ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5500b330 con 0x558f55024a80 2024-05-24T11:46:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f427721ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f55013800 con 0x558f5501ec90 2024-05-24T11:46:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4268dfb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f5501ec90 0x558f55010910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f42695fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f55026a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4268dfb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f5501ec90 0x558f55010910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47644/0 (socket says 172.21.15.39:47644) 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4268dfb700 1 -- 172.21.15.39:0/4088517702 learned_addr learned my addr 172.21.15.39:0/4088517702 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4263fff700 1 --2- 172.21.15.39:0/4088517702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024160 0x558f550264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4268dfb700 1 -- 172.21.15.39:0/4088517702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 msgr2=0x558f55026a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4268dfb700 1 --2- 172.21.15.39:0/4088517702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f55026a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4268dfb700 1 -- 172.21.15.39:0/4088517702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024160 msgr2=0x558f550264d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4268dfb700 1 --2- 172.21.15.39:0/4088517702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024160 0x558f550264d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4268dfb700 1 -- 172.21.15.39:0/4088517702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f54f2c500 con 0x558f5501ec90 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.634+0000 7f4268dfb700 1 --2- 172.21.15.39:0/4088517702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f5501ec90 0x558f55010910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f42580cfca0 tx=0x7f4258009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eda00c4196a3e9b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.635+0000 7f424f7fe700 1 -- 172.21.15.39:0/4088517702 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425809ea50 con 0x558f5501ec90 2024-05-24T11:46:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.635+0000 7f424f7fe700 1 -- 172.21.15.39:0/4088517702 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f425809eec0 con 0x558f5501ec90 2024-05-24T11:46:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.635+0000 7f424f7fe700 1 -- 172.21.15.39:0/4088517702 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42580a2f20 con 0x558f5501ec90 2024-05-24T11:46:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.635+0000 7f427721ab80 1 -- 172.21.15.39:0/4088517702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f5501ec90 msgr2=0x558f55010910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.635+0000 7f427721ab80 1 --2- 172.21.15.39:0/4088517702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f5501ec90 0x558f55010910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f42580cfca0 tx=0x7f4258009310 comp rx=0 tx=0).stop 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.636+0000 7f427721ab80 1 -- 172.21.15.39:0/4088517702 shutdown_connections 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.636+0000 7f427721ab80 1 --2- 172.21.15.39:0/4088517702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f5501ec90 0x558f55010910 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.636+0000 7f427721ab80 1 --2- 172.21.15.39:0/4088517702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f55026a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.636+0000 7f427721ab80 1 --2- 172.21.15.39:0/4088517702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f55024160 0x558f550264d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.636+0000 7f427721ab80 1 -- 172.21.15.39:0/4088517702 >> 172.21.15.39:0/4088517702 conn(0x558f54f741b0 msgr2=0x558f55047670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.636+0000 7f427721ab80 1 -- 172.21.15.39:0/4088517702 shutdown_connections 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.636+0000 7f427721ab80 1 -- 172.21.15.39:0/4088517702 wait complete. 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.636+0000 7f427721ab80 1 -- start start 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.636+0000 7f427721ab80 1 Processor -- start 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f427721ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f5501ec90 0x558f5501dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f427721ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f5501b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f427721ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f5501b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f427721ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5501bd20 con 0x558f5501ec90 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f427721ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5501be90 con 0x558f55024a80 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f427721ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f55095e40 con 0x558f55024160 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f427721ab80 10 client.0 _ll_register_callbacks cb 0x558f54f0bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f42695fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f5501b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f42695fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f5501b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:43188/0 (socket says 172.21.15.39:43188) 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f42695fc700 1 -- 172.21.15.39:0/2594478735 learned_addr learned my addr 172.21.15.39:0/2594478735 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f4268dfb700 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f5501ec90 0x558f5501dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f4263fff700 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f5501b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f42695fc700 1 -- 172.21.15.39:0/2594478735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 msgr2=0x558f5501b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f42695fc700 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f5501b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f42695fc700 1 -- 172.21.15.39:0/2594478735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f5501ec90 msgr2=0x558f5501dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f42695fc700 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f5501ec90 0x558f5501dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f42695fc700 1 -- 172.21.15.39:0/2594478735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f54de7350 con 0x558f55024a80 2024-05-24T11:46:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f4263fff700 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f5501b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:46:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f42695fc700 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f5501b610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f425c014e20 tx=0x7f425c014e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66f93e66cefc2eff server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.637+0000 7f4243fff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:46:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.638+0000 7f4243fff700 1 -- 172.21.15.39:0/2594478735 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c015a20 con 0x558f55024a80 2024-05-24T11:46:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.638+0000 7f4243fff700 1 -- 172.21.15.39:0/2594478735 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f425c015b80 con 0x558f55024a80 2024-05-24T11:46:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.638+0000 7f427721ab80 1 -- 172.21.15.39:0/2594478735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f425809b430 con 0x558f55024a80 2024-05-24T11:46:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.638+0000 7f4243fff700 1 -- 172.21.15.39:0/2594478735 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f425c015d40 con 0x558f55024a80 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f427721ab80 10 client.25682 fetch_fsmap learned FSMap version 29 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f427721ab80 1 -- 172.21.15.39:0/2594478735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558f54debc70 con 0x558f55024a80 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f4243fff700 1 -- 172.21.15.39:0/2594478735 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f425809b430 con 0x558f55024a80 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f4243fff700 1 -- 172.21.15.39:0/2594478735 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f425c02a3f0 con 0x558f55024a80 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f427721ab80 10 client.25682 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f427721ab80 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f427721ab80 20 client.25682 populate_metadata read hostname 'smithi039' 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f427721ab80 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558f550976c0 0x558f55099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f427721ab80 4 client.25682 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:46:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.639+0000 7f427721ab80 1 -- 172.21.15.39:0/2594478735 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558f54f5ee60 con 0x558f550976c0 2024-05-24T11:46:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.640+0000 7f4268dfb700 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558f550976c0 0x558f55099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.641+0000 7f4243fff700 10 client.25682 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.641+0000 7f4268dfb700 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558f550976c0 0x558f55099ae0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922512a1374226fd server_cookie=e0ffc3f1eb36f13c in_seq=0 out_seq=0 2024-05-24T11:46:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.669+0000 7f4243fff700 1 -- 172.21.15.39:0/2594478735 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558f54f5ee60 con 0x558f550976c0 2024-05-24T11:46:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.669+0000 7f4243fff700 10 client.25682 handle_command_reply: tid=0 2024-05-24T11:46:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.669+0000 7f427721ab80 1 client.25682 shutdown 2024-05-24T11:46:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 -- 172.21.15.39:0/2594478735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 msgr2=0x558f5501b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f5501b610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f425c014e20 tx=0x7f425c014e50 comp rx=0 tx=0).stop 2024-05-24T11:46:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 -- 172.21.15.39:0/2594478735 shutdown_connections 2024-05-24T11:46:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558f550976c0 0x558f55099ae0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f55024a80 0x558f5501b610 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f5501ec90 0x558f5501dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 --2- 172.21.15.39:0/2594478735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f55024160 0x558f5501b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 -- 172.21.15.39:0/2594478735 >> 172.21.15.39:0/2594478735 conn(0x558f54f741b0 msgr2=0x558f550264d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f4243fff700 0 client.25682 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 -- 172.21.15.39:0/2594478735 shutdown_connections 2024-05-24T11:46:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 1 -- 172.21.15.39:0/2594478735 wait complete. 2024-05-24T11:46:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.670+0000 7f427721ab80 20 client.25682 trim_cache size 0 max 16384 2024-05-24T11:46:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:46:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:46:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:46:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:46:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.671+0000 7f426b05f700 1 -- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f42540a1310 msgr2=0x7f42540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.671+0000 7f426b05f700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f42540a1310 0x7f42540a37d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f425c007800 tx=0x7f425c007750 comp rx=0 tx=0).stop 2024-05-24T11:46:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.671+0000 7f426b05f700 1 -- 172.21.15.39:0/3336775418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4264101110 msgr2=0x7f4264072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.671+0000 7f426b05f700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4264101110 0x7f4264072660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4258009990 tx=0x7f425800b840 comp rx=0 tx=0).stop 2024-05-24T11:46:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.675+0000 7f426b05f700 1 -- 172.21.15.39:0/3336775418 shutdown_connections 2024-05-24T11:46:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.675+0000 7f426b05f700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426406b7b0 0x7f426406bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.675+0000 7f426b05f700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f42540a1310 0x7f42540a37d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.675+0000 7f426b05f700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4264101110 0x7f4264072660 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.675+0000 7f426b05f700 1 --2- 172.21.15.39:0/3336775418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4264106230 0x7f4264072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.675+0000 7f426b05f700 1 -- 172.21.15.39:0/3336775418 >> 172.21.15.39:0/3336775418 conn(0x7f42640fc9b0 msgr2=0x7f42640fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.678+0000 7f426b05f700 1 -- 172.21.15.39:0/3336775418 shutdown_connections 2024-05-24T11:46:07.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:07.678+0000 7f426b05f700 1 -- 172.21.15.39:0/3336775418 wait complete. 2024-05-24T11:46:07.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:07 smithi053 ceph-mon[47448]: pgmap v243: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 14 MiB/s wr, 1.22k op/s 2024-05-24T11:46:07.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:07 smithi137 ceph-mon[47907]: pgmap v243: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 14 MiB/s wr, 1.22k op/s 2024-05-24T11:46:07.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:07 smithi039 ceph-mon[42919]: pgmap v243: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 14 MiB/s wr, 1.22k op/s 2024-05-24T11:46:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 405.0 s, 1053.0 tps, lat 94.629 ms stddev 37.892 2024-05-24T11:46:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:09 smithi053 ceph-mon[47448]: pgmap v244: 97 pgs: 97 active+clean; 5.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-05-24T11:46:09.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:09 smithi053 ceph-mon[47448]: osdmap e84: 12 total, 12 up, 12 in 2024-05-24T11:46:09.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:09 smithi137 ceph-mon[47907]: pgmap v244: 97 pgs: 97 active+clean; 5.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-05-24T11:46:09.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:09 smithi137 ceph-mon[47907]: osdmap e84: 12 total, 12 up, 12 in 2024-05-24T11:46:09.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:09 smithi039 ceph-mon[42919]: pgmap v244: 97 pgs: 97 active+clean; 5.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-05-24T11:46:09.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:09 smithi039 ceph-mon[42919]: osdmap e84: 12 total, 12 up, 12 in 2024-05-24T11:46:11.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:11 smithi053 ceph-mon[47448]: pgmap v246: 97 pgs: 97 active+clean; 5.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 15 MiB/s wr, 1.33k op/s 2024-05-24T11:46:11.747 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:11 smithi137 ceph-mon[47907]: pgmap v246: 97 pgs: 97 active+clean; 5.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 15 MiB/s wr, 1.33k op/s 2024-05-24T11:46:11.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:11 smithi039 ceph-mon[42919]: pgmap v246: 97 pgs: 97 active+clean; 5.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 15 MiB/s wr, 1.33k op/s 2024-05-24T11:46:13.741 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:13 smithi053 ceph-mon[47448]: pgmap v247: 97 pgs: 97 active+clean; 5.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 15 MiB/s wr, 1.33k op/s 2024-05-24T11:46:13.746 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:13 smithi137 ceph-mon[47907]: pgmap v247: 97 pgs: 97 active+clean; 5.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 15 MiB/s wr, 1.33k op/s 2024-05-24T11:46:13.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:13 smithi039 ceph-mon[42919]: pgmap v247: 97 pgs: 97 active+clean; 5.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 15 MiB/s wr, 1.33k op/s 2024-05-24T11:46:14.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 410.0 s, 1152.0 tps, lat 87.020 ms stddev 32.670 2024-05-24T11:46:15.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:15 smithi053 ceph-mon[47448]: pgmap v248: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 20 MiB/s wr, 1.37k op/s 2024-05-24T11:46:15.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:15 smithi053 ceph-mon[47448]: osdmap e85: 12 total, 12 up, 12 in 2024-05-24T11:46:15.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:15 smithi137 ceph-mon[47907]: pgmap v248: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 20 MiB/s wr, 1.37k op/s 2024-05-24T11:46:15.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:15 smithi137 ceph-mon[47907]: osdmap e85: 12 total, 12 up, 12 in 2024-05-24T11:46:16.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:15 smithi039 ceph-mon[42919]: pgmap v248: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 20 MiB/s wr, 1.37k op/s 2024-05-24T11:46:16.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:15 smithi039 ceph-mon[42919]: osdmap e85: 12 total, 12 up, 12 in 2024-05-24T11:46:17.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:17 smithi053 ceph-mon[47448]: pgmap v250: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 16 MiB/s wr, 871 op/s 2024-05-24T11:46:17.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:17 smithi137 ceph-mon[47907]: pgmap v250: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 16 MiB/s wr, 871 op/s 2024-05-24T11:46:18.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:17 smithi039 ceph-mon[42919]: pgmap v250: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 16 MiB/s wr, 871 op/s 2024-05-24T11:46:19.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 415.0 s, 1123.6 tps, lat 88.993 ms stddev 116.774 2024-05-24T11:46:19.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:19 smithi053 ceph-mon[47448]: pgmap v251: 97 pgs: 97 active+clean; 5.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 27 MiB/s wr, 1.51k op/s 2024-05-24T11:46:19.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:19 smithi137 ceph-mon[47907]: pgmap v251: 97 pgs: 97 active+clean; 5.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 27 MiB/s wr, 1.51k op/s 2024-05-24T11:46:20.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:19 smithi039 ceph-mon[42919]: pgmap v251: 97 pgs: 97 active+clean; 5.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 27 MiB/s wr, 1.51k op/s 2024-05-24T11:46:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.852+0000 7fef9854a700 1 -- 172.21.15.39:0/2934483398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef90105170 msgr2=0x7fef9010ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.852+0000 7fef9854a700 1 --2- 172.21.15.39:0/2934483398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef90105170 0x7fef9010ccd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fef88009f90 tx=0x7fef8800b330 comp rx=0 tx=0).stop 2024-05-24T11:46:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.856+0000 7fef9854a700 1 -- 172.21.15.39:0/2934483398 shutdown_connections 2024-05-24T11:46:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.856+0000 7fef9854a700 1 --2- 172.21.15.39:0/2934483398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef90100b60 0x7fef90100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.856+0000 7fef9854a700 1 --2- 172.21.15.39:0/2934483398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef90105170 0x7fef9010ccd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.856+0000 7fef9854a700 1 --2- 172.21.15.39:0/2934483398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef90101510 0x7fef90104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.856+0000 7fef9854a700 1 -- 172.21.15.39:0/2934483398 >> 172.21.15.39:0/2934483398 conn(0x7fef90075130 msgr2=0x7fef90075540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.857+0000 7fef9854a700 1 -- 172.21.15.39:0/2934483398 shutdown_connections 2024-05-24T11:46:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.857+0000 7fef9854a700 1 -- 172.21.15.39:0/2934483398 wait complete. 2024-05-24T11:46:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.858+0000 7fef9854a700 1 Processor -- start 2024-05-24T11:46:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.858+0000 7fef9854a700 1 -- start start 2024-05-24T11:46:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.858+0000 7fef9854a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef90100b60 0x7fef9010f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.858+0000 7fef9854a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef90105170 0x7fef9010f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.858+0000 7fef9854a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef9010fb40 0x7fef900718e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.858+0000 7fef9854a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90071e20 con 0x7fef90100b60 2024-05-24T11:46:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.858+0000 7fef9854a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90071f90 con 0x7fef90105170 2024-05-24T11:46:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.858+0000 7fef9854a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90072150 con 0x7fef9010fb40 2024-05-24T11:46:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.858+0000 7fef96ae7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef9010fb40 0x7fef900718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.859+0000 7fef96ae7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef9010fb40 0x7fef900718e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:33316/0 (socket says 172.21.15.39:33316) 2024-05-24T11:46:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.859+0000 7fef96ae7700 1 -- 172.21.15.39:0/1970056039 learned_addr learned my addr 172.21.15.39:0/1970056039 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.859+0000 7fef962e6700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef90100b60 0x7fef9010f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.859+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef90105170 0x7fef9010f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.859+0000 7fef95ae5700 1 -- 172.21.15.39:0/1970056039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef9010fb40 msgr2=0x7fef900718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.859+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef9010fb40 0x7fef900718e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.859+0000 7fef95ae5700 1 -- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef90100b60 msgr2=0x7fef9010f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.859+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef90100b60 0x7fef9010f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.859+0000 7fef95ae5700 1 -- 172.21.15.39:0/1970056039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef88009c40 con 0x7fef90105170 2024-05-24T11:46:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.860+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef90105170 0x7fef9010f600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef84005fd0 tx=0x7fef8400eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.860+0000 7fef8f7fe700 1 -- 172.21.15.39:0/1970056039 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84010900 con 0x7fef90105170 2024-05-24T11:46:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.860+0000 7fef8f7fe700 1 -- 172.21.15.39:0/1970056039 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef84018420 con 0x7fef90105170 2024-05-24T11:46:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.860+0000 7fef8f7fe700 1 -- 172.21.15.39:0/1970056039 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8400f680 con 0x7fef90105170 2024-05-24T11:46:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.860+0000 7fef9854a700 1 -- 172.21.15.39:0/1970056039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef90072430 con 0x7fef90105170 2024-05-24T11:46:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.860+0000 7fef9854a700 1 -- 172.21.15.39:0/1970056039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef900728f0 con 0x7fef90105170 2024-05-24T11:46:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.863+0000 7fef8f7fe700 1 -- 172.21.15.39:0/1970056039 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fef8400f7e0 con 0x7fef90105170 2024-05-24T11:46:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.864+0000 7fef8f7fe700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef780a1410 0x7fef780a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.864+0000 7fef8f7fe700 1 -- 172.21.15.39:0/1970056039 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fef84014070 con 0x7fef90105170 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.864+0000 7fef962e6700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef780a1410 0x7fef780a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.865+0000 7fefa4705b80 1 Processor -- start 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.865+0000 7fef962e6700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef780a1410 0x7fef780a38d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef80009c60 tx=0x7fef80009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.865+0000 7fefa4705b80 1 -- start start 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.865+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f83950 0x555a88f83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.865+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f7df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.865+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f7e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.865+0000 7fefa4705b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88ed50f0 con 0x555a88f84270 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.865+0000 7fefa4705b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88ed2ab0 con 0x555a88f83950 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fefa4705b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88ece3a0 con 0x555a88f7e480 2024-05-24T11:46:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef95ae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f7df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef95ae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f7df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32846/0 (socket says 172.21.15.39:32846) 2024-05-24T11:46:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef95ae5700 1 -- 172.21.15.39:0/1985288960 learned_addr learned my addr 172.21.15.39:0/1985288960 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef962e6700 1 --2- 172.21.15.39:0/1985288960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f83950 0x555a88f83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef96ae7700 1 --2- 172.21.15.39:0/1985288960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f7e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef95ae5700 1 -- 172.21.15.39:0/1985288960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 msgr2=0x555a88f7e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1985288960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f7e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef95ae5700 1 -- 172.21.15.39:0/1985288960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f83950 msgr2=0x555a88f83d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1985288960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f83950 0x555a88f83d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef95ae5700 1 -- 172.21.15.39:0/1985288960 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a88d4f350 con 0x555a88f84270 2024-05-24T11:46:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.866+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1985288960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f7df40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef840ca300 tx=0x7fef84099870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d5303b294ec915 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.867+0000 7fef8cff9700 1 -- 172.21.15.39:0/1985288960 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8409a360 con 0x555a88f84270 2024-05-24T11:46:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.867+0000 7fef8cff9700 1 -- 172.21.15.39:0/1985288960 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef84099c60 con 0x555a88f84270 2024-05-24T11:46:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.867+0000 7fef8cff9700 1 -- 172.21.15.39:0/1985288960 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8409a120 con 0x555a88f84270 2024-05-24T11:46:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.868+0000 7fefa4705b80 1 -- 172.21.15.39:0/1985288960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 msgr2=0x555a88f7df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.868+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1985288960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f7df40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef840ca300 tx=0x7fef84099870 comp rx=0 tx=0).stop 2024-05-24T11:46:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.868+0000 7fefa4705b80 1 -- 172.21.15.39:0/1985288960 shutdown_connections 2024-05-24T11:46:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.868+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1985288960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f7e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.868+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1985288960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f83950 0x555a88f83d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.868+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1985288960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f7df40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.868+0000 7fefa4705b80 1 -- 172.21.15.39:0/1985288960 >> 172.21.15.39:0/1985288960 conn(0x555a88edc1b0 msgr2=0x555a88f78840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.869+0000 7fefa4705b80 1 -- 172.21.15.39:0/1985288960 shutdown_connections 2024-05-24T11:46:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.869+0000 7fefa4705b80 1 -- 172.21.15.39:0/1985288960 wait complete. 2024-05-24T11:46:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.869+0000 7fefa4705b80 1 -- start start 2024-05-24T11:46:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.869+0000 7fefa4705b80 1 Processor -- start 2024-05-24T11:46:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.869+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f87620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.869+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f83950 0x555a88f87b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.869+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f84270 0x555a88f880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.869+0000 7fefa4705b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88e6a500 con 0x555a88f83950 2024-05-24T11:46:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.869+0000 7fefa4705b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88e6a1a0 con 0x555a88f84270 2024-05-24T11:46:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fefa4705b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88d51640 con 0x555a88f7e480 2024-05-24T11:46:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fefa4705b80 10 client.0 _ll_register_callbacks cb 0x555a88e73ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:46:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef962e6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f87620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef96ae7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f84270 0x555a88f880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef962e6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f87620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:33336/0 (socket says 172.21.15.39:33336) 2024-05-24T11:46:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef962e6700 1 -- 172.21.15.39:0/3916234745 learned_addr learned my addr 172.21.15.39:0/3916234745 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:20.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef95ae5700 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f83950 0x555a88f87b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef96ae7700 1 -- 172.21.15.39:0/3916234745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 msgr2=0x555a88f87620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef96ae7700 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f87620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef96ae7700 1 -- 172.21.15.39:0/3916234745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f83950 msgr2=0x555a88f87b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef96ae7700 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f83950 0x555a88f87b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef96ae7700 1 -- 172.21.15.39:0/3916234745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef900728f0 con 0x555a88f84270 2024-05-24T11:46:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.870+0000 7fef962e6700 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f87620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:46:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.871+0000 7fef96ae7700 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f84270 0x555a88f880a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef8800d010 tx=0x7fef88007650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6962596d4a3254d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.871+0000 7fef75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:46:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.871+0000 7fef75ffb700 1 -- 172.21.15.39:0/3916234745 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800e040 con 0x555a88f84270 2024-05-24T11:46:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.871+0000 7fef75ffb700 1 -- 172.21.15.39:0/3916234745 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fef88003c00 con 0x555a88f84270 2024-05-24T11:46:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.871+0000 7fefa4705b80 1 -- 172.21.15.39:0/3916234745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a88e27210 con 0x555a88f84270 2024-05-24T11:46:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.871+0000 7fef75ffb700 1 -- 172.21.15.39:0/3916234745 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef88003dc0 con 0x555a88f84270 2024-05-24T11:46:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.871+0000 7fefa4705b80 10 client.25694 fetch_fsmap learned FSMap version 29 2024-05-24T11:46:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.871+0000 7fefa4705b80 1 -- 172.21.15.39:0/3916234745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555a88cfbb00 con 0x555a88f84270 2024-05-24T11:46:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.872+0000 7fef75ffb700 1 -- 172.21.15.39:0/3916234745 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555a88e27210 con 0x555a88f84270 2024-05-24T11:46:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.872+0000 7fef75ffb700 1 -- 172.21.15.39:0/3916234745 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fef88019050 con 0x555a88f84270 2024-05-24T11:46:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.872+0000 7fefa4705b80 10 client.25694 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:46:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.872+0000 7fefa4705b80 10 client.25694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:46:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.872+0000 7fefa4705b80 20 client.25694 populate_metadata read hostname 'smithi039' 2024-05-24T11:46:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.872+0000 7fefa4705b80 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x555a88fa8c70 0x555a88fab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.872+0000 7fefa4705b80 4 client.25694 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:46:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.872+0000 7fefa4705b80 1 -- 172.21.15.39:0/3916234745 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a88f4d7b0 con 0x555a88fa8c70 2024-05-24T11:46:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.873+0000 7fef962e6700 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x555a88fa8c70 0x555a88fab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.873+0000 7fef962e6700 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x555a88fa8c70 0x555a88fab090 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17aea9bbb9589e4a server_cookie=b00ac0d363cab967 in_seq=0 out_seq=0 2024-05-24T11:46:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.873+0000 7fef75ffb700 10 client.25694 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.874+0000 7fef75ffb700 1 -- 172.21.15.39:0/3916234745 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a88f4d7b0 con 0x555a88fa8c70 2024-05-24T11:46:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.874+0000 7fef75ffb700 10 client.25694 handle_command_reply: tid=0 2024-05-24T11:46:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.874+0000 7fefa4705b80 1 client.25694 shutdown 2024-05-24T11:46:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.874+0000 7fefa4705b80 1 -- 172.21.15.39:0/3916234745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f84270 msgr2=0x555a88f880a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.874+0000 7fefa4705b80 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f84270 0x555a88f880a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef8800d010 tx=0x7fef88007650 comp rx=0 tx=0).stop 2024-05-24T11:46:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.875+0000 7fefa4705b80 1 -- 172.21.15.39:0/3916234745 shutdown_connections 2024-05-24T11:46:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.875+0000 7fefa4705b80 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f7e480 0x555a88f87620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.876+0000 7fefa4705b80 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x555a88fa8c70 0x555a88fab090 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.876+0000 7fefa4705b80 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f84270 0x555a88f880a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.876+0000 7fefa4705b80 1 --2- 172.21.15.39:0/3916234745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f83950 0x555a88f87b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.876+0000 7fefa4705b80 1 -- 172.21.15.39:0/3916234745 >> 172.21.15.39:0/3916234745 conn(0x555a88edc1b0 msgr2=0x555a88f7cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.876+0000 7fef75ffb700 0 client.25694 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.879+0000 7fefa4705b80 1 -- 172.21.15.39:0/3916234745 shutdown_connections 2024-05-24T11:46:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.879+0000 7fefa4705b80 1 -- 172.21.15.39:0/3916234745 wait complete. 2024-05-24T11:46:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.879+0000 7fefa4705b80 20 client.25694 trim_cache size 0 max 16384 2024-05-24T11:46:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.908+0000 7fefa4705b80 1 Processor -- start 2024-05-24T11:46:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.908+0000 7fefa4705b80 1 -- start start 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.908+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f83d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.908+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 0x555a88f78910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.908+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f74e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.908+0000 7fefa4705b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88f7a4e0 con 0x555a88f84270 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.908+0000 7fefa4705b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88f8a580 con 0x555a88f7e480 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.908+0000 7fefa4705b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88f85b70 con 0x555a88f83950 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef962e6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f83d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef95ae5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 0x555a88f78910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef962e6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f83d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:53740/0 (socket says 172.21.15.39:53740) 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef962e6700 1 -- 172.21.15.39:0/43233227 learned_addr learned my addr 172.21.15.39:0/43233227 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef96ae7700 1 --2- 172.21.15.39:0/43233227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f74e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef95ae5700 1 -- 172.21.15.39:0/43233227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 msgr2=0x555a88f83d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef95ae5700 1 --2- 172.21.15.39:0/43233227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f83d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef95ae5700 1 -- 172.21.15.39:0/43233227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 msgr2=0x555a88f74e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef95ae5700 1 --2- 172.21.15.39:0/43233227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f74e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef95ae5700 1 -- 172.21.15.39:0/43233227 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a88e94500 con 0x555a88f83950 2024-05-24T11:46:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef962e6700 1 --2- 172.21.15.39:0/43233227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f83d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:46:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.909+0000 7fef95ae5700 1 --2- 172.21.15.39:0/43233227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 0x555a88f78910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fef840ca300 tx=0x7fef840996c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4b878acc31681a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.910+0000 7fef8d7fa700 1 -- 172.21.15.39:0/43233227 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8409da90 con 0x555a88f83950 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.910+0000 7fef8d7fa700 1 -- 172.21.15.39:0/43233227 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef840a20e0 con 0x555a88f83950 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.910+0000 7fef8d7fa700 1 -- 172.21.15.39:0/43233227 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840a1c50 con 0x555a88f83950 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 -- 172.21.15.39:0/43233227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 msgr2=0x555a88f78910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 --2- 172.21.15.39:0/43233227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 0x555a88f78910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fef840ca300 tx=0x7fef840996c0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 -- 172.21.15.39:0/43233227 shutdown_connections 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 --2- 172.21.15.39:0/43233227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f74e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 --2- 172.21.15.39:0/43233227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f83d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 --2- 172.21.15.39:0/43233227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 0x555a88f78910 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 -- 172.21.15.39:0/43233227 >> 172.21.15.39:0/43233227 conn(0x555a88edc1b0 msgr2=0x555a88f86eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 -- 172.21.15.39:0/43233227 shutdown_connections 2024-05-24T11:46:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 -- 172.21.15.39:0/43233227 wait complete. 2024-05-24T11:46:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 -- start start 2024-05-24T11:46:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.911+0000 7fefa4705b80 1 Processor -- start 2024-05-24T11:46:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f9c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 0x555a88f9cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fefa4705b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fefa4705b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88ffd160 con 0x555a88f84270 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fefa4705b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88ffd2a0 con 0x555a88f7e480 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fefa4705b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a88ffd460 con 0x555a88f83950 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fefa4705b80 10 client.0 _ll_register_callbacks cb 0x555a88e73ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef962e6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f9c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef962e6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f9c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:53744/0 (socket says 172.21.15.39:53744) 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef962e6700 1 -- 172.21.15.39:0/1371897177 learned_addr learned my addr 172.21.15.39:0/1371897177 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 0x555a88f9cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef96ae7700 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef962e6700 1 -- 172.21.15.39:0/1371897177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 msgr2=0x555a88f9cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef962e6700 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 0x555a88f9cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef962e6700 1 -- 172.21.15.39:0/1371897177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 msgr2=0x555a88f9d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef962e6700 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f9d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.912+0000 7fef962e6700 1 -- 172.21.15.39:0/1371897177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef80017070 con 0x555a88f7e480 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.913+0000 7fef962e6700 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f9c690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef800041b0 tx=0x7fef800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3844bf2571b02e5b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.913+0000 7fef75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.913+0000 7fef75ffb700 1 -- 172.21.15.39:0/1371897177 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80023040 con 0x555a88f7e480 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.913+0000 7fef75ffb700 1 -- 172.21.15.39:0/1371897177 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fef8001eca0 con 0x555a88f7e480 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.913+0000 7fefa4705b80 1 -- 172.21.15.39:0/1371897177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef84099c60 con 0x555a88f7e480 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.913+0000 7fef75ffb700 1 -- 172.21.15.39:0/1371897177 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef8001ee60 con 0x555a88f7e480 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.913+0000 7fefa4705b80 10 client.25700 fetch_fsmap learned FSMap version 29 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.913+0000 7fefa4705b80 1 -- 172.21.15.39:0/1371897177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555a88d4f350 con 0x555a88f7e480 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.914+0000 7fef75ffb700 1 -- 172.21.15.39:0/1371897177 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef84099c60 con 0x555a88f7e480 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.914+0000 7fef75ffb700 1 -- 172.21.15.39:0/1371897177 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fef80003b80 con 0x555a88f7e480 2024-05-24T11:46:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.914+0000 7fefa4705b80 10 client.25700 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:46:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.914+0000 7fefa4705b80 10 client.25700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:46:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.914+0000 7fefa4705b80 20 client.25700 populate_metadata read hostname 'smithi039' 2024-05-24T11:46:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.914+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x555a88ffec50 0x555a89001070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.914+0000 7fefa4705b80 4 client.25700 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:46:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.914+0000 7fefa4705b80 1 -- 172.21.15.39:0/1371897177 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a88ec6e60 con 0x555a88ffec50 2024-05-24T11:46:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.915+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x555a88ffec50 0x555a89001070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.915+0000 7fef95ae5700 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x555a88ffec50 0x555a89001070 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f02154eef81ca289 server_cookie=e1bf6e5fb6d47fcd in_seq=0 out_seq=0 2024-05-24T11:46:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:20.915+0000 7fef75ffb700 10 client.25700 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:21.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:21 smithi053 ceph-mon[47448]: pgmap v252: 97 pgs: 97 active+clean; 5.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 26 MiB/s wr, 1.49k op/s 2024-05-24T11:46:21.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:21 smithi137 ceph-mon[47907]: pgmap v252: 97 pgs: 97 active+clean; 5.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 26 MiB/s wr, 1.49k op/s 2024-05-24T11:46:22.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:21 smithi039 ceph-mon[42919]: pgmap v252: 97 pgs: 97 active+clean; 5.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 26 MiB/s wr, 1.49k op/s 2024-05-24T11:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.349+0000 7fef75ffb700 1 -- 172.21.15.39:0/1371897177 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a88ec6e60 con 0x555a88ffec50 2024-05-24T11:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.349+0000 7fef75ffb700 10 client.25700 handle_command_reply: tid=0 2024-05-24T11:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.349+0000 7fefa4705b80 1 client.25700 shutdown 2024-05-24T11:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.351+0000 7fefa4705b80 1 -- 172.21.15.39:0/1371897177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 msgr2=0x555a88f9c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.351+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f9c690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef800041b0 tx=0x7fef800047c0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fefa4705b80 1 -- 172.21.15.39:0/1371897177 shutdown_connections 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x555a88ffec50 0x555a89001070 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a88f84270 0x555a88f9d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a88f7e480 0x555a88f9c690 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fefa4705b80 1 --2- 172.21.15.39:0/1371897177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a88f83950 0x555a88f9cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fefa4705b80 1 -- 172.21.15.39:0/1371897177 >> 172.21.15.39:0/1371897177 conn(0x555a88edc1b0 msgr2=0x555a88f87f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fef75ffb700 0 client.25700 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fefa4705b80 1 -- 172.21.15.39:0/1371897177 shutdown_connections 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fefa4705b80 1 -- 172.21.15.39:0/1371897177 wait complete. 2024-05-24T11:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.352+0000 7fefa4705b80 20 client.25700 trim_cache size 0 max 16384 2024-05-24T11:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 -- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef780a1410 msgr2=0x7fef780a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef780a1410 0x7fef780a38d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef80009c60 tx=0x7fef80009260 comp rx=0 tx=0).stop 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 -- 172.21.15.39:0/1970056039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef90105170 msgr2=0x7fef9010f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef90105170 0x7fef9010f600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef84005fd0 tx=0x7fef8400eca0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 -- 172.21.15.39:0/1970056039 shutdown_connections 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef9010fb40 0x7fef900718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fef780a1410 0x7fef780a38d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef90100b60 0x7fef9010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 --2- 172.21.15.39:0/1970056039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef90105170 0x7fef9010f600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 -- 172.21.15.39:0/1970056039 >> 172.21.15.39:0/1970056039 conn(0x7fef90075130 msgr2=0x7fef901062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 -- 172.21.15.39:0/1970056039 shutdown_connections 2024-05-24T11:46:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:22.353+0000 7fef9854a700 1 -- 172.21.15.39:0/1970056039 wait complete. 2024-05-24T11:46:22.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:22 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:46:22.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:22 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:46:23.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:22 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:46:23.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:23 smithi053 ceph-mon[47448]: pgmap v253: 97 pgs: 97 active+clean; 5.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 26 MiB/s wr, 1.49k op/s 2024-05-24T11:46:23.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:23 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:46:23.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:23 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:46:23.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:23 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:46:23.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:23 smithi137 ceph-mon[47907]: pgmap v253: 97 pgs: 97 active+clean; 5.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 26 MiB/s wr, 1.49k op/s 2024-05-24T11:46:23.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:23 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:46:23.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:23 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:46:23.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:23 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:46:24.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:23 smithi039 ceph-mon[42919]: pgmap v253: 97 pgs: 97 active+clean; 5.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 26 MiB/s wr, 1.49k op/s 2024-05-24T11:46:24.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:23 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:46:24.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:23 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:46:24.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:23 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:46:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 420.0 s, 1125.0 tps, lat 89.048 ms stddev 39.104 2024-05-24T11:46:25.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:25 smithi053 ceph-mon[47448]: pgmap v254: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2024-05-24T11:46:25.996 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:25 smithi137 ceph-mon[47907]: pgmap v254: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2024-05-24T11:46:26.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:25 smithi039 ceph-mon[42919]: pgmap v254: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2024-05-24T11:46:27.991 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:27 smithi053 ceph-mon[47448]: pgmap v255: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 21 MiB/s wr, 1.36k op/s 2024-05-24T11:46:27.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:27 smithi137 ceph-mon[47907]: pgmap v255: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 21 MiB/s wr, 1.36k op/s 2024-05-24T11:46:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:27 smithi039 ceph-mon[42919]: pgmap v255: 97 pgs: 97 active+clean; 5.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 21 MiB/s wr, 1.36k op/s 2024-05-24T11:46:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 425.0 s, 1129.2 tps, lat 88.328 ms stddev 29.135 2024-05-24T11:46:29.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:29 smithi137 ceph-mon[47907]: pgmap v256: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 30 MiB/s wr, 1.89k op/s 2024-05-24T11:46:30.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:29 smithi039 ceph-mon[42919]: pgmap v256: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 30 MiB/s wr, 1.89k op/s 2024-05-24T11:46:30.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:29 smithi053 ceph-mon[47448]: pgmap v256: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 30 MiB/s wr, 1.89k op/s 2024-05-24T11:46:32.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:31 smithi039 ceph-mon[42919]: pgmap v257: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-05-24T11:46:32.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:31 smithi053 ceph-mon[47448]: pgmap v257: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-05-24T11:46:32.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:31 smithi137 ceph-mon[47907]: pgmap v257: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-05-24T11:46:34.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:33 smithi039 ceph-mon[42919]: pgmap v258: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-05-24T11:46:34.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 430.0 s, 1095.0 tps, lat 91.370 ms stddev 52.742 2024-05-24T11:46:34.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:33 smithi053 ceph-mon[47448]: pgmap v258: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-05-24T11:46:34.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:33 smithi137 ceph-mon[47907]: pgmap v258: 97 pgs: 97 active+clean; 5.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-05-24T11:46:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.532+0000 7f521a5fc700 1 -- 172.21.15.39:0/3316940430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140fef90 msgr2=0x7f52140ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.532+0000 7f521a5fc700 1 --2- 172.21.15.39:0/3316940430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140fef90 0x7f52140ff410 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009990 tx=0x7f51fc009ca0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.536+0000 7f521a5fc700 1 -- 172.21.15.39:0/3316940430 shutdown_connections 2024-05-24T11:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.536+0000 7f521a5fc700 1 --2- 172.21.15.39:0/3316940430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52140ff950 0x7f521410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.536+0000 7f521a5fc700 1 --2- 172.21.15.39:0/3316940430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5214105f20 0x7f5214106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.536+0000 7f521a5fc700 1 --2- 172.21.15.39:0/3316940430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140fef90 0x7f52140ff410 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.536+0000 7f521a5fc700 1 -- 172.21.15.39:0/3316940430 >> 172.21.15.39:0/3316940430 conn(0x7f5214075f50 msgr2=0x7f52140fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.536+0000 7f521a5fc700 1 -- 172.21.15.39:0/3316940430 shutdown_connections 2024-05-24T11:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.536+0000 7f521a5fc700 1 -- 172.21.15.39:0/3316940430 wait complete. 2024-05-24T11:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.536+0000 7f521a5fc700 1 Processor -- start 2024-05-24T11:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.536+0000 7f521a5fc700 1 -- start start 2024-05-24T11:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f521a5fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52140fef90 0x7f52140725f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f521a5fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140ff950 0x7f5214072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f521a5fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214105f20 0x7f521406b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f521a5fc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521406c000 con 0x7f5214105f20 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f521a5fc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521406c170 con 0x7f52140ff950 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f521a5fc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521406c330 con 0x7f52140fef90 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f5218b99700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214105f20 0x7f521406b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f5218b99700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214105f20 0x7f521406b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40694/0 (socket says 172.21.15.39:40694) 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f5218b99700 1 -- 172.21.15.39:0/762848979 learned_addr learned my addr 172.21.15.39:0/762848979 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f520b7fe700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140ff950 0x7f5214072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f5213fff700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52140fef90 0x7f52140725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f520b7fe700 1 -- 172.21.15.39:0/762848979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52140fef90 msgr2=0x7f52140725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f520b7fe700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52140fef90 0x7f52140725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f520b7fe700 1 -- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214105f20 msgr2=0x7f521406b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f520b7fe700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214105f20 0x7f521406b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.537+0000 7f520b7fe700 1 -- 172.21.15.39:0/762848979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0095f0 con 0x7f52140ff950 2024-05-24T11:46:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.538+0000 7f5218b99700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214105f20 0x7f521406b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:46:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.538+0000 7f520b7fe700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140ff950 0x7f5214072b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0098e0 tx=0x7f51fc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.538+0000 7f5211ffb700 1 -- 172.21.15.39:0/762848979 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc01d070 con 0x7f52140ff950 2024-05-24T11:46:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.538+0000 7f521a5fc700 1 -- 172.21.15.39:0/762848979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5214070180 con 0x7f52140ff950 2024-05-24T11:46:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.538+0000 7f5211ffb700 1 -- 172.21.15.39:0/762848979 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51fc00fb00 con 0x7f52140ff950 2024-05-24T11:46:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.538+0000 7f521a5fc700 1 -- 172.21.15.39:0/762848979 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5214070680 con 0x7f52140ff950 2024-05-24T11:46:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.539+0000 7f5211ffb700 1 -- 172.21.15.39:0/762848979 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc003e70 con 0x7f52140ff950 2024-05-24T11:46:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.544+0000 7f52267b7b80 1 Processor -- start 2024-05-24T11:46:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.545+0000 7f5211ffb700 1 -- 172.21.15.39:0/762848979 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f51fc003fd0 con 0x7f52140ff950 2024-05-24T11:46:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.545+0000 7f52267b7b80 1 -- start start 2024-05-24T11:46:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.545+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd42940 0x55758bd42d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.545+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd46d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.545+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.545+0000 7f52267b7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bc910f0 con 0x55758bd42940 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.545+0000 7f52267b7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bc8eab0 con 0x55758bd47240 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.545+0000 7f52267b7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bc8a3a0 con 0x55758bd43260 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.545+0000 7f5213fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd46d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5213fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd46d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:46314/0 (socket says 172.21.15.39:46314) 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5213fff700 1 -- 172.21.15.39:0/192982966 learned_addr learned my addr 172.21.15.39:0/192982966 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5218b99700 1 --2- 172.21.15.39:0/192982966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd42940 0x55758bd42d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5213fff700 1 -- 172.21.15.39:0/192982966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 msgr2=0x55758bd476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5213fff700 1 --2- 172.21.15.39:0/192982966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5213fff700 1 -- 172.21.15.39:0/192982966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd42940 msgr2=0x55758bd42d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5213fff700 1 --2- 172.21.15.39:0/192982966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd42940 0x55758bd42d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5213fff700 1 -- 172.21.15.39:0/192982966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55758bb0b350 con 0x55758bd43260 2024-05-24T11:46:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5211ffb700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f52000a12d0 0x7f52000a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5213fff700 1 --2- 172.21.15.39:0/192982966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd46d00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f520400b580 tx=0x7f52040096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ddd97664689007b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f520a7fc700 1 -- 172.21.15.39:0/192982966 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204010040 con 0x55758bd43260 2024-05-24T11:46:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5218b99700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f52000a12d0 0x7f52000a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f5211ffb700 1 -- 172.21.15.39:0/762848979 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f51fc0cdae0 con 0x7f52140ff950 2024-05-24T11:46:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f520a7fc700 1 -- 172.21.15.39:0/192982966 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f520400ac80 con 0x55758bd43260 2024-05-24T11:46:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.546+0000 7f520a7fc700 1 -- 172.21.15.39:0/192982966 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204014b10 con 0x55758bd43260 2024-05-24T11:46:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.547+0000 7f52267b7b80 1 -- 172.21.15.39:0/192982966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 msgr2=0x55758bd46d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.547+0000 7f52267b7b80 1 --2- 172.21.15.39:0/192982966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd46d00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f520400b580 tx=0x7f52040096b0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.547+0000 7f5218b99700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f52000a12d0 0x7f52000a3790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f520c009fb0 tx=0x7f520c009210 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.548+0000 7f52267b7b80 1 -- 172.21.15.39:0/192982966 shutdown_connections 2024-05-24T11:46:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.548+0000 7f52267b7b80 1 --2- 172.21.15.39:0/192982966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.548+0000 7f52267b7b80 1 --2- 172.21.15.39:0/192982966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd42940 0x55758bd42d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.548+0000 7f52267b7b80 1 --2- 172.21.15.39:0/192982966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd46d00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.548+0000 7f52267b7b80 1 -- 172.21.15.39:0/192982966 >> 172.21.15.39:0/192982966 conn(0x55758bc981b0 msgr2=0x55758bd34840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.548+0000 7f52267b7b80 1 -- 172.21.15.39:0/192982966 shutdown_connections 2024-05-24T11:46:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.548+0000 7f52267b7b80 1 -- 172.21.15.39:0/192982966 wait complete. 2024-05-24T11:46:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.549+0000 7f52267b7b80 1 -- start start 2024-05-24T11:46:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.549+0000 7f52267b7b80 1 Processor -- start 2024-05-24T11:46:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 0x55758bd4ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd4bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd4c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f52267b7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bc26500 con 0x55758bd43260 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f52267b7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bc261a0 con 0x55758bd47240 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f52267b7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bb0d640 con 0x55758bd42940 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f52267b7b80 10 client.0 _ll_register_callbacks cb 0x55758bc2fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f5218b99700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd4c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f5218b99700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd4c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:42910/0 (socket says 172.21.15.39:42910) 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.550+0000 7f5218b99700 1 -- 172.21.15.39:0/1436561789 learned_addr learned my addr 172.21.15.39:0/1436561789 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f5218b99700 1 -- 172.21.15.39:0/1436561789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 msgr2=0x55758bd4ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f5218b99700 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 0x55758bd4ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f520b7fe700 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd4bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f5218b99700 1 -- 172.21.15.39:0/1436561789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 msgr2=0x55758bd4bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f5218b99700 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd4bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f5218b99700 1 -- 172.21.15.39:0/1436561789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5204009d60 con 0x55758bd47240 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f520b7fe700 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd4bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f5218b99700 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd4c2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f520c014e10 tx=0x7f520c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=499116850277e381 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f51f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f51f37fe700 1 -- 172.21.15.39:0/1436561789 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c011e60 con 0x55758bd47240 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.551+0000 7f51f37fe700 1 -- 172.21.15.39:0/1436561789 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f520c01c040 con 0x55758bd47240 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f52267b7b80 1 -- 172.21.15.39:0/1436561789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55758bbe3210 con 0x55758bd47240 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f51f37fe700 1 -- 172.21.15.39:0/1436561789 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f520c013bc0 con 0x55758bd47240 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f52267b7b80 10 client.25709 fetch_fsmap learned FSMap version 29 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f52267b7b80 1 -- 172.21.15.39:0/1436561789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55758bab7b00 con 0x55758bd47240 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f51f37fe700 1 -- 172.21.15.39:0/1436561789 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55758bbe3210 con 0x55758bd47240 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f51f37fe700 1 -- 172.21.15.39:0/1436561789 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f520c0213f0 con 0x55758bd47240 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f52267b7b80 10 client.25709 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:46:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f52267b7b80 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:46:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f52267b7b80 20 client.25709 populate_metadata read hostname 'smithi039' 2024-05-24T11:46:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55758bd3def0 0x55758bd40350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f52267b7b80 4 client.25709 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:46:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.552+0000 7f52267b7b80 1 -- 172.21.15.39:0/1436561789 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55758bd097b0 con 0x55758bd3def0 2024-05-24T11:46:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.553+0000 7f5213fff700 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55758bd3def0 0x55758bd40350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.554+0000 7f5213fff700 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55758bd3def0 0x55758bd40350 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db9b2517def9cfa server_cookie=ae36f2ec7a33fdc1 in_seq=0 out_seq=0 2024-05-24T11:46:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.554+0000 7f51f37fe700 10 client.25709 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.555+0000 7f51f37fe700 1 -- 172.21.15.39:0/1436561789 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55758bd097b0 con 0x55758bd3def0 2024-05-24T11:46:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.555+0000 7f51f37fe700 10 client.25709 handle_command_reply: tid=0 2024-05-24T11:46:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.555+0000 7f52267b7b80 1 client.25709 shutdown 2024-05-24T11:46:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.555+0000 7f52267b7b80 1 -- 172.21.15.39:0/1436561789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 msgr2=0x55758bd4c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.555+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd4c2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f520c014e10 tx=0x7f520c004750 comp rx=0 tx=0).stop 2024-05-24T11:46:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.558+0000 7f52267b7b80 1 -- 172.21.15.39:0/1436561789 shutdown_connections 2024-05-24T11:46:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.558+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 0x55758bd4ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.558+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55758bd3def0 0x55758bd40350 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.558+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd4c2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.558+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1436561789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd4bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.558+0000 7f52267b7b80 1 -- 172.21.15.39:0/1436561789 >> 172.21.15.39:0/1436561789 conn(0x55758bc981b0 msgr2=0x55758bd38f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.558+0000 7f51f37fe700 0 client.25709 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.558+0000 7f52267b7b80 1 -- 172.21.15.39:0/1436561789 shutdown_connections 2024-05-24T11:46:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.558+0000 7f52267b7b80 1 -- 172.21.15.39:0/1436561789 wait complete. 2024-05-24T11:46:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.559+0000 7f52267b7b80 20 client.25709 trim_cache size 0 max 16384 2024-05-24T11:46:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.583+0000 7f52267b7b80 1 Processor -- start 2024-05-24T11:46:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.583+0000 7f52267b7b80 1 -- start start 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd42940 0x55758bd34910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd53430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd47240 0x55758bd53970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f52267b7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bd3b570 con 0x55758bd47240 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f52267b7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bd45ea0 con 0x55758bd42940 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f52267b7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bd2ca40 con 0x55758bd43260 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5218b99700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd47240 0x55758bd53970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5218b99700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd47240 0x55758bd53970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40720/0 (socket says 172.21.15.39:40720) 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5218b99700 1 -- 172.21.15.39:0/1925834158 learned_addr learned my addr 172.21.15.39:0/1925834158 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5213fff700 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd42940 0x55758bd34910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f520b7fe700 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd53430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5218b99700 1 -- 172.21.15.39:0/1925834158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 msgr2=0x55758bd53430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5218b99700 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd53430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5218b99700 1 -- 172.21.15.39:0/1925834158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd42940 msgr2=0x55758bd34910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5218b99700 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd42940 0x55758bd34910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5218b99700 1 -- 172.21.15.39:0/1925834158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55758bc50500 con 0x55758bd47240 2024-05-24T11:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f5213fff700 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd42940 0x55758bd34910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.584+0000 7f520b7fe700 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd53430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.585+0000 7f5218b99700 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd47240 0x55758bd53970 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f520c000c00 tx=0x7f520c006b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ce3cebceccd1bc server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.585+0000 7f520affd700 1 -- 172.21.15.39:0/1925834158 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c03a070 con 0x55758bd47240 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.585+0000 7f520affd700 1 -- 172.21.15.39:0/1925834158 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f520c0213f0 con 0x55758bd47240 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.585+0000 7f520affd700 1 -- 172.21.15.39:0/1925834158 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c006030 con 0x55758bd47240 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.586+0000 7f52267b7b80 1 -- 172.21.15.39:0/1925834158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd47240 msgr2=0x55758bd53970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.586+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd47240 0x55758bd53970 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f520c000c00 tx=0x7f520c006b60 comp rx=0 tx=0).stop 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.586+0000 7f52267b7b80 1 -- 172.21.15.39:0/1925834158 shutdown_connections 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.586+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd47240 0x55758bd53970 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.586+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd42940 0x55758bd34910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.586+0000 7f52267b7b80 1 --2- 172.21.15.39:0/1925834158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd43260 0x55758bd53430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.586+0000 7f52267b7b80 1 -- 172.21.15.39:0/1925834158 >> 172.21.15.39:0/1925834158 conn(0x55758bc981b0 msgr2=0x55758bd4a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.587+0000 7f52267b7b80 1 -- 172.21.15.39:0/1925834158 shutdown_connections 2024-05-24T11:46:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.587+0000 7f52267b7b80 1 -- 172.21.15.39:0/1925834158 wait complete. 2024-05-24T11:46:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.588+0000 7f52267b7b80 1 -- start start 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.588+0000 7f52267b7b80 1 Processor -- start 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.588+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 0x55758bd53630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.588+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd41ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.588+0000 7f52267b7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.588+0000 7f52267b7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bd341e0 con 0x55758bd43260 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.588+0000 7f52267b7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bd34320 con 0x55758bd47240 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.588+0000 7f52267b7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758bd344e0 con 0x55758bd42940 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.588+0000 7f52267b7b80 10 client.0 _ll_register_callbacks cb 0x55758bc2fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd41ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f520b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd41ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40734/0 (socket says 172.21.15.39:40734) 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f520b7fe700 1 -- 172.21.15.39:0/2395970485 learned_addr learned my addr 172.21.15.39:0/2395970485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f5213fff700 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 0x55758bd53630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f5218b99700 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f520b7fe700 1 -- 172.21.15.39:0/2395970485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 msgr2=0x55758bd53630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f520b7fe700 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 0x55758bd53630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f520b7fe700 1 -- 172.21.15.39:0/2395970485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 msgr2=0x55758bd421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f520b7fe700 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.589+0000 7f520b7fe700 1 -- 172.21.15.39:0/2395970485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55758bb0b350 con 0x55758bd43260 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.590+0000 7f5213fff700 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 0x55758bd53630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.590+0000 7f5218b99700 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.590+0000 7f520b7fe700 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd41ca0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f51fc09aab0 tx=0x7f51fc00ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9107d7a2efda5c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.590+0000 7f51f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:46:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.590+0000 7f51f37fe700 1 -- 172.21.15.39:0/2395970485 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc09a8f0 con 0x55758bd43260 2024-05-24T11:46:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.590+0000 7f51f37fe700 1 -- 172.21.15.39:0/2395970485 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f51fc0d5040 con 0x55758bd43260 2024-05-24T11:46:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.590+0000 7f52267b7b80 1 -- 172.21.15.39:0/2395970485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f520400ac80 con 0x55758bd43260 2024-05-24T11:46:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f51f37fe700 1 -- 172.21.15.39:0/2395970485 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51fc09a710 con 0x55758bd43260 2024-05-24T11:46:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f52267b7b80 10 client.16002 fetch_fsmap learned FSMap version 29 2024-05-24T11:46:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f52267b7b80 1 -- 172.21.15.39:0/2395970485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f5214070680 con 0x55758bd43260 2024-05-24T11:46:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f51f37fe700 1 -- 172.21.15.39:0/2395970485 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f520400ac80 con 0x55758bd43260 2024-05-24T11:46:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f51f37fe700 1 -- 172.21.15.39:0/2395970485 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f51fc09a2a0 con 0x55758bd43260 2024-05-24T11:46:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f52267b7b80 10 client.16002 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:46:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f52267b7b80 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:46:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f52267b7b80 20 client.16002 populate_metadata read hostname 'smithi039' 2024-05-24T11:46:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f52267b7b80 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55758bd4df20 0x55758bdacc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f52267b7b80 4 client.16002 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:46:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.591+0000 7f52267b7b80 1 -- 172.21.15.39:0/2395970485 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55758bc82e60 con 0x55758bd4df20 2024-05-24T11:46:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.592+0000 7f5213fff700 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55758bd4df20 0x55758bdacc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.592+0000 7f5213fff700 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55758bd4df20 0x55758bdacc90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c34a6e618b721d server_cookie=f55c37db1d41041e in_seq=0 out_seq=0 2024-05-24T11:46:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.592+0000 7f51f37fe700 10 client.16002 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.645+0000 7f51f37fe700 1 -- 172.21.15.39:0/2395970485 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55758bc82e60 con 0x55758bd4df20 2024-05-24T11:46:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.645+0000 7f51f37fe700 10 client.16002 handle_command_reply: tid=0 2024-05-24T11:46:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.645+0000 7f52267b7b80 1 client.16002 shutdown 2024-05-24T11:46:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.645+0000 7f52267b7b80 1 -- 172.21.15.39:0/2395970485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 msgr2=0x55758bd41ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.645+0000 7f52267b7b80 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd41ca0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f51fc09aab0 tx=0x7f51fc00ff50 comp rx=0 tx=0).stop 2024-05-24T11:46:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f52267b7b80 1 -- 172.21.15.39:0/2395970485 shutdown_connections 2024-05-24T11:46:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f52267b7b80 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55758bd42940 0x55758bd53630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f52267b7b80 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55758bd4df20 0x55758bdacc90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f52267b7b80 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55758bd47240 0x55758bd421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f52267b7b80 1 --2- 172.21.15.39:0/2395970485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55758bd43260 0x55758bd41ca0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f52267b7b80 1 -- 172.21.15.39:0/2395970485 >> 172.21.15.39:0/2395970485 conn(0x55758bc981b0 msgr2=0x55758bd39650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f51f37fe700 0 client.16002 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:46:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:46:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:46:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:46:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f52267b7b80 1 -- 172.21.15.39:0/2395970485 shutdown_connections 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f52267b7b80 1 -- 172.21.15.39:0/2395970485 wait complete. 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.646+0000 7f52267b7b80 20 client.16002 trim_cache size 0 max 16384 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 -- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f52000a12d0 msgr2=0x7f52000a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f52000a12d0 0x7f52000a3790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f520c009fb0 tx=0x7f520c009210 comp rx=0 tx=0).stop 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 -- 172.21.15.39:0/762848979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140ff950 msgr2=0x7f5214072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140ff950 0x7f5214072b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0098e0 tx=0x7f51fc005740 comp rx=0 tx=0).stop 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 -- 172.21.15.39:0/762848979 shutdown_connections 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52140fef90 0x7f52140725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f52000a12d0 0x7f52000a3790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214105f20 0x7f521406b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 --2- 172.21.15.39:0/762848979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140ff950 0x7f5214072b70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.647+0000 7f521a5fc700 1 -- 172.21.15.39:0/762848979 >> 172.21.15.39:0/762848979 conn(0x7f5214075f50 msgr2=0x7f5214104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.648+0000 7f521a5fc700 1 -- 172.21.15.39:0/762848979 shutdown_connections 2024-05-24T11:46:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:35.648+0000 7f521a5fc700 1 -- 172.21.15.39:0/762848979 wait complete. 2024-05-24T11:46:36.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:35 smithi039 ceph-mon[42919]: pgmap v259: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 27 MiB/s wr, 1.82k op/s 2024-05-24T11:46:36.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:35 smithi053 ceph-mon[47448]: pgmap v259: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 27 MiB/s wr, 1.82k op/s 2024-05-24T11:46:36.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:35 smithi137 ceph-mon[47907]: pgmap v259: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 27 MiB/s wr, 1.82k op/s 2024-05-24T11:46:38.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:37 smithi039 ceph-mon[42919]: pgmap v260: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-05-24T11:46:38.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:37 smithi053 ceph-mon[47448]: pgmap v260: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-05-24T11:46:38.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:37 smithi137 ceph-mon[47907]: pgmap v260: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-05-24T11:46:39.118 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 435.0 s, 1088.5 tps, lat 91.939 ms stddev 28.875 2024-05-24T11:46:40.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:39 smithi039 ceph-mon[42919]: pgmap v261: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 24 MiB/s wr, 1.73k op/s 2024-05-24T11:46:40.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:39 smithi053 ceph-mon[47448]: pgmap v261: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 24 MiB/s wr, 1.73k op/s 2024-05-24T11:46:40.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:39 smithi137 ceph-mon[47907]: pgmap v261: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 24 MiB/s wr, 1.73k op/s 2024-05-24T11:46:42.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:41 smithi039 ceph-mon[42919]: pgmap v262: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:46:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:41 smithi053 ceph-mon[47448]: pgmap v262: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:46:42.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:41 smithi137 ceph-mon[47907]: pgmap v262: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:46:44.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:43 smithi039 ceph-mon[42919]: pgmap v263: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:46:44.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 440.0 s, 1047.1 tps, lat 95.346 ms stddev 32.665 2024-05-24T11:46:44.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:43 smithi053 ceph-mon[47448]: pgmap v263: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:46:44.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:43 smithi137 ceph-mon[47907]: pgmap v263: 97 pgs: 97 active+clean; 5.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-05-24T11:46:46.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:45 smithi039 ceph-mon[42919]: pgmap v264: 97 pgs: 97 active+clean; 5.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 22 MiB/s wr, 1.74k op/s 2024-05-24T11:46:46.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:45 smithi053 ceph-mon[47448]: pgmap v264: 97 pgs: 97 active+clean; 5.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 22 MiB/s wr, 1.74k op/s 2024-05-24T11:46:46.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:45 smithi137 ceph-mon[47907]: pgmap v264: 97 pgs: 97 active+clean; 5.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 22 MiB/s wr, 1.74k op/s 2024-05-24T11:46:48.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:47 smithi039 ceph-mon[42919]: pgmap v265: 97 pgs: 97 active+clean; 5.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2024-05-24T11:46:48.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:47 smithi053 ceph-mon[47448]: pgmap v265: 97 pgs: 97 active+clean; 5.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2024-05-24T11:46:48.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:47 smithi137 ceph-mon[47907]: pgmap v265: 97 pgs: 97 active+clean; 5.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2024-05-24T11:46:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.816+0000 7f865fa45700 1 -- 172.21.15.39:0/847598471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f865810f630 msgr2=0x7f865810fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.816+0000 7f865fa45700 1 --2- 172.21.15.39:0/847598471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f865810f630 0x7f865810fa10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f86540098f0 tx=0x7f8654009c00 comp rx=0 tx=0).stop 2024-05-24T11:46:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.816+0000 7f865fa45700 1 -- 172.21.15.39:0/847598471 shutdown_connections 2024-05-24T11:46:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.816+0000 7f865fa45700 1 --2- 172.21.15.39:0/847598471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f865810f630 0x7f865810fa10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.816+0000 7f865fa45700 1 --2- 172.21.15.39:0/847598471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8658107fa0 0x7f8658108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.816+0000 7f865fa45700 1 --2- 172.21.15.39:0/847598471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865810d440 0x7f865810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.816+0000 7f865fa45700 1 -- 172.21.15.39:0/847598471 >> 172.21.15.39:0/847598471 conn(0x7f865806bf80 msgr2=0x7f865806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.817+0000 7f865fa45700 1 -- 172.21.15.39:0/847598471 shutdown_connections 2024-05-24T11:46:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.817+0000 7f865fa45700 1 -- 172.21.15.39:0/847598471 wait complete. 2024-05-24T11:46:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865fa45700 1 Processor -- start 2024-05-24T11:46:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865fa45700 1 -- start start 2024-05-24T11:46:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865fa45700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8658107fa0 0x7f86581a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865fa45700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f865810d440 0x7f86581a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865fa45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865810f630 0x7f865819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865fa45700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865819df50 con 0x7f865810f630 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865fa45700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865819e0c0 con 0x7f865810d440 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865fa45700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865819e280 con 0x7f8658107fa0 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865d7e1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8658107fa0 0x7f86581a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865dfe2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865810f630 0x7f865819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865d7e1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8658107fa0 0x7f86581a37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:52162/0 (socket says 172.21.15.39:52162) 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865d7e1700 1 -- 172.21.15.39:0/2659316190 learned_addr learned my addr 172.21.15.39:0/2659316190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865cfe0700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f865810d440 0x7f86581a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865d7e1700 1 -- 172.21.15.39:0/2659316190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f865810d440 msgr2=0x7f86581a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865d7e1700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f865810d440 0x7f86581a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865d7e1700 1 -- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865810f630 msgr2=0x7f865819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.818+0000 7f865d7e1700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865810f630 0x7f865819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.819+0000 7f865d7e1700 1 -- 172.21.15.39:0/2659316190 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654009520 con 0x7f8658107fa0 2024-05-24T11:46:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.819+0000 7f865dfe2700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865810f630 0x7f865819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:46:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.819+0000 7f865d7e1700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8658107fa0 0x7f86581a37c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8654009230 tx=0x7f865400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.819+0000 7f864e7fc700 1 -- 172.21.15.39:0/2659316190 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865401c070 con 0x7f8658107fa0 2024-05-24T11:46:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.819+0000 7f865fa45700 1 -- 172.21.15.39:0/2659316190 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865819e500 con 0x7f8658107fa0 2024-05-24T11:46:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.819+0000 7f864e7fc700 1 -- 172.21.15.39:0/2659316190 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f865400baf0 con 0x7f8658107fa0 2024-05-24T11:46:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.819+0000 7f864e7fc700 1 -- 172.21.15.39:0/2659316190 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540038e0 con 0x7f8658107fa0 2024-05-24T11:46:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.819+0000 7f865fa45700 1 -- 172.21.15.39:0/2659316190 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86581aa7b0 con 0x7f8658107fa0 2024-05-24T11:46:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.822+0000 7f864e7fc700 1 -- 172.21.15.39:0/2659316190 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8654020070 con 0x7f8658107fa0 2024-05-24T11:46:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.823+0000 7f864e7fc700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f86440a1390 0x7f86440a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.823+0000 7f866bc00b80 1 Processor -- start 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.823+0000 7f866bc00b80 1 -- start start 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f864e7fc700 1 -- 172.21.15.39:0/2659316190 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f86540cdc90 con 0x7f8658107fa0 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418bed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418bf270 0x55d5418b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418b9480 0x55d5418b98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f866bc00b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5418100f0 con 0x55d5418be950 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f866bc00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54180dab0 con 0x55d5418bf270 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f866bc00b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5418093a0 con 0x55d5418b9480 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f865dfe2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418b9480 0x55d5418b98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f865dfe2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418b9480 0x55d5418b98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:52164/0 (socket says 172.21.15.39:52164) 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f865dfe2700 1 -- 172.21.15.39:0/1751062556 learned_addr learned my addr 172.21.15.39:0/1751062556 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f865d7e1700 1 --2- 172.21.15.39:0/1751062556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f865dfe2700 1 -- 172.21.15.39:0/1751062556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418bf270 msgr2=0x55d5418b8f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:46:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f865dfe2700 1 --2- 172.21.15.39:0/1751062556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418bf270 0x55d5418b8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f865dfe2700 1 -- 172.21.15.39:0/1751062556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 msgr2=0x55d5418bed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f865dfe2700 1 --2- 172.21.15.39:0/1751062556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418bed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.824+0000 7f865dfe2700 1 -- 172.21.15.39:0/1751062556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d54168a350 con 0x55d5418b9480 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.825+0000 7f865dfe2700 1 --2- 172.21.15.39:0/1751062556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418b9480 0x55d5418b98a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f865001a440 tx=0x7f865001a9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78fbcd8dd2fad964 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.825+0000 7f86437fe700 1 -- 172.21.15.39:0/1751062556 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865001f070 con 0x55d5418b9480 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.825+0000 7f86437fe700 1 -- 172.21.15.39:0/1751062556 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8650020040 con 0x55d5418b9480 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.825+0000 7f86437fe700 1 -- 172.21.15.39:0/1751062556 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650026bb0 con 0x55d5418b9480 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f865cfe0700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f86440a1390 0x7f86440a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f866bc00b80 1 -- 172.21.15.39:0/1751062556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418b9480 msgr2=0x55d5418b98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f866bc00b80 1 --2- 172.21.15.39:0/1751062556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418b9480 0x55d5418b98a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f865001a440 tx=0x7f865001a9f0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f866bc00b80 1 -- 172.21.15.39:0/1751062556 shutdown_connections 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f866bc00b80 1 --2- 172.21.15.39:0/1751062556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418b9480 0x55d5418b98a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f866bc00b80 1 --2- 172.21.15.39:0/1751062556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418bed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f866bc00b80 1 --2- 172.21.15.39:0/1751062556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418bf270 0x55d5418b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f866bc00b80 1 -- 172.21.15.39:0/1751062556 >> 172.21.15.39:0/1751062556 conn(0x55d5418171b0 msgr2=0x55d5418b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f866bc00b80 1 -- 172.21.15.39:0/1751062556 shutdown_connections 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f866bc00b80 1 -- 172.21.15.39:0/1751062556 wait complete. 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.826+0000 7f865cfe0700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f86440a1390 0x7f86440a3850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55d5418b4790 tx=0x7f8648009190 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.827+0000 7f866bc00b80 1 -- start start 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.827+0000 7f866bc00b80 1 Processor -- start 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.827+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418b9480 0x55d5418c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.827+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418be950 0x55d5418c2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.827+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.827+0000 7f866bc00b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5417a5500 con 0x55d5418b9480 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.827+0000 7f866bc00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5417a51a0 con 0x55d5418be950 2024-05-24T11:46:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.827+0000 7f866bc00b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54168c640 con 0x55d5418bf270 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.827+0000 7f866bc00b80 10 client.0 _ll_register_callbacks cb 0x55d5417aead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865cfe0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418be950 0x55d5418c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865d7e1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418b9480 0x55d5418c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865cfe0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418be950 0x55d5418c2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:42464/0 (socket says 172.21.15.39:42464) 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865dfe2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865d7e1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418b9480 0x55d5418c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51980/0 (socket says 172.21.15.39:51980) 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865cfe0700 1 -- 172.21.15.39:0/642974731 learned_addr learned my addr 172.21.15.39:0/642974731 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865dfe2700 1 -- 172.21.15.39:0/642974731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418be950 msgr2=0x55d5418c2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865dfe2700 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418be950 0x55d5418c2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865dfe2700 1 -- 172.21.15.39:0/642974731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418b9480 msgr2=0x55d5418c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865dfe2700 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418b9480 0x55d5418c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865dfe2700 1 -- 172.21.15.39:0/642974731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865001a0f0 con 0x55d5418bf270 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865d7e1700 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418b9480 0x55d5418c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f865dfe2700 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c2bb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8650000f80 tx=0x7f8650027f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba6356c2b657912 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f8640ff9700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.828+0000 7f8640ff9700 1 -- 172.21.15.39:0/642974731 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865001f040 con 0x55d5418bf270 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f8640ff9700 1 -- 172.21.15.39:0/642974731 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8650003c00 con 0x55d5418bf270 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f866bc00b80 1 -- 172.21.15.39:0/642974731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d541762210 con 0x55d5418bf270 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f8640ff9700 1 -- 172.21.15.39:0/642974731 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86500265f0 con 0x55d5418bf270 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f866bc00b80 10 client.25633 fetch_fsmap learned FSMap version 29 2024-05-24T11:46:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f866bc00b80 1 -- 172.21.15.39:0/642974731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d541636b00 con 0x55d5418bf270 2024-05-24T11:46:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f8640ff9700 1 -- 172.21.15.39:0/642974731 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d541762210 con 0x55d5418bf270 2024-05-24T11:46:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f8640ff9700 1 -- 172.21.15.39:0/642974731 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f865002d050 con 0x55d5418bf270 2024-05-24T11:46:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f866bc00b80 10 client.25633 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:46:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f866bc00b80 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:46:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.829+0000 7f866bc00b80 20 client.25633 populate_metadata read hostname 'smithi039' 2024-05-24T11:46:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.830+0000 7f866bc00b80 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d5418d2a40 0x55d5418d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.830+0000 7f866bc00b80 4 client.25633 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.830+0000 7f866bc00b80 1 -- 172.21.15.39:0/642974731 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5418887b0 con 0x55d5418d2a40 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.830+0000 7f865d7e1700 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d5418d2a40 0x55d5418d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.831+0000 7f8640ff9700 10 client.25633 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.831+0000 7f865d7e1700 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d5418d2a40 0x55d5418d4e60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a2dbc72b8a81138 server_cookie=9b76a63ca75fafa5 in_seq=0 out_seq=0 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.831+0000 7f8640ff9700 1 -- 172.21.15.39:0/642974731 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5418887b0 con 0x55d5418d2a40 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.831+0000 7f8640ff9700 10 client.25633 handle_command_reply: tid=0 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.831+0000 7f866bc00b80 1 client.25633 shutdown 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 -- 172.21.15.39:0/642974731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 msgr2=0x55d5418c2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c2bb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8650000f80 tx=0x7f8650027f50 comp rx=0 tx=0).stop 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 -- 172.21.15.39:0/642974731 shutdown_connections 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c2bb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d5418d2a40 0x55d5418d4e60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418b9480 0x55d5418c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 --2- 172.21.15.39:0/642974731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418be950 0x55d5418c2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 -- 172.21.15.39:0/642974731 >> 172.21.15.39:0/642974731 conn(0x55d5418171b0 msgr2=0x55d5418bd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f8640ff9700 0 client.25633 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 -- 172.21.15.39:0/642974731 shutdown_connections 2024-05-24T11:46:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 1 -- 172.21.15.39:0/642974731 wait complete. 2024-05-24T11:46:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.832+0000 7f866bc00b80 20 client.25633 trim_cache size 0 max 16384 2024-05-24T11:46:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.860+0000 7f866bc00b80 1 Processor -- start 2024-05-24T11:46:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.860+0000 7f866bc00b80 1 -- start start 2024-05-24T11:46:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 0x55d5418b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418be950 0x55d5418b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418bf270 0x55d5418b9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f866bc00b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5418b9da0 con 0x55d5418bf270 2024-05-24T11:46:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f866bc00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5418b5670 con 0x55d5418b9480 2024-05-24T11:46:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f866bc00b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5418b4930 con 0x55d5418be950 2024-05-24T11:46:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865cfe0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418be950 0x55d5418b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865d7e1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 0x55d5418b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865cfe0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418be950 0x55d5418b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:52178/0 (socket says 172.21.15.39:52178) 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865d7e1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 0x55d5418b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:42470/0 (socket says 172.21.15.39:42470) 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865cfe0700 1 -- 172.21.15.39:0/2629521999 learned_addr learned my addr 172.21.15.39:0/2629521999 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865dfe2700 1 --2- 172.21.15.39:0/2629521999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418bf270 0x55d5418b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865cfe0700 1 -- 172.21.15.39:0/2629521999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 msgr2=0x55d5418b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865cfe0700 1 --2- 172.21.15.39:0/2629521999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 0x55d5418b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865cfe0700 1 -- 172.21.15.39:0/2629521999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418bf270 msgr2=0x55d5418b9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865cfe0700 1 --2- 172.21.15.39:0/2629521999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418bf270 0x55d5418b9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.861+0000 7f865cfe0700 1 -- 172.21.15.39:0/2629521999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5417cf500 con 0x55d5418be950 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.862+0000 7f865cfe0700 1 --2- 172.21.15.39:0/2629521999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418be950 0x55d5418b3910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f864800e8d0 tx=0x7f8648011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15b5937a312875e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.862+0000 7f8643fff700 1 -- 172.21.15.39:0/2629521999 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864800ee50 con 0x55d5418be950 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.862+0000 7f8643fff700 1 -- 172.21.15.39:0/2629521999 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8648015b20 con 0x55d5418be950 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.862+0000 7f8643fff700 1 -- 172.21.15.39:0/2629521999 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480138a0 con 0x55d5418be950 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 -- 172.21.15.39:0/2629521999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418be950 msgr2=0x55d5418b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 --2- 172.21.15.39:0/2629521999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418be950 0x55d5418b3910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f864800e8d0 tx=0x7f8648011000 comp rx=0 tx=0).stop 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 -- 172.21.15.39:0/2629521999 shutdown_connections 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 --2- 172.21.15.39:0/2629521999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418bf270 0x55d5418b9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 --2- 172.21.15.39:0/2629521999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 0x55d5418b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 --2- 172.21.15.39:0/2629521999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418be950 0x55d5418b3910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 -- 172.21.15.39:0/2629521999 >> 172.21.15.39:0/2629521999 conn(0x55d5418171b0 msgr2=0x55d5418b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 -- 172.21.15.39:0/2629521999 shutdown_connections 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 -- 172.21.15.39:0/2629521999 wait complete. 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 -- start start 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.863+0000 7f866bc00b80 1 Processor -- start 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 0x55d5418c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f866bc00b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f866bc00b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5418c83d0 con 0x55d5418be950 2024-05-24T11:46:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f866bc00b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5418c8540 con 0x55d5418b9480 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f866bc00b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5418c8700 con 0x55d5418bf270 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f866bc00b80 10 client.0 _ll_register_callbacks cb 0x55d5417aead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f865cfe0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f865dfe2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f865dfe2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:52184/0 (socket says 172.21.15.39:52184) 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f865dfe2700 1 -- 172.21.15.39:0/3735957787 learned_addr learned my addr 172.21.15.39:0/3735957787 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f865dfe2700 1 -- 172.21.15.39:0/3735957787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 msgr2=0x55d5418c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f865dfe2700 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 0x55d5418c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f865dfe2700 1 -- 172.21.15.39:0/3735957787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 msgr2=0x55d5418ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.864+0000 7f865cfe0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52002/0 (socket says 172.21.15.39:52002) 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f865dfe2700 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f865dfe2700 1 -- 172.21.15.39:0/3735957787 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864801a040 con 0x55d5418bf270 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f865cfe0700 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f865dfe2700 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c7c70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8650025040 tx=0x7f8650030f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460b59560e00234e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f8640ff9700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f8640ff9700 1 -- 172.21.15.39:0/3735957787 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865002edd0 con 0x55d5418bf270 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f8640ff9700 1 -- 172.21.15.39:0/3735957787 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8650030650 con 0x55d5418bf270 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f866bc00b80 1 -- 172.21.15.39:0/3735957787 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8650020040 con 0x55d5418bf270 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f8640ff9700 1 -- 172.21.15.39:0/3735957787 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f865001a9f0 con 0x55d5418bf270 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f866bc00b80 10 client.25639 fetch_fsmap learned FSMap version 29 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.865+0000 7f866bc00b80 1 -- 172.21.15.39:0/3735957787 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d54168ec70 con 0x55d5418bf270 2024-05-24T11:46:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.866+0000 7f8640ff9700 1 -- 172.21.15.39:0/3735957787 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f865004a020 con 0x55d5418bf270 2024-05-24T11:46:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.866+0000 7f8640ff9700 1 -- 172.21.15.39:0/3735957787 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8650018960 con 0x55d5418bf270 2024-05-24T11:46:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.867+0000 7f866bc00b80 10 client.25639 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:46:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.867+0000 7f866bc00b80 10 client.25639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:46:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.867+0000 7f866bc00b80 20 client.25639 populate_metadata read hostname 'smithi039' 2024-05-24T11:46:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.867+0000 7f866bc00b80 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d54196cdf0 0x55d54196f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:46:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.867+0000 7f866bc00b80 4 client.25639 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:46:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.867+0000 7f866bc00b80 1 -- 172.21.15.39:0/3735957787 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d541801e60 con 0x55d54196cdf0 2024-05-24T11:46:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.867+0000 7f865d7e1700 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d54196cdf0 0x55d54196f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:46:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.867+0000 7f8640ff9700 10 client.25639 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:48.867+0000 7f865d7e1700 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d54196cdf0 0x55d54196f210 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8843e40fa156324c server_cookie=7cd9d4ea8eed953e in_seq=0 out_seq=0 2024-05-24T11:46:49.118 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 445.0 s, 1088.2 tps, lat 92.029 ms stddev 26.097 2024-05-24T11:46:50.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:49 smithi039 ceph-mon[42919]: pgmap v266: 97 pgs: 97 active+clean; 5.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-05-24T11:46:50.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:49 smithi053 ceph-mon[47448]: pgmap v266: 97 pgs: 97 active+clean; 5.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-05-24T11:46:50.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:49 smithi137 ceph-mon[47907]: pgmap v266: 97 pgs: 97 active+clean; 5.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 21 MiB/s wr, 1.72k op/s 2024-05-24T11:46:52.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:51 smithi039 ceph-mon[42919]: pgmap v267: 97 pgs: 97 active+clean; 5.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-05-24T11:46:52.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:51 smithi053 ceph-mon[47448]: pgmap v267: 97 pgs: 97 active+clean; 5.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-05-24T11:46:52.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:51 smithi137 ceph-mon[47907]: pgmap v267: 97 pgs: 97 active+clean; 5.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-05-24T11:46:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.291+0000 7f8640ff9700 1 -- 172.21.15.39:0/3735957787 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d541801e60 con 0x55d54196cdf0 2024-05-24T11:46:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.291+0000 7f8640ff9700 10 client.25639 handle_command_reply: tid=0 2024-05-24T11:46:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.291+0000 7f866bc00b80 1 client.25639 shutdown 2024-05-24T11:46:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 -- 172.21.15.39:0/3735957787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 msgr2=0x55d5418c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c7c70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8650025040 tx=0x7f8650030f70 comp rx=0 tx=0).stop 2024-05-24T11:46:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 -- 172.21.15.39:0/3735957787 shutdown_connections 2024-05-24T11:46:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5418bf270 0x55d5418c7c70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d54196cdf0 0x55d54196f210 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d5418b9480 0x55d5418c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 --2- 172.21.15.39:0/3735957787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d5418be950 0x55d5418ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f8640ff9700 0 client.25639 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:46:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 -- 172.21.15.39:0/3735957787 >> 172.21.15.39:0/3735957787 conn(0x55d5418171b0 msgr2=0x55d5418b8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 -- 172.21.15.39:0/3735957787 shutdown_connections 2024-05-24T11:46:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.292+0000 7f866bc00b80 1 -- 172.21.15.39:0/3735957787 wait complete. 2024-05-24T11:46:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.293+0000 7f866bc00b80 20 client.25639 trim_cache size 0 max 16384 2024-05-24T11:46:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:46:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:46:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:46:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 -- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f86440a1390 msgr2=0x7f86440a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f86440a1390 0x7f86440a3850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55d5418b4790 tx=0x7f8648009190 comp rx=0 tx=0).stop 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 -- 172.21.15.39:0/2659316190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8658107fa0 msgr2=0x7f86581a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8658107fa0 0x7f86581a37c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8654009230 tx=0x7f865400fa00 comp rx=0 tx=0).stop 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 -- 172.21.15.39:0/2659316190 shutdown_connections 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8658107fa0 0x7f86581a37c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f86440a1390 0x7f86440a3850 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865810f630 0x7f865819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 --2- 172.21.15.39:0/2659316190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f865810d440 0x7f86581a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 -- 172.21.15.39:0/2659316190 >> 172.21.15.39:0/2659316190 conn(0x7f865806bf80 msgr2=0x7f8658109950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 -- 172.21.15.39:0/2659316190 shutdown_connections 2024-05-24T11:46:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:46:52.294+0000 7f865fa45700 1 -- 172.21.15.39:0/2659316190 wait complete. 2024-05-24T11:46:54.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:53 smithi039 ceph-mon[42919]: pgmap v268: 97 pgs: 97 active+clean; 5.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-05-24T11:46:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 450.0 s, 1049.2 tps, lat 95.044 ms stddev 87.406 2024-05-24T11:46:54.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:53 smithi053 ceph-mon[47448]: pgmap v268: 97 pgs: 97 active+clean; 5.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-05-24T11:46:54.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:53 smithi137 ceph-mon[47907]: pgmap v268: 97 pgs: 97 active+clean; 5.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-05-24T11:46:56.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:55 smithi039 ceph-mon[42919]: pgmap v269: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 21 MiB/s wr, 1.77k op/s 2024-05-24T11:46:56.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:55 smithi053 ceph-mon[47448]: pgmap v269: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 21 MiB/s wr, 1.77k op/s 2024-05-24T11:46:56.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:55 smithi137 ceph-mon[47907]: pgmap v269: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 21 MiB/s wr, 1.77k op/s 2024-05-24T11:46:58.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:57 smithi039 ceph-mon[42919]: pgmap v270: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:46:58.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:57 smithi053 ceph-mon[47448]: pgmap v270: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:46:58.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:57 smithi137 ceph-mon[47907]: pgmap v270: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:46:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 455.0 s, 1020.2 tps, lat 98.099 ms stddev 45.704 2024-05-24T11:47:00.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:46:59 smithi039 ceph-mon[42919]: pgmap v271: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-05-24T11:47:00.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:46:59 smithi053 ceph-mon[47448]: pgmap v271: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-05-24T11:47:00.249 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:46:59 smithi137 ceph-mon[47907]: pgmap v271: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-05-24T11:47:02.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:01 smithi039 ceph-mon[42919]: pgmap v272: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:47:02.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:01 smithi053 ceph-mon[47448]: pgmap v272: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:47:02.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:01 smithi137 ceph-mon[47907]: pgmap v272: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:47:04.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:03 smithi039 ceph-mon[42919]: pgmap v273: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:47:04.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 460.0 s, 1068.7 tps, lat 93.695 ms stddev 34.766 2024-05-24T11:47:04.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:03 smithi053 ceph-mon[47448]: pgmap v273: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:47:04.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:03 smithi137 ceph-mon[47907]: pgmap v273: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:47:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.475+0000 7f2682677700 1 -- 172.21.15.39:0/1164676352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c1009c0 msgr2=0x7f267c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.475+0000 7f2682677700 1 --2- 172.21.15.39:0/1164676352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c1009c0 0x7f267c104a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2670009990 tx=0x7f2670009ca0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.476+0000 7f2682677700 1 -- 172.21.15.39:0/1164676352 shutdown_connections 2024-05-24T11:47:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.476+0000 7f2682677700 1 --2- 172.21.15.39:0/1164676352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f267c105130 0x7f267c069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.476+0000 7f2682677700 1 --2- 172.21.15.39:0/1164676352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f267c100010 0x7f267c1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.476+0000 7f2682677700 1 --2- 172.21.15.39:0/1164676352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c1009c0 0x7f267c104a00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.476+0000 7f2682677700 1 -- 172.21.15.39:0/1164676352 >> 172.21.15.39:0/1164676352 conn(0x7f267c076f40 msgr2=0x7f267c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.477+0000 7f2682677700 1 -- 172.21.15.39:0/1164676352 shutdown_connections 2024-05-24T11:47:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.477+0000 7f2682677700 1 -- 172.21.15.39:0/1164676352 wait complete. 2024-05-24T11:47:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.477+0000 7f2682677700 1 Processor -- start 2024-05-24T11:47:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.477+0000 7f2682677700 1 -- start start 2024-05-24T11:47:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f2682677700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f267c100010 0x7f267c19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f2682677700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f267c105130 0x7f267c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f2682677700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c10db90 0x7f267c10dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f2682677700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c10e750 con 0x7f267c105130 2024-05-24T11:47:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f2682677700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c10e8c0 con 0x7f267c10db90 2024-05-24T11:47:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f2682677700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c199b20 con 0x7f267c100010 2024-05-24T11:47:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f2680c14700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c10db90 0x7f267c10dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f267bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f267c100010 0x7f267c19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f2680c14700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c10db90 0x7f267c10dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:56744/0 (socket says 172.21.15.39:56744) 2024-05-24T11:47:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.478+0000 7f2680c14700 1 -- 172.21.15.39:0/3341345 learned_addr learned my addr 172.21.15.39:0/3341345 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f2680c14700 1 -- 172.21.15.39:0/3341345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f267c100010 msgr2=0x7f267c19e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f2680c14700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f267c100010 0x7f267c19e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f2680c14700 1 -- 172.21.15.39:0/3341345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f267c105130 msgr2=0x7f267c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f2680c14700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f267c105130 0x7f267c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f2680c14700 1 -- 172.21.15.39:0/3341345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700095f0 con 0x7f267c10db90 2024-05-24T11:47:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f2680c14700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c10db90 0x7f267c10dff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f266c00b4a0 tx=0x7f266c00b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f26797fa700 1 -- 172.21.15.39:0/3341345 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c011830 con 0x7f267c10db90 2024-05-24T11:47:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f2682677700 1 -- 172.21.15.39:0/3341345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c199e00 con 0x7f267c10db90 2024-05-24T11:47:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f26797fa700 1 -- 172.21.15.39:0/3341345 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f266c011e20 con 0x7f267c10db90 2024-05-24T11:47:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.479+0000 7f2682677700 1 -- 172.21.15.39:0/3341345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c19a360 con 0x7f267c10db90 2024-05-24T11:47:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.482+0000 7f26797fa700 1 -- 172.21.15.39:0/3341345 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c010d40 con 0x7f267c10db90 2024-05-24T11:47:05.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.484+0000 7f268e832b80 1 Processor -- start 2024-05-24T11:47:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.485+0000 7f26797fa700 1 -- 172.21.15.39:0/3341345 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f266c011990 con 0x7f267c10db90 2024-05-24T11:47:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.486+0000 7f26797fa700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f26640a1290 0x7f26640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.486+0000 7f26797fa700 1 -- 172.21.15.39:0/3341345 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f266c0cc330 con 0x7f267c10db90 2024-05-24T11:47:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.486+0000 7f267b7fe700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f26640a1290 0x7f26640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.486+0000 7f268e832b80 1 -- start start 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f268e832b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971abd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f268e832b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 0x5565971a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f268e832b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f268e832b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565970fd0f0 con 0x5565971ac270 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f268e832b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565970faab0 con 0x5565971a6480 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f268e832b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565970f63a0 con 0x5565971ab950 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:37966/0 (socket says 172.21.15.39:37966) 2024-05-24T11:47:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267bfff700 1 -- 172.21.15.39:0/2288237449 learned_addr learned my addr 172.21.15.39:0/2288237449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267b7fe700 1 --2- 172.21.15.39:0/2288237449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 0x5565971a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f2680c14700 1 --2- 172.21.15.39:0/2288237449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267bfff700 1 -- 172.21.15.39:0/2288237449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 msgr2=0x5565971a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267bfff700 1 --2- 172.21.15.39:0/2288237449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267bfff700 1 -- 172.21.15.39:0/2288237449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 msgr2=0x5565971a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267bfff700 1 --2- 172.21.15.39:0/2288237449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 0x5565971a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267bfff700 1 -- 172.21.15.39:0/2288237449 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556596f77350 con 0x5565971ab950 2024-05-24T11:47:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.487+0000 7f267b7fe700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f26640a1290 0x7f26640a3750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f267000b2e0 tx=0x7f2670005ea0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.488+0000 7f267bfff700 1 --2- 172.21.15.39:0/2288237449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971abd30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2674011650 tx=0x7f2674009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f148df078935184d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.488+0000 7f26627fc700 1 -- 172.21.15.39:0/2288237449 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400e040 con 0x5565971ab950 2024-05-24T11:47:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.489+0000 7f26627fc700 1 -- 172.21.15.39:0/2288237449 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f267400f040 con 0x5565971ab950 2024-05-24T11:47:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.489+0000 7f26627fc700 1 -- 172.21.15.39:0/2288237449 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2674015810 con 0x5565971ab950 2024-05-24T11:47:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 -- 172.21.15.39:0/2288237449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 msgr2=0x5565971abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 --2- 172.21.15.39:0/2288237449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971abd30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2674011650 tx=0x7f2674009920 comp rx=0 tx=0).stop 2024-05-24T11:47:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 -- 172.21.15.39:0/2288237449 shutdown_connections 2024-05-24T11:47:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 --2- 172.21.15.39:0/2288237449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971abd30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 --2- 172.21.15.39:0/2288237449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 --2- 172.21.15.39:0/2288237449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 0x5565971a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 -- 172.21.15.39:0/2288237449 >> 172.21.15.39:0/2288237449 conn(0x5565971041b0 msgr2=0x5565971a0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 -- 172.21.15.39:0/2288237449 shutdown_connections 2024-05-24T11:47:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 -- 172.21.15.39:0/2288237449 wait complete. 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.490+0000 7f268e832b80 1 -- start start 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f268e832b80 1 Processor -- start 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f268e832b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971a6480 0x5565971b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f268e832b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f268e832b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f268e832b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556597092500 con 0x5565971a6480 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f268e832b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565970921a0 con 0x5565971ac270 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f268e832b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556596f79640 con 0x5565971ab950 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f268e832b80 10 client.0 _ll_register_callbacks cb 0x55659709bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f2680c14700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f2680c14700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:56748/0 (socket says 172.21.15.39:56748) 2024-05-24T11:47:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.491+0000 7f2680c14700 1 -- 172.21.15.39:0/809073023 learned_addr learned my addr 172.21.15.39:0/809073023 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f267bfff700 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971a6480 0x5565971b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f267b7fe700 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f2680c14700 1 -- 172.21.15.39:0/809073023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 msgr2=0x5565971af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f2680c14700 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f2680c14700 1 -- 172.21.15.39:0/809073023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971a6480 msgr2=0x5565971b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f2680c14700 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971a6480 0x5565971b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f2680c14700 1 -- 172.21.15.39:0/809073023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2674011330 con 0x5565971ac270 2024-05-24T11:47:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f2680c14700 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971afae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f266c0c9c50 tx=0x7f266c010480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c06e575416989028 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f265b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:47:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f265b7fe700 1 -- 172.21.15.39:0/809073023 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c09d170 con 0x5565971ac270 2024-05-24T11:47:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f265b7fe700 1 -- 172.21.15.39:0/809073023 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f266c099c60 con 0x5565971ac270 2024-05-24T11:47:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.492+0000 7f268e832b80 1 -- 172.21.15.39:0/809073023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55659704f210 con 0x5565971ac270 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f265b7fe700 1 -- 172.21.15.39:0/809073023 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f266c099560 con 0x5565971ac270 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f268e832b80 10 client.25736 fetch_fsmap learned FSMap version 29 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f268e832b80 1 -- 172.21.15.39:0/809073023 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556596f23b00 con 0x5565971ac270 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f265b7fe700 1 -- 172.21.15.39:0/809073023 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f266c063020 con 0x5565971ac270 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f265b7fe700 1 -- 172.21.15.39:0/809073023 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f266c098a50 con 0x5565971ac270 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f268e832b80 10 client.25736 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f268e832b80 10 client.25736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f268e832b80 20 client.25736 populate_metadata read hostname 'smithi039' 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f268e832b80 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5565971bfba0 0x5565971c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f268e832b80 4 client.25736 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:47:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.493+0000 7f268e832b80 1 -- 172.21.15.39:0/809073023 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565971757b0 con 0x5565971bfba0 2024-05-24T11:47:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.497+0000 7f267bfff700 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5565971bfba0 0x5565971c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.497+0000 7f265b7fe700 10 client.25736 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.497+0000 7f267bfff700 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5565971bfba0 0x5565971c1f90 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53db15195923d16 server_cookie=13d2170a3c855e85 in_seq=0 out_seq=0 2024-05-24T11:47:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f265b7fe700 1 -- 172.21.15.39:0/809073023 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565971757b0 con 0x5565971bfba0 2024-05-24T11:47:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f265b7fe700 10 client.25736 handle_command_reply: tid=0 2024-05-24T11:47:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f268e832b80 1 client.25736 shutdown 2024-05-24T11:47:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f268e832b80 1 -- 172.21.15.39:0/809073023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 msgr2=0x5565971afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f268e832b80 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971afae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f266c0c9c50 tx=0x7f266c010480 comp rx=0 tx=0).stop 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f268e832b80 1 -- 172.21.15.39:0/809073023 shutdown_connections 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f268e832b80 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5565971bfba0 0x5565971c1f90 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f268e832b80 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971afae0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f265b7fe700 0 client.25736 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f268e832b80 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971a6480 0x5565971b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f268e832b80 1 --2- 172.21.15.39:0/809073023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.498+0000 7f268e832b80 1 -- 172.21.15.39:0/809073023 >> 172.21.15.39:0/809073023 conn(0x5565971041b0 msgr2=0x5565971a4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.499+0000 7f268e832b80 1 -- 172.21.15.39:0/809073023 shutdown_connections 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.499+0000 7f268e832b80 1 -- 172.21.15.39:0/809073023 wait complete. 2024-05-24T11:47:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.499+0000 7f268e832b80 20 client.25736 trim_cache size 0 max 16384 2024-05-24T11:47:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.523+0000 7f268e832b80 1 Processor -- start 2024-05-24T11:47:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.523+0000 7f268e832b80 1 -- start start 2024-05-24T11:47:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f268e832b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f268e832b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f268e832b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 0x5565971b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f268e832b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565971c71a0 con 0x5565971ac270 2024-05-24T11:47:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f268e832b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565971ada20 con 0x5565971a6480 2024-05-24T11:47:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f268e832b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565971aa130 con 0x5565971ab950 2024-05-24T11:47:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f267bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f267b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f267bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:56750/0 (socket says 172.21.15.39:56750) 2024-05-24T11:47:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f267bfff700 1 -- 172.21.15.39:0/1384590134 learned_addr learned my addr 172.21.15.39:0/1384590134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.524+0000 7f2680c14700 1 --2- 172.21.15.39:0/1384590134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 0x5565971b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f267b7fe700 1 -- 172.21.15.39:0/1384590134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 msgr2=0x5565971a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f267b7fe700 1 --2- 172.21.15.39:0/1384590134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f267b7fe700 1 -- 172.21.15.39:0/1384590134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 msgr2=0x5565971b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f267b7fe700 1 --2- 172.21.15.39:0/1384590134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 0x5565971b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f267b7fe700 1 -- 172.21.15.39:0/1384590134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565970bc500 con 0x5565971ab950 2024-05-24T11:47:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f267bfff700 1 --2- 172.21.15.39:0/1384590134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:47:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f267b7fe700 1 --2- 172.21.15.39:0/1384590134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971b1ce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2670006010 tx=0x7f2670025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=358d40a1c2e84f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f2662ffd700 1 -- 172.21.15.39:0/1384590134 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267001d070 con 0x5565971ab950 2024-05-24T11:47:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f2662ffd700 1 -- 172.21.15.39:0/1384590134 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26700253f0 con 0x5565971ab950 2024-05-24T11:47:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.525+0000 7f2662ffd700 1 -- 172.21.15.39:0/1384590134 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267002f650 con 0x5565971ab950 2024-05-24T11:47:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.526+0000 7f268e832b80 1 -- 172.21.15.39:0/1384590134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 msgr2=0x5565971b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.526+0000 7f268e832b80 1 --2- 172.21.15.39:0/1384590134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971b1ce0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2670006010 tx=0x7f2670025000 comp rx=0 tx=0).stop 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.526+0000 7f268e832b80 1 -- 172.21.15.39:0/1384590134 shutdown_connections 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.526+0000 7f268e832b80 1 --2- 172.21.15.39:0/1384590134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ac270 0x5565971b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.526+0000 7f268e832b80 1 --2- 172.21.15.39:0/1384590134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971a6480 0x5565971a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.526+0000 7f268e832b80 1 --2- 172.21.15.39:0/1384590134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971ab950 0x5565971b1ce0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.526+0000 7f268e832b80 1 -- 172.21.15.39:0/1384590134 >> 172.21.15.39:0/1384590134 conn(0x5565971041b0 msgr2=0x5565971af490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.526+0000 7f268e832b80 1 -- 172.21.15.39:0/1384590134 shutdown_connections 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.527+0000 7f268e832b80 1 -- 172.21.15.39:0/1384590134 wait complete. 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.527+0000 7f268e832b80 1 -- start start 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.527+0000 7f268e832b80 1 Processor -- start 2024-05-24T11:47:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.527+0000 7f268e832b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971a6480 0x5565971b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.527+0000 7f268e832b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ab950 0x5565971b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.527+0000 7f268e832b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.527+0000 7f268e832b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565971b5cf0 con 0x5565971ab950 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.527+0000 7f268e832b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565971b5e60 con 0x5565971ac270 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.527+0000 7f268e832b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565971b6020 con 0x5565971a6480 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f268e832b80 10 client.0 _ll_register_callbacks cb 0x55659709bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971a6480 0x5565971b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971a6480 0x5565971b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38002/0 (socket says 172.21.15.39:38002) 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267bfff700 1 -- 172.21.15.39:0/1569079805 learned_addr learned my addr 172.21.15.39:0/1569079805 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f2680c14700 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267bfff700 1 -- 172.21.15.39:0/1569079805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 msgr2=0x5565971b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267bfff700 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267b7fe700 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ab950 0x5565971b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267bfff700 1 -- 172.21.15.39:0/1569079805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ab950 msgr2=0x5565971b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267bfff700 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ab950 0x5565971b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267bfff700 1 -- 172.21.15.39:0/1569079805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556596f77350 con 0x5565971a6480 2024-05-24T11:47:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f2680c14700 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:47:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267b7fe700 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ab950 0x5565971b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:47:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f267bfff700 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971a6480 0x5565971b6d70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f267400b010 tx=0x7f2674009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ef8836a3f15ee1e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.528+0000 7f265b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:47:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.530+0000 7f265b7fe700 1 -- 172.21.15.39:0/1569079805 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400e050 con 0x5565971a6480 2024-05-24T11:47:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.530+0000 7f265b7fe700 1 -- 172.21.15.39:0/1569079805 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2674024070 con 0x5565971a6480 2024-05-24T11:47:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.530+0000 7f268e832b80 1 -- 172.21.15.39:0/1569079805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f267400f040 con 0x5565971a6480 2024-05-24T11:47:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.530+0000 7f265b7fe700 1 -- 172.21.15.39:0/1569079805 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2674026b70 con 0x5565971a6480 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f268e832b80 10 client.25654 fetch_fsmap learned FSMap version 29 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f268e832b80 1 -- 172.21.15.39:0/1569079805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f267c0699f0 con 0x5565971a6480 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f265b7fe700 1 -- 172.21.15.39:0/1569079805 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f267400f040 con 0x5565971a6480 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f265b7fe700 1 -- 172.21.15.39:0/1569079805 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2674007620 con 0x5565971a6480 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f268e832b80 10 client.25654 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f268e832b80 10 client.25654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f268e832b80 20 client.25654 populate_metadata read hostname 'smithi039' 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f268e832b80 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556597217590 0x5565972199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f268e832b80 4 client.25654 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:47:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.531+0000 7f268e832b80 1 -- 172.21.15.39:0/1569079805 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565970eee60 con 0x556597217590 2024-05-24T11:47:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.534+0000 7f267b7fe700 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556597217590 0x5565972199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.535+0000 7f267b7fe700 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556597217590 0x5565972199b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92516e21f71a224 server_cookie=ded2dfa45169481a in_seq=0 out_seq=0 2024-05-24T11:47:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.535+0000 7f265b7fe700 10 client.25654 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f265b7fe700 1 -- 172.21.15.39:0/1569079805 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565970eee60 con 0x556597217590 2024-05-24T11:47:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f265b7fe700 10 client.25654 handle_command_reply: tid=0 2024-05-24T11:47:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 client.25654 shutdown 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 -- 172.21.15.39:0/1569079805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971a6480 msgr2=0x5565971b6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971a6480 0x5565971b6d70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f267400b010 tx=0x7f2674009950 comp rx=0 tx=0).stop 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 -- 172.21.15.39:0/1569079805 shutdown_connections 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565971a6480 0x5565971b6d70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556597217590 0x5565972199b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5565971ac270 0x5565971b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 --2- 172.21.15.39:0/1569079805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5565971ab950 0x5565971b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 -- 172.21.15.39:0/1569079805 >> 172.21.15.39:0/1569079805 conn(0x5565971041b0 msgr2=0x5565971a93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f265b7fe700 0 client.25654 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.568+0000 7f268e832b80 1 -- 172.21.15.39:0/1569079805 shutdown_connections 2024-05-24T11:47:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.569+0000 7f268e832b80 1 -- 172.21.15.39:0/1569079805 wait complete. 2024-05-24T11:47:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.569+0000 7f268e832b80 20 client.25654 trim_cache size 0 max 16384 2024-05-24T11:47:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:47:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:47:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:47:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:47:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 -- 172.21.15.39:0/3341345 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f26640a1290 msgr2=0x7f26640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f26640a1290 0x7f26640a3750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f267000b2e0 tx=0x7f2670005ea0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 -- 172.21.15.39:0/3341345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c10db90 msgr2=0x7f267c10dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c10db90 0x7f267c10dff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f266c00b4a0 tx=0x7f266c00b7b0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 -- 172.21.15.39:0/3341345 shutdown_connections 2024-05-24T11:47:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f267c100010 0x7f267c19e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f26640a1290 0x7f26640a3750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c10db90 0x7f267c10dff0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 --2- 172.21.15.39:0/3341345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f267c105130 0x7f267c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 -- 172.21.15.39:0/3341345 >> 172.21.15.39:0/3341345 conn(0x7f267c076f40 msgr2=0x7f267c106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 -- 172.21.15.39:0/3341345 shutdown_connections 2024-05-24T11:47:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:05.570+0000 7f2682677700 1 -- 172.21.15.39:0/3341345 wait complete. 2024-05-24T11:47:06.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:05 smithi039 ceph-mon[42919]: pgmap v274: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:47:06.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:05 smithi053 ceph-mon[47448]: pgmap v274: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:47:06.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:05 smithi137 ceph-mon[47907]: pgmap v274: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:47:08.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:07 smithi053 ceph-mon[47448]: pgmap v275: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:47:08.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:07 smithi137 ceph-mon[47907]: pgmap v275: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:47:08.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:07 smithi039 ceph-mon[42919]: pgmap v275: 97 pgs: 97 active+clean; 5.7 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:47:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 465.0 s, 1023.9 tps, lat 95.374 ms stddev 32.938 2024-05-24T11:47:10.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:09 smithi053 ceph-mon[47448]: pgmap v276: 97 pgs: 97 active+clean; 5.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-05-24T11:47:10.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:09 smithi053 ceph-mon[47448]: osdmap e86: 12 total, 12 up, 12 in 2024-05-24T11:47:10.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:09 smithi137 ceph-mon[47907]: pgmap v276: 97 pgs: 97 active+clean; 5.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-05-24T11:47:10.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:09 smithi137 ceph-mon[47907]: osdmap e86: 12 total, 12 up, 12 in 2024-05-24T11:47:10.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:09 smithi039 ceph-mon[42919]: pgmap v276: 97 pgs: 97 active+clean; 5.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-05-24T11:47:10.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:09 smithi039 ceph-mon[42919]: osdmap e86: 12 total, 12 up, 12 in 2024-05-24T11:47:12.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:11 smithi137 ceph-mon[47907]: pgmap v278: 97 pgs: 97 active+clean; 5.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-05-24T11:47:12.250 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:11 smithi053 ceph-mon[47448]: pgmap v278: 97 pgs: 97 active+clean; 5.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-05-24T11:47:12.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:11 smithi039 ceph-mon[42919]: pgmap v278: 97 pgs: 97 active+clean; 5.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-05-24T11:47:14.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 470.0 s, 1015.4 tps, lat 100.792 ms stddev 48.192 2024-05-24T11:47:14.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:13 smithi053 ceph-mon[47448]: pgmap v279: 97 pgs: 97 active+clean; 5.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-05-24T11:47:14.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:13 smithi137 ceph-mon[47907]: pgmap v279: 97 pgs: 97 active+clean; 5.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-05-24T11:47:14.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:13 smithi039 ceph-mon[42919]: pgmap v279: 97 pgs: 97 active+clean; 5.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-05-24T11:47:16.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:15 smithi053 ceph-mon[47448]: pgmap v280: 97 pgs: 97 active+clean; 5.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2024-05-24T11:47:16.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:15 smithi137 ceph-mon[47907]: pgmap v280: 97 pgs: 97 active+clean; 5.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2024-05-24T11:47:16.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:15 smithi039 ceph-mon[42919]: pgmap v280: 97 pgs: 97 active+clean; 5.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2024-05-24T11:47:18.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:17 smithi053 ceph-mon[47448]: pgmap v281: 97 pgs: 97 active+clean; 5.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2024-05-24T11:47:18.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:17 smithi137 ceph-mon[47907]: pgmap v281: 97 pgs: 97 active+clean; 5.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2024-05-24T11:47:18.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:17 smithi039 ceph-mon[42919]: pgmap v281: 97 pgs: 97 active+clean; 5.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2024-05-24T11:47:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.745+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/1071071989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d98107ec0 msgr2=0x7f8d98114380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.745+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/1071071989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d98107ec0 0x7f8d98114380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009f90 tx=0x7f8d9000b330 comp rx=0 tx=0).stop 2024-05-24T11:47:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.745+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/1071071989 shutdown_connections 2024-05-24T11:47:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.745+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/1071071989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d98107ec0 0x7f8d98114380 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.745+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/1071071989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d9810d5c0 0x7f8d9810d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.745+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/1071071989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d98107500 0x7f8d98107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.745+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/1071071989 >> 172.21.15.39:0/1071071989 conn(0x7f8d98076dc0 msgr2=0x7f8d98105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:18.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.745+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/1071071989 shutdown_connections 2024-05-24T11:47:18.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.745+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/1071071989 wait complete. 2024-05-24T11:47:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.746+0000 7f8d9d8d5700 1 Processor -- start 2024-05-24T11:47:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.746+0000 7f8d9d8d5700 1 -- start start 2024-05-24T11:47:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.746+0000 7f8d9d8d5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d98107500 0x7f8d981a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.746+0000 7f8d9d8d5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d98107ec0 0x7f8d981a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.746+0000 7f8d9d8d5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d9810d5c0 0x7f8d9819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.746+0000 7f8d9d8d5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819dd70 con 0x7f8d98107500 2024-05-24T11:47:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.746+0000 7f8d9d8d5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819dee0 con 0x7f8d98107ec0 2024-05-24T11:47:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.746+0000 7f8d9d8d5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819e0a0 con 0x7f8d9810d5c0 2024-05-24T11:47:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.747+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d98107ec0 0x7f8d981a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.747+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d98107ec0 0x7f8d981a3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:38932/0 (socket says 172.21.15.39:38932) 2024-05-24T11:47:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.747+0000 7f8d967fc700 1 -- 172.21.15.39:0/3359509408 learned_addr learned my addr 172.21.15.39:0/3359509408 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.747+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d98107500 0x7f8d981a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.747+0000 7f8d977fe700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d9810d5c0 0x7f8d9819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.748+0000 7f8d967fc700 1 -- 172.21.15.39:0/3359509408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d9810d5c0 msgr2=0x7f8d9819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.748+0000 7f8d967fc700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d9810d5c0 0x7f8d9819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.748+0000 7f8d967fc700 1 -- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d98107500 msgr2=0x7f8d981a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.748+0000 7f8d967fc700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d98107500 0x7f8d981a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.748+0000 7f8d967fc700 1 -- 172.21.15.39:0/3359509408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90009c40 con 0x7f8d98107ec0 2024-05-24T11:47:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.748+0000 7f8d967fc700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d98107ec0 0x7f8d981a3ad0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00e9a0 tx=0x7f8d8c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.749+0000 7f8d9c8d3700 1 -- 172.21.15.39:0/3359509408 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c009790 con 0x7f8d98107ec0 2024-05-24T11:47:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.749+0000 7f8d9c8d3700 1 -- 172.21.15.39:0/3359509408 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d8c00ce50 con 0x7f8d98107ec0 2024-05-24T11:47:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.749+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/3359509408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9819e380 con 0x7f8d98107ec0 2024-05-24T11:47:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.749+0000 7f8d9c8d3700 1 -- 172.21.15.39:0/3359509408 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c018420 con 0x7f8d98107ec0 2024-05-24T11:47:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.750+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/3359509408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d981aa580 con 0x7f8d98107ec0 2024-05-24T11:47:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.753+0000 7f8d9c8d3700 1 -- 172.21.15.39:0/3359509408 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8d8c010430 con 0x7f8d98107ec0 2024-05-24T11:47:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.754+0000 7f8d9c8d3700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d800a13e0 0x7f8d800a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.754+0000 7f8d9c8d3700 1 -- 172.21.15.39:0/3359509408 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8d8c014070 con 0x7f8d98107ec0 2024-05-24T11:47:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.754+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d800a13e0 0x7f8d800a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.756+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d800a13e0 0x7f8d800a38a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005fd0 tx=0x7f8d88009310 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.758+0000 7f8da9a90b80 1 Processor -- start 2024-05-24T11:47:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.763+0000 7f8da9a90b80 1 -- start start 2024-05-24T11:47:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.763+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d6950 0x55c72f3d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.763+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d7270 0x55c72f3d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.763+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d1480 0x55c72f3d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.763+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f3280f0 con 0x55c72f3d6950 2024-05-24T11:47:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.763+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f325ab0 con 0x55c72f3d7270 2024-05-24T11:47:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.763+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f3213a0 con 0x55c72f3d1480 2024-05-24T11:47:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.763+0000 7f8d96ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d6950 0x55c72f3d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.763+0000 7f8d96ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d6950 0x55c72f3d6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56100/0 (socket says 172.21.15.39:56100) 2024-05-24T11:47:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.764+0000 7f8d96ffd700 1 -- 172.21.15.39:0/980883766 learned_addr learned my addr 172.21.15.39:0/980883766 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.764+0000 7f8d967fc700 1 --2- 172.21.15.39:0/980883766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d7270 0x55c72f3d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.764+0000 7f8d977fe700 1 --2- 172.21.15.39:0/980883766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d1480 0x55c72f3d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.764+0000 7f8d967fc700 1 -- 172.21.15.39:0/980883766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d1480 msgr2=0x55c72f3d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.764+0000 7f8d967fc700 1 --2- 172.21.15.39:0/980883766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d1480 0x55c72f3d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.764+0000 7f8d967fc700 1 -- 172.21.15.39:0/980883766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d6950 msgr2=0x55c72f3d6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.764+0000 7f8d967fc700 1 --2- 172.21.15.39:0/980883766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d6950 0x55c72f3d6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.764+0000 7f8d967fc700 1 -- 172.21.15.39:0/980883766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c72f1a2350 con 0x55c72f3d7270 2024-05-24T11:47:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.764+0000 7f8d967fc700 1 --2- 172.21.15.39:0/980883766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d7270 0x55c72f3d0f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0ca990 tx=0x7f8d8c010790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95e2843fbd295fca server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.765+0000 7f8d7dffb700 1 -- 172.21.15.39:0/980883766 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c09e180 con 0x55c72f3d7270 2024-05-24T11:47:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.765+0000 7f8d7dffb700 1 -- 172.21.15.39:0/980883766 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d8c0a27d0 con 0x55c72f3d7270 2024-05-24T11:47:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.765+0000 7f8d7dffb700 1 -- 172.21.15.39:0/980883766 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c09a810 con 0x55c72f3d7270 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 -- 172.21.15.39:0/980883766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d7270 msgr2=0x55c72f3d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/980883766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d7270 0x55c72f3d0f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0ca990 tx=0x7f8d8c010790 comp rx=0 tx=0).stop 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 -- 172.21.15.39:0/980883766 shutdown_connections 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/980883766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d1480 0x55c72f3d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/980883766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d6950 0x55c72f3d6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/980883766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d7270 0x55c72f3d0f40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 -- 172.21.15.39:0/980883766 >> 172.21.15.39:0/980883766 conn(0x55c72f32f1b0 msgr2=0x55c72f3cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 -- 172.21.15.39:0/980883766 shutdown_connections 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 -- 172.21.15.39:0/980883766 wait complete. 2024-05-24T11:47:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.766+0000 7f8da9a90b80 1 -- start start 2024-05-24T11:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8da9a90b80 1 Processor -- start 2024-05-24T11:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3dd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f2bd500 con 0x55c72f3d7270 2024-05-24T11:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f2bd1a0 con 0x55c72f3d1480 2024-05-24T11:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f1a4640 con 0x55c72f3d6950 2024-05-24T11:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8da9a90b80 10 client.0 _ll_register_callbacks cb 0x55c72f2c6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:47:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3da4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:49386/0 (socket says 172.21.15.39:49386) 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.767+0000 7f8d967fc700 1 -- 172.21.15.39:0/3231735223 learned_addr learned my addr 172.21.15.39:0/3231735223 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d977fe700 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d967fc700 1 -- 172.21.15.39:0/3231735223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 msgr2=0x55c72f3dd250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d967fc700 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3dd250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d967fc700 1 -- 172.21.15.39:0/3231735223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 msgr2=0x55c72f3dac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d967fc700 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3dac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d967fc700 1 -- 172.21.15.39:0/3231735223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d981aa580 con 0x55c72f3d6950 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d967fc700 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3da4d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00e970 tx=0x7f8d8c099b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ff637633b9c6ab server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d76ffd700 1 -- 172.21.15.39:0/3231735223 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c09a7c0 con 0x55c72f3d6950 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d76ffd700 1 -- 172.21.15.39:0/3231735223 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8d8c0b8030 con 0x55c72f3d6950 2024-05-24T11:47:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8da9a90b80 1 -- 172.21.15.39:0/3231735223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c72f27a210 con 0x55c72f3d6950 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.768+0000 7f8d76ffd700 1 -- 172.21.15.39:0/3231735223 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d8c0a2340 con 0x55c72f3d6950 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8da9a90b80 10 client.25666 fetch_fsmap learned FSMap version 29 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8da9a90b80 1 -- 172.21.15.39:0/3231735223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c72f14eb00 con 0x55c72f3d6950 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8d76ffd700 1 -- 172.21.15.39:0/3231735223 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c72f27a210 con 0x55c72f3d6950 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8d76ffd700 1 -- 172.21.15.39:0/3231735223 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d8c096140 con 0x55c72f3d6950 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8da9a90b80 10 client.25666 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8da9a90b80 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8da9a90b80 20 client.25666 populate_metadata read hostname 'smithi039' 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c72f3fb870 0x55c72f3fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8da9a90b80 4 client.25666 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8da9a90b80 1 -- 172.21.15.39:0/3231735223 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c72f3a07b0 con 0x55c72f3fb870 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.769+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c72f3fb870 0x55c72f3fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.770+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c72f3fb870 0x55c72f3fdc90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc526455992bd139 server_cookie=b11fd18dfb70ffec in_seq=0 out_seq=0 2024-05-24T11:47:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.770+0000 7f8d76ffd700 10 client.25666 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.771+0000 7f8d76ffd700 1 -- 172.21.15.39:0/3231735223 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c72f3a07b0 con 0x55c72f3fb870 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.771+0000 7f8d76ffd700 10 client.25666 handle_command_reply: tid=0 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.771+0000 7f8da9a90b80 1 client.25666 shutdown 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.773+0000 7f8da9a90b80 1 -- 172.21.15.39:0/3231735223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 msgr2=0x55c72f3da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.773+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3da4d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00e970 tx=0x7f8d8c099b20 comp rx=0 tx=0).stop 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8da9a90b80 1 -- 172.21.15.39:0/3231735223 shutdown_connections 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c72f3fb870 0x55c72f3fdc90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3dac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3dd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/3231735223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3da4d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8da9a90b80 1 -- 172.21.15.39:0/3231735223 >> 172.21.15.39:0/3231735223 conn(0x55c72f32f1b0 msgr2=0x55c72f3cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8d76ffd700 0 client.25666 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8da9a90b80 1 -- 172.21.15.39:0/3231735223 shutdown_connections 2024-05-24T11:47:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8da9a90b80 1 -- 172.21.15.39:0/3231735223 wait complete. 2024-05-24T11:47:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.774+0000 7f8da9a90b80 20 client.25666 trim_cache size 0 max 16384 2024-05-24T11:47:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.800+0000 7f8da9a90b80 1 Processor -- start 2024-05-24T11:47:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.800+0000 7f8da9a90b80 1 -- start start 2024-05-24T11:47:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.800+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.800+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.800+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3dcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.800+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f3d8a40 con 0x55c72f3d7270 2024-05-24T11:47:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.800+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f3d5130 con 0x55c72f3d1480 2024-05-24T11:47:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.800+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f3ee7d0 con 0x55c72f3d6950 2024-05-24T11:47:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d96ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3d0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:49392/0 (socket says 172.21.15.39:49392) 2024-05-24T11:47:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d967fc700 1 -- 172.21.15.39:0/2865317426 learned_addr learned my addr 172.21.15.39:0/2865317426 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d977fe700 1 --2- 172.21.15.39:0/2865317426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3dcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d96ffd700 1 -- 172.21.15.39:0/2865317426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 msgr2=0x55c72f3d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/2865317426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d96ffd700 1 -- 172.21.15.39:0/2865317426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 msgr2=0x55c72f3dcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/2865317426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3dcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d96ffd700 1 -- 172.21.15.39:0/2865317426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c72f2e7500 con 0x55c72f3d1480 2024-05-24T11:47:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.801+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/2865317426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3cff30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55c72f3fe540 tx=0x7f8d88017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=252b260252ecac00 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.802+0000 7f8d7e7fc700 1 -- 172.21.15.39:0/2865317426 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88024040 con 0x55c72f3d1480 2024-05-24T11:47:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.802+0000 7f8d7e7fc700 1 -- 172.21.15.39:0/2865317426 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d88026d00 con 0x55c72f3d1480 2024-05-24T11:47:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.802+0000 7f8d7e7fc700 1 -- 172.21.15.39:0/2865317426 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88003a70 con 0x55c72f3d1480 2024-05-24T11:47:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.803+0000 7f8da9a90b80 1 -- 172.21.15.39:0/2865317426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 msgr2=0x55c72f3cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.803+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/2865317426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3cff30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55c72f3fe540 tx=0x7f8d88017000 comp rx=0 tx=0).stop 2024-05-24T11:47:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.803+0000 7f8da9a90b80 1 -- 172.21.15.39:0/2865317426 shutdown_connections 2024-05-24T11:47:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.803+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/2865317426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3dcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.803+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/2865317426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3cff30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.803+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/2865317426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.803+0000 7f8da9a90b80 1 -- 172.21.15.39:0/2865317426 >> 172.21.15.39:0/2865317426 conn(0x55c72f32f1b0 msgr2=0x55c72f3c7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.804+0000 7f8da9a90b80 1 -- 172.21.15.39:0/2865317426 shutdown_connections 2024-05-24T11:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.804+0000 7f8da9a90b80 1 -- 172.21.15.39:0/2865317426 wait complete. 2024-05-24T11:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.804+0000 7f8da9a90b80 1 -- start start 2024-05-24T11:47:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.804+0000 7f8da9a90b80 1 Processor -- start 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3f3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8da9a90b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f3f1880 con 0x55c72f3d7270 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f3f19f0 con 0x55c72f3d1480 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8da9a90b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c72f461d80 con 0x55c72f3d6950 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8da9a90b80 10 client.0 _ll_register_callbacks cb 0x55c72f2c6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8d96ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8d977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8d977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3f11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56128/0 (socket says 172.21.15.39:56128) 2024-05-24T11:47:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.805+0000 7f8d977fe700 1 -- 172.21.15.39:0/630142866 learned_addr learned my addr 172.21.15.39:0/630142866 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8d96ffd700 1 -- 172.21.15.39:0/630142866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 msgr2=0x55c72f3f0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3f0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8d96ffd700 1 -- 172.21.15.39:0/630142866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 msgr2=0x55c72f3f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8d96ffd700 1 -- 172.21.15.39:0/630142866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d88017070 con 0x55c72f3d1480 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8d96ffd700 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3f3010 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d88028040 tx=0x7f8d88009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c3c38c2ba75f90c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8d76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8d76ffd700 1 -- 172.21.15.39:0/630142866 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88024050 con 0x55c72f3d1480 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8d76ffd700 1 -- 172.21.15.39:0/630142866 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8d880055d0 con 0x55c72f3d1480 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.806+0000 7f8da9a90b80 1 -- 172.21.15.39:0/630142866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d8c0a27d0 con 0x55c72f3d1480 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.807+0000 7f8d76ffd700 1 -- 172.21.15.39:0/630142866 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d88005790 con 0x55c72f3d1480 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.807+0000 7f8da9a90b80 10 client.25754 fetch_fsmap learned FSMap version 29 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.807+0000 7f8da9a90b80 1 -- 172.21.15.39:0/630142866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c72f1a6c70 con 0x55c72f3d1480 2024-05-24T11:47:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.807+0000 7f8d76ffd700 1 -- 172.21.15.39:0/630142866 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d8c0a27d0 con 0x55c72f3d1480 2024-05-24T11:47:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.808+0000 7f8d76ffd700 1 -- 172.21.15.39:0/630142866 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d880037a0 con 0x55c72f3d1480 2024-05-24T11:47:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.808+0000 7f8da9a90b80 10 client.25754 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:47:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.808+0000 7f8da9a90b80 10 client.25754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:47:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.808+0000 7f8da9a90b80 20 client.25754 populate_metadata read hostname 'smithi039' 2024-05-24T11:47:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.808+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c72f463470 0x55c72f465890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.808+0000 7f8da9a90b80 4 client.25754 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:47:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.808+0000 7f8da9a90b80 1 -- 172.21.15.39:0/630142866 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c72f319e60 con 0x55c72f463470 2024-05-24T11:47:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.808+0000 7f8d967fc700 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c72f463470 0x55c72f465890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.809+0000 7f8d76ffd700 10 client.25754 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:18.809+0000 7f8d967fc700 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c72f463470 0x55c72f465890 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921142cb0e2d910d server_cookie=99fac49855194712 in_seq=0 out_seq=0 2024-05-24T11:47:19.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 475.0 s, 1086.4 tps, lat 92.237 ms stddev 29.379 2024-05-24T11:47:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.587+0000 7f8d76ffd700 1 -- 172.21.15.39:0/630142866 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c72f319e60 con 0x55c72f463470 2024-05-24T11:47:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.587+0000 7f8d76ffd700 10 client.25754 handle_command_reply: tid=0 2024-05-24T11:47:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 client.25754 shutdown 2024-05-24T11:47:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 -- 172.21.15.39:0/630142866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 msgr2=0x55c72f3f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3f3010 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d88028040 tx=0x7f8d88009fb0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 -- 172.21.15.39:0/630142866 shutdown_connections 2024-05-24T11:47:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c72f463470 0x55c72f465890 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c72f3d7270 0x55c72f3f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c72f3d1480 0x55c72f3f3010 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 --2- 172.21.15.39:0/630142866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c72f3d6950 0x55c72f3f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 -- 172.21.15.39:0/630142866 >> 172.21.15.39:0/630142866 conn(0x55c72f32f1b0 msgr2=0x55c72f3d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8d76ffd700 0 client.25754 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 -- 172.21.15.39:0/630142866 shutdown_connections 2024-05-24T11:47:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.588+0000 7f8da9a90b80 1 -- 172.21.15.39:0/630142866 wait complete. 2024-05-24T11:47:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.589+0000 7f8da9a90b80 20 client.25754 trim_cache size 0 max 16384 2024-05-24T11:47:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:47:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:47:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:47:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:47:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.589+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d800a13e0 msgr2=0x7f8d800a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.589+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d800a13e0 0x7f8d800a38a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005fd0 tx=0x7f8d88009310 comp rx=0 tx=0).stop 2024-05-24T11:47:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.589+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/3359509408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d98107ec0 msgr2=0x7f8d981a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.589+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d98107ec0 0x7f8d981a3ad0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00e9a0 tx=0x7f8d8c00ecb0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.592+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/3359509408 shutdown_connections 2024-05-24T11:47:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.592+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d9810d5c0 0x7f8d9819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.592+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d800a13e0 0x7f8d800a38a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.592+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d98107500 0x7f8d981a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.592+0000 7f8d9d8d5700 1 --2- 172.21.15.39:0/3359509408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d98107ec0 0x7f8d981a3ad0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.592+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/3359509408 >> 172.21.15.39:0/3359509408 conn(0x7f8d98076dc0 msgr2=0x7f8d98077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.592+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/3359509408 shutdown_connections 2024-05-24T11:47:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:19.592+0000 7f8d9d8d5700 1 -- 172.21.15.39:0/3359509408 wait complete. 2024-05-24T11:47:20.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:19 smithi053 ceph-mon[47448]: pgmap v282: 97 pgs: 97 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 23 MiB/s wr, 1.38k op/s 2024-05-24T11:47:20.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:19 smithi053 ceph-mon[47448]: osdmap e87: 12 total, 12 up, 12 in 2024-05-24T11:47:20.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:19 smithi137 ceph-mon[47907]: pgmap v282: 97 pgs: 97 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 23 MiB/s wr, 1.38k op/s 2024-05-24T11:47:20.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:19 smithi137 ceph-mon[47907]: osdmap e87: 12 total, 12 up, 12 in 2024-05-24T11:47:20.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:19 smithi039 ceph-mon[42919]: pgmap v282: 97 pgs: 97 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 23 MiB/s wr, 1.38k op/s 2024-05-24T11:47:20.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:19 smithi039 ceph-mon[42919]: osdmap e87: 12 total, 12 up, 12 in 2024-05-24T11:47:22.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:21 smithi053 ceph-mon[47448]: pgmap v284: 97 pgs: 97 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 23 MiB/s wr, 1.38k op/s 2024-05-24T11:47:22.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:21 smithi137 ceph-mon[47907]: pgmap v284: 97 pgs: 97 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 23 MiB/s wr, 1.38k op/s 2024-05-24T11:47:22.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:21 smithi039 ceph-mon[42919]: pgmap v284: 97 pgs: 97 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 23 MiB/s wr, 1.38k op/s 2024-05-24T11:47:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 480.0 s, 907.0 tps, lat 109.794 ms stddev 186.661 2024-05-24T11:47:24.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:23 smithi053 ceph-mon[47448]: pgmap v285: 97 pgs: 97 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 23 MiB/s wr, 1.38k op/s 2024-05-24T11:47:24.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:23 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:47:24.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:23 smithi137 ceph-mon[47907]: pgmap v285: 97 pgs: 97 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 23 MiB/s wr, 1.38k op/s 2024-05-24T11:47:24.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:23 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:47:24.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:23 smithi039 ceph-mon[42919]: pgmap v285: 97 pgs: 97 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 23 MiB/s wr, 1.38k op/s 2024-05-24T11:47:24.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:23 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:47:25.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:24 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:47:25.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:24 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:47:25.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:24 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:47:25.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:24 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:47:25.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:24 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:47:25.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:24 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:47:25.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:24 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:47:25.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:24 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:47:25.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:24 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:47:26.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:25 smithi053 ceph-mon[47448]: pgmap v286: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 24 MiB/s wr, 1.50k op/s 2024-05-24T11:47:26.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:25 smithi137 ceph-mon[47907]: pgmap v286: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 24 MiB/s wr, 1.50k op/s 2024-05-24T11:47:26.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:25 smithi039 ceph-mon[42919]: pgmap v286: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 24 MiB/s wr, 1.50k op/s 2024-05-24T11:47:28.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:27 smithi053 ceph-mon[47448]: pgmap v287: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 24 MiB/s wr, 1.50k op/s 2024-05-24T11:47:28.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:27 smithi137 ceph-mon[47907]: pgmap v287: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 24 MiB/s wr, 1.50k op/s 2024-05-24T11:47:28.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:27 smithi039 ceph-mon[42919]: pgmap v287: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 24 MiB/s wr, 1.50k op/s 2024-05-24T11:47:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 485.0 s, 1030.6 tps, lat 97.278 ms stddev 74.874 2024-05-24T11:47:30.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:29 smithi053 ceph-mon[47448]: pgmap v288: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 22 MiB/s wr, 1.43k op/s 2024-05-24T11:47:30.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:29 smithi137 ceph-mon[47907]: pgmap v288: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 22 MiB/s wr, 1.43k op/s 2024-05-24T11:47:30.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:29 smithi039 ceph-mon[42919]: pgmap v288: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 22 MiB/s wr, 1.43k op/s 2024-05-24T11:47:32.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:31 smithi053 ceph-mon[47448]: pgmap v289: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 21 MiB/s wr, 1.34k op/s 2024-05-24T11:47:32.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:31 smithi137 ceph-mon[47907]: pgmap v289: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 21 MiB/s wr, 1.34k op/s 2024-05-24T11:47:32.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:31 smithi039 ceph-mon[42919]: pgmap v289: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 21 MiB/s wr, 1.34k op/s 2024-05-24T11:47:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.767+0000 7f8753063700 1 -- 172.21.15.39:0/682734260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c100760 msgr2=0x7f874c10cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.767+0000 7f8753063700 1 --2- 172.21.15.39:0/682734260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c100760 0x7f874c10cc00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f874400d210 tx=0x7f874400d520 comp rx=0 tx=0).stop 2024-05-24T11:47:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.767+0000 7f8753063700 1 -- 172.21.15.39:0/682734260 shutdown_connections 2024-05-24T11:47:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.767+0000 7f8753063700 1 --2- 172.21.15.39:0/682734260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f874c105e60 0x7f874c106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.767+0000 7f8753063700 1 --2- 172.21.15.39:0/682734260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c100760 0x7f874c10cc00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.767+0000 7f8753063700 1 --2- 172.21.15.39:0/682734260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f874c0ffda0 0x7f874c100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.768+0000 7f8753063700 1 -- 172.21.15.39:0/682734260 >> 172.21.15.39:0/682734260 conn(0x7f874c076f10 msgr2=0x7f874c077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.768+0000 7f8753063700 1 -- 172.21.15.39:0/682734260 shutdown_connections 2024-05-24T11:47:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.769+0000 7f8753063700 1 -- 172.21.15.39:0/682734260 wait complete. 2024-05-24T11:47:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.769+0000 7f8753063700 1 Processor -- start 2024-05-24T11:47:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.769+0000 7f8753063700 1 -- start start 2024-05-24T11:47:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8753063700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f874c0ffda0 0x7f874c072710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8753063700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f874c100760 0x7f874c072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8753063700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c105e60 0x7f874c070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8753063700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c070bd0 con 0x7f874c105e60 2024-05-24T11:47:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8753063700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c070d40 con 0x7f874c0ffda0 2024-05-24T11:47:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8753063700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c070f00 con 0x7f874c100760 2024-05-24T11:47:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f874bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f874c100760 0x7f874c072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8750dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f874c0ffda0 0x7f874c072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f874bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f874c100760 0x7f874c072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:58782/0 (socket says 172.21.15.39:58782) 2024-05-24T11:47:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8750dff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f874c0ffda0 0x7f874c072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:50380/0 (socket says 172.21.15.39:50380) 2024-05-24T11:47:32.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f874bfff700 1 -- 172.21.15.39:0/4128350646 learned_addr learned my addr 172.21.15.39:0/4128350646 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8751600700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c105e60 0x7f874c070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f874bfff700 1 -- 172.21.15.39:0/4128350646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f874c0ffda0 msgr2=0x7f874c072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f874bfff700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f874c0ffda0 0x7f874c072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f874bfff700 1 -- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c105e60 msgr2=0x7f874c070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f874bfff700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c105e60 0x7f874c070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f874bfff700 1 -- 172.21.15.39:0/4128350646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874400be50 con 0x7f874c100760 2024-05-24T11:47:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f8750dff700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f874c0ffda0 0x7f874c072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:47:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.770+0000 7f874bfff700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f874c100760 0x7f874c072c50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f873800e9c0 tx=0x7f873800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.771+0000 7f8749ffb700 1 -- 172.21.15.39:0/4128350646 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800cbd0 con 0x7f874c100760 2024-05-24T11:47:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.771+0000 7f8753063700 1 -- 172.21.15.39:0/4128350646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c06b8b0 con 0x7f874c100760 2024-05-24T11:47:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.771+0000 7f8749ffb700 1 -- 172.21.15.39:0/4128350646 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f873800eeb0 con 0x7f874c100760 2024-05-24T11:47:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.771+0000 7f8749ffb700 1 -- 172.21.15.39:0/4128350646 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738005420 con 0x7f874c100760 2024-05-24T11:47:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.771+0000 7f8753063700 1 -- 172.21.15.39:0/4128350646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c06be10 con 0x7f874c100760 2024-05-24T11:47:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.774+0000 7f8749ffb700 1 -- 172.21.15.39:0/4128350646 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f87380055a0 con 0x7f874c100760 2024-05-24T11:47:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.774+0000 7f8749ffb700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f873c0a1390 0x7f873c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.775+0000 7f8749ffb700 1 -- 172.21.15.39:0/4128350646 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8738014070 con 0x7f874c100760 2024-05-24T11:47:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.775+0000 7f8750dff700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f873c0a1390 0x7f873c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.775+0000 7f875f21eb80 1 Processor -- start 2024-05-24T11:47:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.775+0000 7f8750dff700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f873c0a1390 0x7f873c0a3850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f874c0fe410 tx=0x7f8740009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.775+0000 7f875f21eb80 1 -- start start 2024-05-24T11:47:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a3a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f875f21eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a2f90f0 con 0x55691a3a7950 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f875f21eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a2f6ab0 con 0x55691a3a8270 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f875f21eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a2f23a0 con 0x55691a3a2480 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f874bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f874bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:50392/0 (socket says 172.21.15.39:50392) 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f874bfff700 1 -- 172.21.15.39:0/4010153122 learned_addr learned my addr 172.21.15.39:0/4010153122 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f8750dff700 1 --2- 172.21.15.39:0/4010153122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f874bfff700 1 -- 172.21.15.39:0/4010153122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 msgr2=0x55691a3a28a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f874bfff700 1 --2- 172.21.15.39:0/4010153122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a3a28a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f874bfff700 1 -- 172.21.15.39:0/4010153122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 msgr2=0x55691a3a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f874bfff700 1 --2- 172.21.15.39:0/4010153122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3a7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.776+0000 7f874bfff700 1 -- 172.21.15.39:0/4010153122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55691a173350 con 0x55691a3a8270 2024-05-24T11:47:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.777+0000 7f874bfff700 1 --2- 172.21.15.39:0/4010153122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3a1f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8738096550 tx=0x7f873800cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7da013813a42b10a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.777+0000 7f8736ffd700 1 -- 172.21.15.39:0/4010153122 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380a24d0 con 0x55691a3a8270 2024-05-24T11:47:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.777+0000 7f8736ffd700 1 -- 172.21.15.39:0/4010153122 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8738016df0 con 0x55691a3a8270 2024-05-24T11:47:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.777+0000 7f8736ffd700 1 -- 172.21.15.39:0/4010153122 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873809a4b0 con 0x55691a3a8270 2024-05-24T11:47:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 -- 172.21.15.39:0/4010153122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 msgr2=0x55691a3a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 --2- 172.21.15.39:0/4010153122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3a1f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8738096550 tx=0x7f873800cdb0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 -- 172.21.15.39:0/4010153122 shutdown_connections 2024-05-24T11:47:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 --2- 172.21.15.39:0/4010153122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a3a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 --2- 172.21.15.39:0/4010153122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 --2- 172.21.15.39:0/4010153122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3a1f40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 -- 172.21.15.39:0/4010153122 >> 172.21.15.39:0/4010153122 conn(0x55691a3001b0 msgr2=0x55691a39c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 -- 172.21.15.39:0/4010153122 shutdown_connections 2024-05-24T11:47:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 -- 172.21.15.39:0/4010153122 wait complete. 2024-05-24T11:47:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.778+0000 7f875f21eb80 1 -- start start 2024-05-24T11:47:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.779+0000 7f875f21eb80 1 Processor -- start 2024-05-24T11:47:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.779+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a3ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.779+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.779+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3abb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.779+0000 7f875f21eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a28e500 con 0x55691a3a7950 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.779+0000 7f875f21eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a28e1a0 con 0x55691a3a8270 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.779+0000 7f875f21eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a175640 con 0x55691a3a2480 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.779+0000 7f875f21eb80 10 client.0 _ll_register_callbacks cb 0x55691a297ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f874bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f874bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3ab530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43768/0 (socket says 172.21.15.39:43768) 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f874bfff700 1 -- 172.21.15.39:0/1670328035 learned_addr learned my addr 172.21.15.39:0/1670328035 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f8750dff700 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a3ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f8751600700 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f874bfff700 1 -- 172.21.15.39:0/1670328035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 msgr2=0x55691a3ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f874bfff700 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a3ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f874bfff700 1 -- 172.21.15.39:0/1670328035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 msgr2=0x55691a3abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f874bfff700 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f874bfff700 1 -- 172.21.15.39:0/1670328035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c06be10 con 0x55691a3a7950 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f874bfff700 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3ab530 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8738096500 tx=0x7f8738099770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3647659d39a3b301 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.780+0000 7f872bfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.781+0000 7f872bfff700 1 -- 172.21.15.39:0/1670328035 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873807a050 con 0x55691a3a7950 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.781+0000 7f872bfff700 1 -- 172.21.15.39:0/1670328035 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f87380a24d0 con 0x55691a3a7950 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.781+0000 7f875f21eb80 1 -- 172.21.15.39:0/1670328035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55691a24b210 con 0x55691a3a7950 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.781+0000 7f872bfff700 1 -- 172.21.15.39:0/1670328035 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f873809a2b0 con 0x55691a3a7950 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.781+0000 7f875f21eb80 10 client.16056 fetch_fsmap learned FSMap version 29 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.781+0000 7f875f21eb80 1 -- 172.21.15.39:0/1670328035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55691a11fb00 con 0x55691a3a7950 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.781+0000 7f872bfff700 1 -- 172.21.15.39:0/1670328035 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87380a6070 con 0x55691a3a7950 2024-05-24T11:47:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.782+0000 7f872bfff700 1 -- 172.21.15.39:0/1670328035 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f87380a2040 con 0x55691a3a7950 2024-05-24T11:47:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.782+0000 7f875f21eb80 10 client.16056 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.782+0000 7f875f21eb80 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.782+0000 7f875f21eb80 20 client.16056 populate_metadata read hostname 'smithi039' 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.782+0000 7f875f21eb80 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55691a3c4100 0x55691a3c6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.782+0000 7f875f21eb80 4 client.16056 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.782+0000 7f875f21eb80 1 -- 172.21.15.39:0/1670328035 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55691a3717b0 con 0x55691a3c4100 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.782+0000 7f8750dff700 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55691a3c4100 0x55691a3c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.783+0000 7f872bfff700 10 client.16056 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.783+0000 7f8750dff700 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55691a3c4100 0x55691a3c6520 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303c7a9864382e5c server_cookie=97c86266ea498e02 in_seq=0 out_seq=0 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f872bfff700 1 -- 172.21.15.39:0/1670328035 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55691a3717b0 con 0x55691a3c4100 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f872bfff700 10 client.16056 handle_command_reply: tid=0 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f875f21eb80 1 client.16056 shutdown 2024-05-24T11:47:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f875f21eb80 1 -- 172.21.15.39:0/1670328035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 msgr2=0x55691a3ab530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f875f21eb80 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3ab530 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8738096500 tx=0x7f8738099770 comp rx=0 tx=0).stop 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f875f21eb80 1 -- 172.21.15.39:0/1670328035 shutdown_connections 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f875f21eb80 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a3ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f875f21eb80 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55691a3c4100 0x55691a3c6520 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f875f21eb80 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f875f21eb80 1 --2- 172.21.15.39:0/1670328035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3ab530 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f875f21eb80 1 -- 172.21.15.39:0/1670328035 >> 172.21.15.39:0/1670328035 conn(0x55691a3001b0 msgr2=0x55691a3a64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.784+0000 7f872bfff700 0 client.16056 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.785+0000 7f875f21eb80 1 -- 172.21.15.39:0/1670328035 shutdown_connections 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.785+0000 7f875f21eb80 1 -- 172.21.15.39:0/1670328035 wait complete. 2024-05-24T11:47:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.785+0000 7f875f21eb80 20 client.16056 trim_cache size 0 max 16384 2024-05-24T11:47:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.809+0000 7f875f21eb80 1 Processor -- start 2024-05-24T11:47:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.809+0000 7f875f21eb80 1 -- start start 2024-05-24T11:47:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.809+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a39c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.809+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3a6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.809+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3bb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.809+0000 7f875f21eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a3cb720 con 0x55691a3a7950 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.809+0000 7f875f21eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a3a9a80 con 0x55691a3a8270 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.809+0000 7f875f21eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a3a1170 con 0x55691a3a2480 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8751600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3bb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8751600700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3bb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:50412/0 (socket says 172.21.15.39:50412) 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8751600700 1 -- 172.21.15.39:0/106679346 learned_addr learned my addr 172.21.15.39:0/106679346 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8750dff700 1 --2- 172.21.15.39:0/106679346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a39c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f874bfff700 1 --2- 172.21.15.39:0/106679346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8750dff700 1 -- 172.21.15.39:0/106679346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 msgr2=0x55691a3bb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8750dff700 1 --2- 172.21.15.39:0/106679346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3bb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8750dff700 1 -- 172.21.15.39:0/106679346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 msgr2=0x55691a3a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8750dff700 1 --2- 172.21.15.39:0/106679346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3a6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8750dff700 1 -- 172.21.15.39:0/106679346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55691a3a2b30 con 0x55691a3a2480 2024-05-24T11:47:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f8750dff700 1 --2- 172.21.15.39:0/106679346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a39c910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8740013010 tx=0x7f8740014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6074f41fbabf6ea server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.810+0000 7f87377fe700 1 -- 172.21.15.39:0/106679346 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740024040 con 0x55691a3a2480 2024-05-24T11:47:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.811+0000 7f87377fe700 1 -- 172.21.15.39:0/106679346 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f874002a4c0 con 0x55691a3a2480 2024-05-24T11:47:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.811+0000 7f87377fe700 1 -- 172.21.15.39:0/106679346 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874001f890 con 0x55691a3a2480 2024-05-24T11:47:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.811+0000 7f875f21eb80 1 -- 172.21.15.39:0/106679346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 msgr2=0x55691a39c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.812+0000 7f875f21eb80 1 --2- 172.21.15.39:0/106679346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a39c910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8740013010 tx=0x7f8740014000 comp rx=0 tx=0).stop 2024-05-24T11:47:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.812+0000 7f875f21eb80 1 -- 172.21.15.39:0/106679346 shutdown_connections 2024-05-24T11:47:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.812+0000 7f875f21eb80 1 --2- 172.21.15.39:0/106679346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a2480 0x55691a39c910 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8740013010 tx=0x7f8740014000 comp rx=0 tx=0).stop 2024-05-24T11:47:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.812+0000 7f875f21eb80 1 --2- 172.21.15.39:0/106679346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a8270 0x55691a3bb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.812+0000 7f875f21eb80 1 --2- 172.21.15.39:0/106679346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a7950 0x55691a3a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.812+0000 7f875f21eb80 1 -- 172.21.15.39:0/106679346 >> 172.21.15.39:0/106679346 conn(0x55691a3001b0 msgr2=0x55691a3a0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.813+0000 7f875f21eb80 1 -- 172.21.15.39:0/106679346 shutdown_connections 2024-05-24T11:47:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.813+0000 7f875f21eb80 1 -- 172.21.15.39:0/106679346 wait complete. 2024-05-24T11:47:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.813+0000 7f875f21eb80 1 -- start start 2024-05-24T11:47:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f875f21eb80 1 Processor -- start 2024-05-24T11:47:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a7950 0x55691a3bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a8270 0x55691a3bba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f875f21eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a53f0 0x55691a3a5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f875f21eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a3a5f80 con 0x55691a3a53f0 2024-05-24T11:47:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f875f21eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a3a60f0 con 0x55691a3a7950 2024-05-24T11:47:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f875f21eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691a3a6230 con 0x55691a3a8270 2024-05-24T11:47:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f875f21eb80 10 client.0 _ll_register_callbacks cb 0x55691a297ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:47:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f8751600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a53f0 0x55691a3a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f8751600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a53f0 0x55691a3a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43792/0 (socket says 172.21.15.39:43792) 2024-05-24T11:47:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f8751600700 1 -- 172.21.15.39:0/3570088084 learned_addr learned my addr 172.21.15.39:0/3570088084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.814+0000 7f8750dff700 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a7950 0x55691a3bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.815+0000 7f874bfff700 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a8270 0x55691a3bba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.815+0000 7f8751600700 1 -- 172.21.15.39:0/3570088084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a8270 msgr2=0x55691a3bba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.815+0000 7f8751600700 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a8270 0x55691a3bba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.815+0000 7f8751600700 1 -- 172.21.15.39:0/3570088084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a7950 msgr2=0x55691a3bb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.815+0000 7f8751600700 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a7950 0x55691a3bb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.815+0000 7f8751600700 1 -- 172.21.15.39:0/3570088084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8740017070 con 0x55691a3a53f0 2024-05-24T11:47:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.815+0000 7f8750dff700 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a7950 0x55691a3bb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:47:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.816+0000 7f8751600700 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a53f0 0x55691a3a5870 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f874400d8a0 tx=0x7f8744009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4450d5c21cc15cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.816+0000 7f872bfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:47:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.816+0000 7f872bfff700 1 -- 172.21.15.39:0/3570088084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744010070 con 0x55691a3a53f0 2024-05-24T11:47:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.816+0000 7f872bfff700 1 -- 172.21.15.39:0/3570088084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8744009400 con 0x55691a3a53f0 2024-05-24T11:47:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.816+0000 7f875f21eb80 1 -- 172.21.15.39:0/3570088084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8738016df0 con 0x55691a3a53f0 2024-05-24T11:47:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.817+0000 7f872bfff700 1 -- 172.21.15.39:0/3570088084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8744003a00 con 0x55691a3a53f0 2024-05-24T11:47:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.817+0000 7f875f21eb80 10 client.16062 fetch_fsmap learned FSMap version 29 2024-05-24T11:47:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.817+0000 7f875f21eb80 1 -- 172.21.15.39:0/3570088084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55691a2b8500 con 0x55691a3a53f0 2024-05-24T11:47:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.818+0000 7f872bfff700 1 -- 172.21.15.39:0/3570088084 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8738016df0 con 0x55691a3a53f0 2024-05-24T11:47:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.818+0000 7f872bfff700 1 -- 172.21.15.39:0/3570088084 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f874400d240 con 0x55691a3a53f0 2024-05-24T11:47:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.818+0000 7f875f21eb80 10 client.16062 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:47:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.818+0000 7f875f21eb80 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:47:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.818+0000 7f875f21eb80 20 client.16062 populate_metadata read hostname 'smithi039' 2024-05-24T11:47:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.818+0000 7f875f21eb80 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55691a3b02e0 0x55691a3b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.818+0000 7f875f21eb80 4 client.16062 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:47:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.818+0000 7f875f21eb80 1 -- 172.21.15.39:0/3570088084 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55691a2eae60 con 0x55691a3b02e0 2024-05-24T11:47:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.818+0000 7f8750dff700 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55691a3b02e0 0x55691a3b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.819+0000 7f8750dff700 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55691a3b02e0 0x55691a3b2700 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24f82ca1aece6aa server_cookie=7cc9189ef9bf98b in_seq=0 out_seq=0 2024-05-24T11:47:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.819+0000 7f872bfff700 10 client.16062 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.866+0000 7f872bfff700 1 -- 172.21.15.39:0/3570088084 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55691a2eae60 con 0x55691a3b02e0 2024-05-24T11:47:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.866+0000 7f872bfff700 10 client.16062 handle_command_reply: tid=0 2024-05-24T11:47:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.867+0000 7f875f21eb80 1 client.16062 shutdown 2024-05-24T11:47:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.867+0000 7f875f21eb80 1 -- 172.21.15.39:0/3570088084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a53f0 msgr2=0x55691a3a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.867+0000 7f875f21eb80 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a53f0 0x55691a3a5870 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f874400d8a0 tx=0x7f8744009f50 comp rx=0 tx=0).stop 2024-05-24T11:47:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f875f21eb80 1 -- 172.21.15.39:0/3570088084 shutdown_connections 2024-05-24T11:47:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f875f21eb80 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55691a3b02e0 0x55691a3b2700 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f875f21eb80 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55691a3a53f0 0x55691a3a5870 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f875f21eb80 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55691a3a7950 0x55691a3bb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f875f21eb80 1 --2- 172.21.15.39:0/3570088084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55691a3a8270 0x55691a3bba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f875f21eb80 1 -- 172.21.15.39:0/3570088084 >> 172.21.15.39:0/3570088084 conn(0x55691a3001b0 msgr2=0x55691a39e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f872bfff700 0 client.16062 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f875f21eb80 1 -- 172.21.15.39:0/3570088084 shutdown_connections 2024-05-24T11:47:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f875f21eb80 1 -- 172.21.15.39:0/3570088084 wait complete. 2024-05-24T11:47:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.868+0000 7f875f21eb80 20 client.16062 trim_cache size 0 max 16384 2024-05-24T11:47:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:47:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:47:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:47:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:47:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 -- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f873c0a1390 msgr2=0x7f873c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f873c0a1390 0x7f873c0a3850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f874c0fe410 tx=0x7f8740009260 comp rx=0 tx=0).stop 2024-05-24T11:47:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 -- 172.21.15.39:0/4128350646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f874c100760 msgr2=0x7f874c072c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f874c100760 0x7f874c072c50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f873800e9c0 tx=0x7f873800ecd0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 -- 172.21.15.39:0/4128350646 shutdown_connections 2024-05-24T11:47:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f873c0a1390 0x7f873c0a3850 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c105e60 0x7f874c070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f874c0ffda0 0x7f874c072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 --2- 172.21.15.39:0/4128350646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f874c100760 0x7f874c072c50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 -- 172.21.15.39:0/4128350646 >> 172.21.15.39:0/4128350646 conn(0x7f874c076f10 msgr2=0x7f874c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.869+0000 7f8753063700 1 -- 172.21.15.39:0/4128350646 shutdown_connections 2024-05-24T11:47:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:32.870+0000 7f8753063700 1 -- 172.21.15.39:0/4128350646 wait complete. 2024-05-24T11:47:34.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 490.0 s, 1078.8 tps, lat 92.143 ms stddev 31.938 2024-05-24T11:47:34.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:33 smithi053 ceph-mon[47448]: pgmap v290: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 18 MiB/s wr, 1.19k op/s 2024-05-24T11:47:34.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:33 smithi137 ceph-mon[47907]: pgmap v290: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 18 MiB/s wr, 1.19k op/s 2024-05-24T11:47:34.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:33 smithi039 ceph-mon[42919]: pgmap v290: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 18 MiB/s wr, 1.19k op/s 2024-05-24T11:47:36.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:35 smithi053 ceph-mon[47448]: pgmap v291: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2024-05-24T11:47:36.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:35 smithi137 ceph-mon[47907]: pgmap v291: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2024-05-24T11:47:36.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:35 smithi039 ceph-mon[42919]: pgmap v291: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2024-05-24T11:47:38.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:37 smithi053 ceph-mon[47448]: pgmap v292: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 17 MiB/s wr, 1.11k op/s 2024-05-24T11:47:38.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:37 smithi137 ceph-mon[47907]: pgmap v292: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 17 MiB/s wr, 1.11k op/s 2024-05-24T11:47:38.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:37 smithi039 ceph-mon[42919]: pgmap v292: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 17 MiB/s wr, 1.11k op/s 2024-05-24T11:47:39.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 495.0 s, 1058.8 tps, lat 94.937 ms stddev 30.303 2024-05-24T11:47:40.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:39 smithi053 ceph-mon[47448]: pgmap v293: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2024-05-24T11:47:40.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:39 smithi137 ceph-mon[47907]: pgmap v293: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2024-05-24T11:47:40.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:39 smithi039 ceph-mon[42919]: pgmap v293: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2024-05-24T11:47:42.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:41 smithi053 ceph-mon[47448]: pgmap v294: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-05-24T11:47:42.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:41 smithi137 ceph-mon[47907]: pgmap v294: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-05-24T11:47:42.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:41 smithi039 ceph-mon[42919]: pgmap v294: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-05-24T11:47:44.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 500.0 s, 1004.6 tps, lat 99.667 ms stddev 30.088 2024-05-24T11:47:44.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:43 smithi137 ceph-mon[47907]: pgmap v295: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-05-24T11:47:44.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:43 smithi039 ceph-mon[42919]: pgmap v295: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-05-24T11:47:44.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:43 smithi053 ceph-mon[47448]: pgmap v295: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-05-24T11:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.046+0000 7f5bfc92d700 1 -- 172.21.15.39:0/226800337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 msgr2=0x7f5bf8108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.046+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/226800337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 0x7f5bf8108450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000b0a0 tx=0x7f5bf000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.046+0000 7f5bfc92d700 1 -- 172.21.15.39:0/226800337 shutdown_connections 2024-05-24T11:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.046+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/226800337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 0x7f5bf8108450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.046+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/226800337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bf810f630 0x7f5bf810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.046+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/226800337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bf810d440 0x7f5bf810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.046+0000 7f5bfc92d700 1 -- 172.21.15.39:0/226800337 >> 172.21.15.39:0/226800337 conn(0x7f5bf806bf80 msgr2=0x7f5bf806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.046+0000 7f5bfc92d700 1 -- 172.21.15.39:0/226800337 shutdown_connections 2024-05-24T11:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.046+0000 7f5bfc92d700 1 -- 172.21.15.39:0/226800337 wait complete. 2024-05-24T11:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.047+0000 7f5bfc92d700 1 Processor -- start 2024-05-24T11:47:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.055+0000 7f5bfc92d700 1 -- start start 2024-05-24T11:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.055+0000 7f5bfc92d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 0x7f5bf8078db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.055+0000 7f5bfc92d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bf810d440 0x7f5bf8079310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.055+0000 7f5bfc92d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bf810f630 0x7f5bf8073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.055+0000 7f5bfc92d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf8074750 con 0x7f5bf810d440 2024-05-24T11:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.055+0000 7f5bfc92d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80748c0 con 0x7f5bf810f630 2024-05-24T11:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.055+0000 7f5bfc92d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf8074a80 con 0x7f5bf8107fa0 2024-05-24T11:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.055+0000 7f5bf659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 0x7f5bf8078db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 0x7f5bf8078db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:60318/0 (socket says 172.21.15.39:60318) 2024-05-24T11:47:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf659c700 1 -- 172.21.15.39:0/432968089 learned_addr learned my addr 172.21.15.39:0/432968089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bf810d440 0x7f5bf8079310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bf810f630 0x7f5bf8073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf659c700 1 -- 172.21.15.39:0/432968089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bf810f630 msgr2=0x7f5bf8073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf659c700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bf810f630 0x7f5bf8073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf659c700 1 -- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bf810d440 msgr2=0x7f5bf8079310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf659c700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bf810d440 0x7f5bf8079310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf659c700 1 -- 172.21.15.39:0/432968089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0009d00 con 0x7f5bf8107fa0 2024-05-24T11:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bf810d440 0x7f5bf8079310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bf659c700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 0x7f5bf8078db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5be000e970 tx=0x7f5be000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bef7fe700 1 -- 172.21.15.39:0/432968089 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0009790 con 0x7f5bf8107fa0 2024-05-24T11:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bef7fe700 1 -- 172.21.15.39:0/432968089 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5be000eec0 con 0x7f5bf8107fa0 2024-05-24T11:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bef7fe700 1 -- 172.21.15.39:0/432968089 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0003c10 con 0x7f5bf8107fa0 2024-05-24T11:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bfc92d700 1 -- 172.21.15.39:0/432968089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf8074d60 con 0x7f5bf8107fa0 2024-05-24T11:47:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.056+0000 7f5bfc92d700 1 -- 172.21.15.39:0/432968089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf8080e10 con 0x7f5bf8107fa0 2024-05-24T11:47:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.059+0000 7f5bef7fe700 1 -- 172.21.15.39:0/432968089 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5be0018430 con 0x7f5bf8107fa0 2024-05-24T11:47:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.060+0000 7f5bef7fe700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5be40a1310 0x7f5be40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.060+0000 7f5bef7fe700 1 -- 172.21.15.39:0/432968089 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5be0014070 con 0x7f5bf8107fa0 2024-05-24T11:47:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.060+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5be40a1310 0x7f5be40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.061+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5be40a1310 0x7f5be40a37d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5be8006fd0 tx=0x7f5be8008040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.063+0000 7f5c08ae8b80 1 Processor -- start 2024-05-24T11:47:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5c08ae8b80 1 -- start start 2024-05-24T11:47:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea21d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea1bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea1c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dae9730f0 con 0x559daea21950 2024-05-24T11:47:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dae970ab0 con 0x559daea1c480 2024-05-24T11:47:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dae96c3a0 con 0x559daea22270 2024-05-24T11:47:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5bf6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea1c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5bf6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea1c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:40934/0 (socket says 172.21.15.39:40934) 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5bf6d9d700 1 -- 172.21.15.39:0/1685159799 learned_addr learned my addr 172.21.15.39:0/1685159799 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5bf659c700 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea21d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.064+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea1bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5bf6d9d700 1 -- 172.21.15.39:0/1685159799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 msgr2=0x559daea1bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea1bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5bf6d9d700 1 -- 172.21.15.39:0/1685159799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 msgr2=0x559daea21d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea21d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5bf6d9d700 1 -- 172.21.15.39:0/1685159799 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dae7ed350 con 0x559daea1c480 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea1bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:47:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea1c8a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000b0a0 tx=0x7f5bf00095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c99583dfc7bfd61 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5becff9700 1 -- 172.21.15.39:0/1685159799 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000e040 con 0x559daea1c480 2024-05-24T11:47:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5becff9700 1 -- 172.21.15.39:0/1685159799 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bf0007e50 con 0x559daea1c480 2024-05-24T11:47:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.065+0000 7f5becff9700 1 -- 172.21.15.39:0/1685159799 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0003d30 con 0x559daea1c480 2024-05-24T11:47:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.069+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1685159799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 msgr2=0x559daea1c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.069+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea1c8a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000b0a0 tx=0x7f5bf00095a0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.069+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1685159799 shutdown_connections 2024-05-24T11:47:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.069+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea1c8a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.069+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea21d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.069+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/1685159799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea1bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.069+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1685159799 >> 172.21.15.39:0/1685159799 conn(0x559dae97a1b0 msgr2=0x559daea16840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.069+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1685159799 shutdown_connections 2024-05-24T11:47:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.070+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1685159799 wait complete. 2024-05-24T11:47:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.070+0000 7f5c08ae8b80 1 -- start start 2024-05-24T11:47:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.070+0000 7f5c08ae8b80 1 Processor -- start 2024-05-24T11:47:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.071+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 0x559daea281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.071+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.071+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 0x559daea25ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.071+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dae908500 con 0x559daea21950 2024-05-24T11:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.071+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dae9081a0 con 0x559daea22270 2024-05-24T11:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.071+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dae7ef640 con 0x559daea1c480 2024-05-24T11:47:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.071+0000 7f5c08ae8b80 10 client.0 _ll_register_callbacks cb 0x559dae911ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.072+0000 7f5bf5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.072+0000 7f5bf5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea254b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47578/0 (socket says 172.21.15.39:47578) 2024-05-24T11:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.072+0000 7f5bf5d9b700 1 -- 172.21.15.39:0/778037178 learned_addr learned my addr 172.21.15.39:0/778037178 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.072+0000 7f5bf659c700 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 0x559daea281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.072+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 0x559daea25ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.073+0000 7f5bf5d9b700 1 -- 172.21.15.39:0/778037178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 msgr2=0x559daea281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.073+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 0x559daea281e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.073+0000 7f5bf5d9b700 1 -- 172.21.15.39:0/778037178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 msgr2=0x559daea25ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.073+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 0x559daea25ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.073+0000 7f5bf5d9b700 1 -- 172.21.15.39:0/778037178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be800c650 con 0x559daea21950 2024-05-24T11:47:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.073+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea254b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5be800cd20 tx=0x7f5be8004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abb2af5e118a6f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.073+0000 7f5bddffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:47:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.074+0000 7f5bddffb700 1 -- 172.21.15.39:0/778037178 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8013e10 con 0x559daea21950 2024-05-24T11:47:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.074+0000 7f5bddffb700 1 -- 172.21.15.39:0/778037178 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5be801f440 con 0x559daea21950 2024-05-24T11:47:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.074+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/778037178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dae8c5210 con 0x559daea21950 2024-05-24T11:47:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5bddffb700 1 -- 172.21.15.39:0/778037178 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be801f600 con 0x559daea21950 2024-05-24T11:47:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5c08ae8b80 10 client.16071 fetch_fsmap learned FSMap version 29 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/778037178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f5bf810b0e0 con 0x559daea21950 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5bddffb700 1 -- 172.21.15.39:0/778037178 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559dae8c5210 con 0x559daea21950 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5bddffb700 1 -- 172.21.15.39:0/778037178 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5be8027830 con 0x559daea21950 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5c08ae8b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5c08ae8b80 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5c08ae8b80 20 client.16071 populate_metadata read hostname 'smithi039' 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559daea46d30 0x559daea49150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5c08ae8b80 4 client.16071 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.075+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/778037178 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dae9eb7b0 con 0x559daea46d30 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.076+0000 7f5bf659c700 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559daea46d30 0x559daea49150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.076+0000 7f5bddffb700 10 client.16071 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.076+0000 7f5bf659c700 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559daea46d30 0x559daea49150 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e3f2fe8d3c9de18 server_cookie=e168a8e1b6b0726 in_seq=0 out_seq=0 2024-05-24T11:47:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.077+0000 7f5bddffb700 1 -- 172.21.15.39:0/778037178 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559dae9eb7b0 con 0x559daea46d30 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.077+0000 7f5bddffb700 10 client.16071 handle_command_reply: tid=0 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.077+0000 7f5c08ae8b80 1 client.16071 shutdown 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.077+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/778037178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 msgr2=0x559daea254b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.077+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea254b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5be800cd20 tx=0x7f5be8004800 comp rx=0 tx=0).stop 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/778037178 shutdown_connections 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 0x559daea281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559daea46d30 0x559daea49150 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 0x559daea25ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/778037178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea254b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/778037178 >> 172.21.15.39:0/778037178 conn(0x559dae97a1b0 msgr2=0x559daea1af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5bddffb700 0 client.16071 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/778037178 shutdown_connections 2024-05-24T11:47:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/778037178 wait complete. 2024-05-24T11:47:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.078+0000 7f5c08ae8b80 20 client.16071 trim_cache size 0 max 16384 2024-05-24T11:47:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.100+0000 7f5c08ae8b80 1 Processor -- start 2024-05-24T11:47:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5c08ae8b80 1 -- start start 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 0x559daea21d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea16910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 0x559daea27d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559daea260e0 con 0x559daea21950 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559daea263c0 con 0x559daea22270 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559daea21580 con 0x559daea1c480 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5bf659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 0x559daea21d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5bf659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 0x559daea21d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:60352/0 (socket says 172.21.15.39:60352) 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5bf659c700 1 -- 172.21.15.39:0/1677157258 learned_addr learned my addr 172.21.15.39:0/1677157258 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5bf6d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 0x559daea27d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.101+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/1677157258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea16910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.102+0000 7f5bf6d9d700 1 -- 172.21.15.39:0/1677157258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 msgr2=0x559daea21d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.102+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/1677157258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 0x559daea21d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.102+0000 7f5bf6d9d700 1 -- 172.21.15.39:0/1677157258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 msgr2=0x559daea16910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.102+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/1677157258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea16910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.102+0000 7f5bf6d9d700 1 -- 172.21.15.39:0/1677157258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dae799b00 con 0x559daea22270 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.102+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/1677157258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 0x559daea27d10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000b070 tx=0x7f5bf0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb161448e0eecb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.102+0000 7f5bed7fa700 1 -- 172.21.15.39:0/1677157258 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000e040 con 0x559daea22270 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.102+0000 7f5bed7fa700 1 -- 172.21.15.39:0/1677157258 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bf00074a0 con 0x559daea22270 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.102+0000 7f5bed7fa700 1 -- 172.21.15.39:0/1677157258 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0003e00 con 0x559daea22270 2024-05-24T11:47:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1677157258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 msgr2=0x559daea27d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/1677157258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 0x559daea27d10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000b070 tx=0x7f5bf0000f40 comp rx=0 tx=0).stop 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1677157258 shutdown_connections 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/1677157258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea1c480 0x559daea21d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/1677157258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea22270 0x559daea27d10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/1677157258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea16910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1677157258 >> 172.21.15.39:0/1677157258 conn(0x559dae97a1b0 msgr2=0x559daea12e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1677157258 shutdown_connections 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/1677157258 wait complete. 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.103+0000 7f5c08ae8b80 1 -- start start 2024-05-24T11:47:46.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5c08ae8b80 1 Processor -- start 2024-05-24T11:47:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea2c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea2ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5c08ae8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea29f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559daea2a560 con 0x559daea21950 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559daea2a6a0 con 0x559daea1c480 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5c08ae8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559daea2a860 con 0x559daea22270 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5c08ae8b80 10 client.0 _ll_register_callbacks cb 0x559dae911ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5bf659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea2c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5bf659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea2c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:40948/0 (socket says 172.21.15.39:40948) 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5bf659c700 1 -- 172.21.15.39:0/2080371860 learned_addr learned my addr 172.21.15.39:0/2080371860 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea2ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.104+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea29f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bf659c700 1 -- 172.21.15.39:0/2080371860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 msgr2=0x559daea29f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bf659c700 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea29f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bf659c700 1 -- 172.21.15.39:0/2080371860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 msgr2=0x559daea2ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bf659c700 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea2ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bf659c700 1 -- 172.21.15.39:0/2080371860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dae7ed350 con 0x559daea1c480 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bf6d9d700 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea29f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bf659c700 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea2c950 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5be0005950 tx=0x7f5be00972c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13a295e6dfb4df91 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bddffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bddffb700 1 -- 172.21.15.39:0/2080371860 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be00a3350 con 0x559daea1c480 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bddffb700 1 -- 172.21.15.39:0/2080371860 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5be00975e0 con 0x559daea1c480 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/2080371860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bf0007e50 con 0x559daea1c480 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5bddffb700 1 -- 172.21.15.39:0/2080371860 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be00977a0 con 0x559daea1c480 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5c08ae8b80 10 client.25784 fetch_fsmap learned FSMap version 29 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.105+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/2080371860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559dae932500 con 0x559daea1c480 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.106+0000 7f5bddffb700 1 -- 172.21.15.39:0/2080371860 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bf0007e50 con 0x559daea1c480 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.106+0000 7f5bddffb700 1 -- 172.21.15.39:0/2080371860 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5be009a3c0 con 0x559daea1c480 2024-05-24T11:47:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.107+0000 7f5c08ae8b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:47:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.107+0000 7f5c08ae8b80 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:47:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.107+0000 7f5c08ae8b80 20 client.25784 populate_metadata read hostname 'smithi039' 2024-05-24T11:47:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.107+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559daea9d4c0 0x559daea9f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:47:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.107+0000 7f5c08ae8b80 4 client.25784 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:47:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.107+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/2080371860 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559dae964e60 con 0x559daea9d4c0 2024-05-24T11:47:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.107+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559daea9d4c0 0x559daea9f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:47:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.108+0000 7f5bf5d9b700 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559daea9d4c0 0x559daea9f8e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f1e134be2bae15 server_cookie=9cb5cff0d5f8daf4 in_seq=0 out_seq=0 2024-05-24T11:47:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:46.108+0000 7f5bddffb700 10 client.25784 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:46.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:45 smithi039 ceph-mon[42919]: pgmap v296: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2024-05-24T11:47:46.493 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:45 smithi053 ceph-mon[47448]: pgmap v296: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2024-05-24T11:47:46.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:45 smithi137 ceph-mon[47907]: pgmap v296: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 22 MiB/s wr, 1.71k op/s 2024-05-24T11:47:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5bddffb700 1 -- 172.21.15.39:0/2080371860 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559dae964e60 con 0x559daea9d4c0 2024-05-24T11:47:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5bddffb700 10 client.25784 handle_command_reply: tid=0 2024-05-24T11:47:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 client.25784 shutdown 2024-05-24T11:47:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/2080371860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 msgr2=0x559daea2c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea2c950 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5be0005950 tx=0x7f5be00972c0 comp rx=0 tx=0).stop 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/2080371860 shutdown_connections 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559daea22270 0x559daea29f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559daea9d4c0 0x559daea9f8e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559daea1c480 0x559daea2c950 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5bddffb700 0 client.25784 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 --2- 172.21.15.39:0/2080371860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559daea21950 0x559daea2ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/2080371860 >> 172.21.15.39:0/2080371860 conn(0x559dae97a1b0 msgr2=0x559daea25400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/2080371860 shutdown_connections 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 1 -- 172.21.15.39:0/2080371860 wait complete. 2024-05-24T11:47:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.933+0000 7f5c08ae8b80 20 client.25784 trim_cache size 0 max 16384 2024-05-24T11:47:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:47:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:47:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:47:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:47:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 -- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5be40a1310 msgr2=0x7f5be40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5be40a1310 0x7f5be40a37d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5be8006fd0 tx=0x7f5be8008040 comp rx=0 tx=0).stop 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 -- 172.21.15.39:0/432968089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 msgr2=0x7f5bf8078db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 0x7f5bf8078db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5be000e970 tx=0x7f5be000ec80 comp rx=0 tx=0).stop 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 -- 172.21.15.39:0/432968089 shutdown_connections 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bf8107fa0 0x7f5bf8078db0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5be40a1310 0x7f5be40a37d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bf810f630 0x7f5bf8073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 --2- 172.21.15.39:0/432968089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bf810d440 0x7f5bf8079310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 -- 172.21.15.39:0/432968089 >> 172.21.15.39:0/432968089 conn(0x7f5bf806bf80 msgr2=0x7f5bf80719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 -- 172.21.15.39:0/432968089 shutdown_connections 2024-05-24T11:47:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:47:47.934+0000 7f5bfc92d700 1 -- 172.21.15.39:0/432968089 wait complete. 2024-05-24T11:47:48.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:47 smithi039 ceph-mon[42919]: pgmap v297: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-05-24T11:47:48.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:47 smithi053 ceph-mon[47448]: pgmap v297: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-05-24T11:47:48.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:47 smithi137 ceph-mon[47907]: pgmap v297: 97 pgs: 97 active+clean; 5.6 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-05-24T11:47:49.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 505.0 s, 982.0 tps, lat 101.832 ms stddev 63.076 2024-05-24T11:47:50.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:50 smithi039 ceph-mon[42919]: pgmap v298: 97 pgs: 97 active+clean; 5.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 22 MiB/s wr, 1.77k op/s 2024-05-24T11:47:50.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:50 smithi053 ceph-mon[47448]: pgmap v298: 97 pgs: 97 active+clean; 5.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 22 MiB/s wr, 1.77k op/s 2024-05-24T11:47:50.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:50 smithi137 ceph-mon[47907]: pgmap v298: 97 pgs: 97 active+clean; 5.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 22 MiB/s wr, 1.77k op/s 2024-05-24T11:47:52.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:52 smithi039 ceph-mon[42919]: pgmap v299: 97 pgs: 97 active+clean; 5.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:47:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:52 smithi053 ceph-mon[47448]: pgmap v299: 97 pgs: 97 active+clean; 5.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:47:52.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:52 smithi137 ceph-mon[47907]: pgmap v299: 97 pgs: 97 active+clean; 5.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:47:54.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 510.0 s, 1002.4 tps, lat 98.573 ms stddev 43.891 2024-05-24T11:47:54.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:54 smithi039 ceph-mon[42919]: pgmap v300: 97 pgs: 97 active+clean; 5.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:47:54.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:54 smithi053 ceph-mon[47448]: pgmap v300: 97 pgs: 97 active+clean; 5.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:47:54.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:54 smithi137 ceph-mon[47907]: pgmap v300: 97 pgs: 97 active+clean; 5.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2024-05-24T11:47:56.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:56 smithi039 ceph-mon[42919]: pgmap v301: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 21 MiB/s wr, 1.73k op/s 2024-05-24T11:47:56.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:56 smithi053 ceph-mon[47448]: pgmap v301: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 21 MiB/s wr, 1.73k op/s 2024-05-24T11:47:56.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:56 smithi137 ceph-mon[47907]: pgmap v301: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 21 MiB/s wr, 1.73k op/s 2024-05-24T11:47:58.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:47:58 smithi039 ceph-mon[42919]: pgmap v302: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:47:58.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:47:58 smithi053 ceph-mon[47448]: pgmap v302: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:47:58.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:47:58 smithi137 ceph-mon[47907]: pgmap v302: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:47:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 515.0 s, 967.0 tps, lat 104.768 ms stddev 140.631 2024-05-24T11:48:00.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:00 smithi039 ceph-mon[42919]: pgmap v303: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 22 MiB/s wr, 1.81k op/s 2024-05-24T11:48:00.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:00 smithi053 ceph-mon[47448]: pgmap v303: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 22 MiB/s wr, 1.81k op/s 2024-05-24T11:48:00.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:00 smithi137 ceph-mon[47907]: pgmap v303: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 22 MiB/s wr, 1.81k op/s 2024-05-24T11:48:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.124+0000 7fbb52f9c700 1 -- 172.21.15.39:0/1614356072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb4c107fa0 msgr2=0x7fbb4c108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.124+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/1614356072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb4c107fa0 0x7fbb4c108450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb4401c0d0 tx=0x7fbb4401c3e0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.124+0000 7fbb52f9c700 1 -- 172.21.15.39:0/1614356072 shutdown_connections 2024-05-24T11:48:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.124+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/1614356072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb4c107fa0 0x7fbb4c108450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.124+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/1614356072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c10f630 0x7fbb4c10fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.124+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/1614356072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c10d440 0x7fbb4c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.124+0000 7fbb52f9c700 1 -- 172.21.15.39:0/1614356072 >> 172.21.15.39:0/1614356072 conn(0x7fbb4c06bf80 msgr2=0x7fbb4c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.124+0000 7fbb52f9c700 1 -- 172.21.15.39:0/1614356072 shutdown_connections 2024-05-24T11:48:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.124+0000 7fbb52f9c700 1 -- 172.21.15.39:0/1614356072 wait complete. 2024-05-24T11:48:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.125+0000 7fbb52f9c700 1 Processor -- start 2024-05-24T11:48:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.125+0000 7fbb52f9c700 1 -- start start 2024-05-24T11:48:01.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb52f9c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c107fa0 0x7fbb4c1a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb52f9c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb4c10d440 0x7fbb4c1a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb52f9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c10f630 0x7fbb4c19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb52f9c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c19e080 con 0x7fbb4c107fa0 2024-05-24T11:48:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb52f9c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c19e1f0 con 0x7fbb4c10f630 2024-05-24T11:48:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb52f9c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c19e3b0 con 0x7fbb4c10d440 2024-05-24T11:48:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb50d38700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c107fa0 0x7fbb4c1a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb50d38700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c107fa0 0x7fbb4c1a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49560/0 (socket says 172.21.15.39:49560) 2024-05-24T11:48:01.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb50d38700 1 -- 172.21.15.39:0/832013175 learned_addr learned my addr 172.21.15.39:0/832013175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb50d38700 1 -- 172.21.15.39:0/832013175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb4c10d440 msgr2=0x7fbb4c1a3e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-24T11:48:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb50d38700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb4c10d440 0x7fbb4c1a3e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb50d38700 1 -- 172.21.15.39:0/832013175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c10f630 msgr2=0x7fbb4c19da20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:48:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.126+0000 7fbb50d38700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c10f630 0x7fbb4c19da20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.127+0000 7fbb50d38700 1 -- 172.21.15.39:0/832013175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4401ad10 con 0x7fbb4c107fa0 2024-05-24T11:48:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.127+0000 7fbb50d38700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c107fa0 0x7fbb4c1a38f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000b500 tx=0x7fbb4000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.130+0000 7fbb49ffb700 1 -- 172.21.15.39:0/832013175 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40011830 con 0x7fbb4c107fa0 2024-05-24T11:48:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.130+0000 7fbb49ffb700 1 -- 172.21.15.39:0/832013175 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb40011e20 con 0x7fbb4c107fa0 2024-05-24T11:48:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.130+0000 7fbb52f9c700 1 -- 172.21.15.39:0/832013175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c19e690 con 0x7fbb4c107fa0 2024-05-24T11:48:01.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.130+0000 7fbb52f9c700 1 -- 172.21.15.39:0/832013175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c1aa8e0 con 0x7fbb4c107fa0 2024-05-24T11:48:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.130+0000 7fbb49ffb700 1 -- 172.21.15.39:0/832013175 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40010e30 con 0x7fbb4c107fa0 2024-05-24T11:48:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.133+0000 7fbb49ffb700 1 -- 172.21.15.39:0/832013175 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbb4001a440 con 0x7fbb4c107fa0 2024-05-24T11:48:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.134+0000 7fbb49ffb700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbb3c0a13e0 0x7fbb3c0a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.134+0000 7fbb49ffb700 1 -- 172.21.15.39:0/832013175 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbb40010380 con 0x7fbb4c107fa0 2024-05-24T11:48:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.135+0000 7fbb4bfff700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbb3c0a13e0 0x7fbb3c0a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.136+0000 7fbb4bfff700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbb3c0a13e0 0x7fbb3c0a38a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c19ebe0 tx=0x7fbb38006cb0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.140+0000 7fbb5f157b80 1 Processor -- start 2024-05-24T11:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.140+0000 7fbb5f157b80 1 -- start start 2024-05-24T11:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.140+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.140+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 0x55c1a4244f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.140+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 0x55c1a42458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.140+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a419c0f0 con 0x55c1a4245480 2024-05-24T11:48:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.140+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a4199ab0 con 0x55c1a424a950 2024-05-24T11:48:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.140+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a41953a0 con 0x55c1a424b270 2024-05-24T11:48:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.141+0000 7fbb51539700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 0x55c1a42458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.141+0000 7fbb51539700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 0x55c1a42458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49574/0 (socket says 172.21.15.39:49574) 2024-05-24T11:48:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.141+0000 7fbb51539700 1 -- 172.21.15.39:0/1386312155 learned_addr learned my addr 172.21.15.39:0/1386312155 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.141+0000 7fbb50d38700 1 --2- 172.21.15.39:0/1386312155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.141+0000 7fbb51539700 1 -- 172.21.15.39:0/1386312155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 msgr2=0x55c1a4244f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:48:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.141+0000 7fbb51539700 1 --2- 172.21.15.39:0/1386312155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 0x55c1a4244f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.141+0000 7fbb51539700 1 -- 172.21.15.39:0/1386312155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 msgr2=0x55c1a424ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.141+0000 7fbb51539700 1 --2- 172.21.15.39:0/1386312155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.141+0000 7fbb51539700 1 -- 172.21.15.39:0/1386312155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1a4016350 con 0x55c1a4245480 2024-05-24T11:48:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.142+0000 7fbb51539700 1 --2- 172.21.15.39:0/1386312155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 0x55c1a42458a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbb4401cdc0 tx=0x7fbb4401cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02bbb5e6d8898c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.142+0000 7fbb36ffd700 1 -- 172.21.15.39:0/1386312155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4401f040 con 0x55c1a4245480 2024-05-24T11:48:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.142+0000 7fbb36ffd700 1 -- 172.21.15.39:0/1386312155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb44019e00 con 0x55c1a4245480 2024-05-24T11:48:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.142+0000 7fbb36ffd700 1 -- 172.21.15.39:0/1386312155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44003d30 con 0x55c1a4245480 2024-05-24T11:48:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.143+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1386312155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 msgr2=0x55c1a42458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.143+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1386312155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 0x55c1a42458a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbb4401cdc0 tx=0x7fbb4401cdf0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.143+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1386312155 shutdown_connections 2024-05-24T11:48:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.143+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1386312155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 0x55c1a42458a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.143+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1386312155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.143+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1386312155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 0x55c1a4244f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.143+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1386312155 >> 172.21.15.39:0/1386312155 conn(0x55c1a41a31b0 msgr2=0x55c1a423f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.144+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1386312155 shutdown_connections 2024-05-24T11:48:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.144+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1386312155 wait complete. 2024-05-24T11:48:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.144+0000 7fbb5f157b80 1 -- start start 2024-05-24T11:48:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.144+0000 7fbb5f157b80 1 Processor -- start 2024-05-24T11:48:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.145+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 0x55c1a4251120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.145+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.145+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a424b270 0x55c1a424e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.145+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a4131500 con 0x55c1a424b270 2024-05-24T11:48:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.145+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a41311a0 con 0x55c1a424a950 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.145+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a4018640 con 0x55c1a4245480 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.145+0000 7fbb5f157b80 10 client.0 _ll_register_callbacks cb 0x55c1a413aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb4bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb4bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:60550/0 (socket says 172.21.15.39:60550) 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb4bfff700 1 -- 172.21.15.39:0/1465847775 learned_addr learned my addr 172.21.15.39:0/1465847775 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb50d38700 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 0x55c1a4251120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb51539700 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a424b270 0x55c1a424e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb4bfff700 1 -- 172.21.15.39:0/1465847775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 msgr2=0x55c1a4251120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb4bfff700 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 0x55c1a4251120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb4bfff700 1 -- 172.21.15.39:0/1465847775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a424b270 msgr2=0x55c1a424e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb4bfff700 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a424b270 0x55c1a424e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb4bfff700 1 -- 172.21.15.39:0/1465847775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c10b0e0 con 0x55c1a424a950 2024-05-24T11:48:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb4bfff700 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424e410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbb38012430 tx=0x7fbb38004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a660adb310d78698 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.146+0000 7fbb2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:48:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.147+0000 7fbb2bfff700 1 -- 172.21.15.39:0/1465847775 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3801a070 con 0x55c1a424a950 2024-05-24T11:48:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.147+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1465847775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1a40ee210 con 0x55c1a424a950 2024-05-24T11:48:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.147+0000 7fbb2bfff700 1 -- 172.21.15.39:0/1465847775 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbb38012c40 con 0x55c1a424a950 2024-05-24T11:48:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.147+0000 7fbb2bfff700 1 -- 172.21.15.39:0/1465847775 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb380152e0 con 0x55c1a424a950 2024-05-24T11:48:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.147+0000 7fbb5f157b80 10 client.25790 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.147+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1465847775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c1a3fc2b00 con 0x55c1a424a950 2024-05-24T11:48:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.147+0000 7fbb2bfff700 1 -- 172.21.15.39:0/1465847775 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb3802b020 con 0x55c1a424a950 2024-05-24T11:48:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.148+0000 7fbb2bfff700 1 -- 172.21.15.39:0/1465847775 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbb380263a0 con 0x55c1a424a950 2024-05-24T11:48:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.148+0000 7fbb5f157b80 10 client.25790 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.149+0000 7fbb5f157b80 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.149+0000 7fbb5f157b80 20 client.25790 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.149+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c1a425ec30 0x55c1a4261050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.149+0000 7fbb5f157b80 4 client.25790 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:48:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.149+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1465847775 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1a42147b0 con 0x55c1a425ec30 2024-05-24T11:48:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.149+0000 7fbb50d38700 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c1a425ec30 0x55c1a4261050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.149+0000 7fbb50d38700 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c1a425ec30 0x55c1a4261050 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=475bf77b303aca59 server_cookie=cc0647bfb08b1d9f in_seq=0 out_seq=0 2024-05-24T11:48:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.149+0000 7fbb2bfff700 10 client.25790 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.150+0000 7fbb2bfff700 1 -- 172.21.15.39:0/1465847775 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1a42147b0 con 0x55c1a425ec30 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.150+0000 7fbb2bfff700 10 client.25790 handle_command_reply: tid=0 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.150+0000 7fbb5f157b80 1 client.25790 shutdown 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.150+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1465847775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 msgr2=0x55c1a424e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.150+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424e410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbb38012430 tx=0x7fbb38004750 comp rx=0 tx=0).stop 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.151+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1465847775 shutdown_connections 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.151+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 0x55c1a4251120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.151+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c1a425ec30 0x55c1a4261050 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.151+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a424b270 0x55c1a424e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.151+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/1465847775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a424e410 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.151+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1465847775 >> 172.21.15.39:0/1465847775 conn(0x55c1a41a31b0 msgr2=0x55c1a42494a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.151+0000 7fbb2bfff700 0 client.25790 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.153+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1465847775 shutdown_connections 2024-05-24T11:48:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.153+0000 7fbb5f157b80 1 -- 172.21.15.39:0/1465847775 wait complete. 2024-05-24T11:48:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.153+0000 7fbb5f157b80 20 client.25790 trim_cache size 0 max 16384 2024-05-24T11:48:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.175+0000 7fbb5f157b80 1 Processor -- start 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb5f157b80 1 -- start start 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 0x55c1a4241230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a424a950 0x55c1a423f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424b270 0x55c1a42659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a425b650 con 0x55c1a424a950 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a4237a40 con 0x55c1a424b270 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a425c260 con 0x55c1a4245480 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb51539700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424b270 0x55c1a42659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb51539700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424b270 0x55c1a42659b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:60554/0 (socket says 172.21.15.39:60554) 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb51539700 1 -- 172.21.15.39:0/3879508513 learned_addr learned my addr 172.21.15.39:0/3879508513 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.176+0000 7fbb50d38700 1 --2- 172.21.15.39:0/3879508513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 0x55c1a4241230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.177+0000 7fbb50d38700 1 -- 172.21.15.39:0/3879508513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424b270 msgr2=0x55c1a42659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.177+0000 7fbb50d38700 1 --2- 172.21.15.39:0/3879508513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424b270 0x55c1a42659b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.177+0000 7fbb50d38700 1 -- 172.21.15.39:0/3879508513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a424a950 msgr2=0x55c1a423f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.177+0000 7fbb50d38700 1 --2- 172.21.15.39:0/3879508513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a424a950 0x55c1a423f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.177+0000 7fbb50d38700 1 -- 172.21.15.39:0/3879508513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1a415b500 con 0x55c1a4245480 2024-05-24T11:48:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.177+0000 7fbb50d38700 1 --2- 172.21.15.39:0/3879508513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 0x55c1a4241230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbb400ca7c0 tx=0x7fbb40004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d04e54e3cc189c11 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.177+0000 7fbb377fe700 1 -- 172.21.15.39:0/3879508513 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb400ba0b0 con 0x55c1a4245480 2024-05-24T11:48:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.178+0000 7fbb377fe700 1 -- 172.21.15.39:0/3879508513 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb400b2070 con 0x55c1a4245480 2024-05-24T11:48:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.178+0000 7fbb377fe700 1 -- 172.21.15.39:0/3879508513 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4009e120 con 0x55c1a4245480 2024-05-24T11:48:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.178+0000 7fbb5f157b80 1 -- 172.21.15.39:0/3879508513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 msgr2=0x55c1a4241230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.178+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/3879508513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 0x55c1a4241230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbb400ca7c0 tx=0x7fbb40004860 comp rx=0 tx=0).stop 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.179+0000 7fbb5f157b80 1 -- 172.21.15.39:0/3879508513 shutdown_connections 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.179+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/3879508513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a4245480 0x55c1a4241230 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.179+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/3879508513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424b270 0x55c1a42659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.179+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/3879508513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a424a950 0x55c1a423f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.179+0000 7fbb5f157b80 1 -- 172.21.15.39:0/3879508513 >> 172.21.15.39:0/3879508513 conn(0x55c1a41a31b0 msgr2=0x55c1a423be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.179+0000 7fbb5f157b80 1 -- 172.21.15.39:0/3879508513 shutdown_connections 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.179+0000 7fbb5f157b80 1 -- 172.21.15.39:0/3879508513 wait complete. 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.179+0000 7fbb5f157b80 1 -- start start 2024-05-24T11:48:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.179+0000 7fbb5f157b80 1 Processor -- start 2024-05-24T11:48:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 0x55c1a4253e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a4254360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb5f157b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 0x55c1a42b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a42b4670 con 0x55c1a4245480 2024-05-24T11:48:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a42b47b0 con 0x55c1a424a950 2024-05-24T11:48:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb5f157b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1a42b48f0 con 0x55c1a424b270 2024-05-24T11:48:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb5f157b80 10 client.0 _ll_register_callbacks cb 0x55c1a413aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb51539700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 0x55c1a42b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb51539700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 0x55c1a42b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:53036/0 (socket says 172.21.15.39:53036) 2024-05-24T11:48:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb51539700 1 -- 172.21.15.39:0/548790743 learned_addr learned my addr 172.21.15.39:0/548790743 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.180+0000 7fbb4bfff700 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a4254360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.181+0000 7fbb51539700 1 -- 172.21.15.39:0/548790743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 msgr2=0x55c1a4254360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.181+0000 7fbb51539700 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a4254360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.181+0000 7fbb51539700 1 -- 172.21.15.39:0/548790743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 msgr2=0x55c1a4253e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.181+0000 7fbb51539700 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 0x55c1a4253e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.181+0000 7fbb51539700 1 -- 172.21.15.39:0/548790743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c1aa8e0 con 0x55c1a424b270 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.181+0000 7fbb51539700 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 0x55c1a42b4130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb4401aa30 tx=0x7fbb44018720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286eb51fd6b02660 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.181+0000 7fbb2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.182+0000 7fbb2bfff700 1 -- 172.21.15.39:0/548790743 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4401f040 con 0x55c1a424b270 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.182+0000 7fbb2bfff700 1 -- 172.21.15.39:0/548790743 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbb44003c10 con 0x55c1a424b270 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.182+0000 7fbb5f157b80 1 -- 172.21.15.39:0/548790743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb44019e00 con 0x55c1a424b270 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.182+0000 7fbb2bfff700 1 -- 172.21.15.39:0/548790743 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb44018940 con 0x55c1a424b270 2024-05-24T11:48:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.182+0000 7fbb5f157b80 10 client.25717 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.182+0000 7fbb5f157b80 1 -- 172.21.15.39:0/548790743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c1a401ac70 con 0x55c1a424b270 2024-05-24T11:48:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb2bfff700 1 -- 172.21.15.39:0/548790743 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb44019e00 con 0x55c1a424b270 2024-05-24T11:48:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb2bfff700 1 -- 172.21.15.39:0/548790743 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbb4402a040 con 0x55c1a424b270 2024-05-24T11:48:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb5f157b80 10 client.25717 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb5f157b80 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb5f157b80 20 client.25717 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c1a42b5c20 0x55c1a42b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb5f157b80 4 client.25717 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:48:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb5f157b80 1 -- 172.21.15.39:0/548790743 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c1a418de60 con 0x55c1a42b5c20 2024-05-24T11:48:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb50d38700 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c1a42b5c20 0x55c1a42b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb2bfff700 10 client.25717 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:01.184+0000 7fbb50d38700 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c1a42b5c20 0x55c1a42b8010 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0d330dc8f1486c server_cookie=812bd5ddc51e7a91 in_seq=0 out_seq=0 2024-05-24T11:48:02.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:02 smithi039 ceph-mon[42919]: pgmap v304: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:48:02.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:02 smithi053 ceph-mon[47448]: pgmap v304: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:48:02.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:02 smithi137 ceph-mon[47907]: pgmap v304: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:48:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.932+0000 7fbb2bfff700 1 -- 172.21.15.39:0/548790743 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c1a418de60 con 0x55c1a42b5c20 2024-05-24T11:48:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.932+0000 7fbb2bfff700 10 client.25717 handle_command_reply: tid=0 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.932+0000 7fbb5f157b80 1 client.25717 shutdown 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 -- 172.21.15.39:0/548790743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 msgr2=0x55c1a42b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 0x55c1a42b4130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb4401aa30 tx=0x7fbb44018720 comp rx=0 tx=0).stop 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 -- 172.21.15.39:0/548790743 shutdown_connections 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a424b270 0x55c1a42b4130 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55c1a42b5c20 0x55c1a42b8010 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c1a4245480 0x55c1a4253e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 --2- 172.21.15.39:0/548790743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c1a424a950 0x55c1a4254360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 -- 172.21.15.39:0/548790743 >> 172.21.15.39:0/548790743 conn(0x55c1a41a31b0 msgr2=0x55c1a42659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb2bfff700 0 client.25717 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 -- 172.21.15.39:0/548790743 shutdown_connections 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 1 -- 172.21.15.39:0/548790743 wait complete. 2024-05-24T11:48:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.933+0000 7fbb5f157b80 20 client.25717 trim_cache size 0 max 16384 2024-05-24T11:48:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:48:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:48:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:48:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:48:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.934+0000 7fbb52f9c700 1 -- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbb3c0a13e0 msgr2=0x7fbb3c0a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.934+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbb3c0a13e0 0x7fbb3c0a38a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c19ebe0 tx=0x7fbb38006cb0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.934+0000 7fbb52f9c700 1 -- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c107fa0 msgr2=0x7fbb4c1a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.934+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c107fa0 0x7fbb4c1a38f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000b500 tx=0x7fbb4000b8c0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.935+0000 7fbb52f9c700 1 -- 172.21.15.39:0/832013175 shutdown_connections 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.935+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbb3c0a13e0 0x7fbb3c0a38a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.935+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c10f630 0x7fbb4c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.935+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c107fa0 0x7fbb4c1a38f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.935+0000 7fbb52f9c700 1 --2- 172.21.15.39:0/832013175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb4c10d440 0x7fbb4c1a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.935+0000 7fbb52f9c700 1 -- 172.21.15.39:0/832013175 >> 172.21.15.39:0/832013175 conn(0x7fbb4c06bf80 msgr2=0x7fbb4c0719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.935+0000 7fbb52f9c700 1 -- 172.21.15.39:0/832013175 shutdown_connections 2024-05-24T11:48:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:02.935+0000 7fbb52f9c700 1 -- 172.21.15.39:0/832013175 wait complete. 2024-05-24T11:48:04.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 520.0 s, 932.2 tps, lat 106.782 ms stddev 55.953 2024-05-24T11:48:04.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:04 smithi039 ceph-mon[42919]: pgmap v305: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:48:04.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:04 smithi039 ceph-mon[42919]: osdmap e88: 12 total, 12 up, 12 in 2024-05-24T11:48:04.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:04 smithi053 ceph-mon[47448]: pgmap v305: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:48:04.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:04 smithi053 ceph-mon[47448]: osdmap e88: 12 total, 12 up, 12 in 2024-05-24T11:48:04.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:04 smithi137 ceph-mon[47907]: pgmap v305: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:48:04.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:04 smithi137 ceph-mon[47907]: osdmap e88: 12 total, 12 up, 12 in 2024-05-24T11:48:06.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:06 smithi039 ceph-mon[42919]: pgmap v307: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-05-24T11:48:06.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:06 smithi053 ceph-mon[47448]: pgmap v307: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-05-24T11:48:06.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:06 smithi137 ceph-mon[47907]: pgmap v307: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-05-24T11:48:08.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:08 smithi039 ceph-mon[42919]: pgmap v308: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-05-24T11:48:08.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:08 smithi053 ceph-mon[47448]: pgmap v308: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-05-24T11:48:08.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:08 smithi137 ceph-mon[47907]: pgmap v308: 97 pgs: 19 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 43 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-05-24T11:48:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 525.0 s, 892.2 tps, lat 112.513 ms stddev 111.692 2024-05-24T11:48:10.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:10 smithi039 ceph-mon[42919]: pgmap v309: 97 pgs: 97 active+clean; 5.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:48:10.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:10 smithi039 ceph-mon[42919]: osdmap e89: 12 total, 12 up, 12 in 2024-05-24T11:48:10.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:10 smithi053 ceph-mon[47448]: pgmap v309: 97 pgs: 97 active+clean; 5.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:48:10.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:10 smithi053 ceph-mon[47448]: osdmap e89: 12 total, 12 up, 12 in 2024-05-24T11:48:10.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:10 smithi137 ceph-mon[47907]: pgmap v309: 97 pgs: 97 active+clean; 5.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:48:10.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:10 smithi137 ceph-mon[47907]: osdmap e89: 12 total, 12 up, 12 in 2024-05-24T11:48:12.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:12 smithi039 ceph-mon[42919]: pgmap v311: 97 pgs: 97 active+clean; 5.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 18 MiB/s wr, 1.60k op/s 2024-05-24T11:48:12.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:12 smithi053 ceph-mon[47448]: pgmap v311: 97 pgs: 97 active+clean; 5.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 18 MiB/s wr, 1.60k op/s 2024-05-24T11:48:12.499 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:12 smithi137 ceph-mon[47907]: pgmap v311: 97 pgs: 97 active+clean; 5.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 18 MiB/s wr, 1.60k op/s 2024-05-24T11:48:14.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 530.0 s, 960.4 tps, lat 104.017 ms stddev 33.034 2024-05-24T11:48:14.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:14 smithi053 ceph-mon[47448]: pgmap v312: 97 pgs: 97 active+clean; 5.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-05-24T11:48:14.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:14 smithi039 ceph-mon[42919]: pgmap v312: 97 pgs: 97 active+clean; 5.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-05-24T11:48:14.596 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:14 smithi137 ceph-mon[47907]: pgmap v312: 97 pgs: 97 active+clean; 5.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-05-24T11:48:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.110+0000 7f9d44ebf700 1 -- 172.21.15.39:0/4155660824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d40107500 msgr2=0x7f9d40107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.110+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/4155660824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d40107500 0x7f9d40107980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009990 tx=0x7f9d28009ca0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.110+0000 7f9d44ebf700 1 -- 172.21.15.39:0/4155660824 shutdown_connections 2024-05-24T11:48:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.110+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/4155660824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d40107ec0 0x7f9d40114380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.110+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/4155660824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d4010d5c0 0x7f9d4010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.110+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/4155660824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d40107500 0x7f9d40107980 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.110+0000 7f9d44ebf700 1 -- 172.21.15.39:0/4155660824 >> 172.21.15.39:0/4155660824 conn(0x7f9d40076dc0 msgr2=0x7f9d40105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.111+0000 7f9d44ebf700 1 -- 172.21.15.39:0/4155660824 shutdown_connections 2024-05-24T11:48:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.111+0000 7f9d44ebf700 1 -- 172.21.15.39:0/4155660824 wait complete. 2024-05-24T11:48:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.112+0000 7f9d44ebf700 1 Processor -- start 2024-05-24T11:48:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d44ebf700 1 -- start start 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d44ebf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40107500 0x7f9d4019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d44ebf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d40107ec0 0x7f9d4019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d44ebf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d401a2340 0x7f9d401a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d44ebf700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d401a2d30 con 0x7f9d40107500 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d44ebf700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d401a2ea0 con 0x7f9d40107ec0 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d44ebf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d401a3060 con 0x7f9d401a2340 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d40107ec0 0x7f9d4019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d40107ec0 0x7f9d4019df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:38710/0 (socket says 172.21.15.39:38710) 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d35d9b700 1 -- 172.21.15.39:0/1159939082 learned_addr learned my addr 172.21.15.39:0/1159939082 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.113+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d401a2340 0x7f9d401a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.114+0000 7f9d3e59c700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40107500 0x7f9d4019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.114+0000 7f9d3e59c700 1 -- 172.21.15.39:0/1159939082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d401a2340 msgr2=0x7f9d401a27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.114+0000 7f9d3e59c700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d401a2340 0x7f9d401a27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.114+0000 7f9d3e59c700 1 -- 172.21.15.39:0/1159939082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d40107ec0 msgr2=0x7f9d4019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.114+0000 7f9d3e59c700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d40107ec0 0x7f9d4019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.114+0000 7f9d3e59c700 1 -- 172.21.15.39:0/1159939082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d280095f0 con 0x7f9d40107500 2024-05-24T11:48:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.114+0000 7f9d3e59c700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40107500 0x7f9d4019da30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9d4006a030 tx=0x7f9d3000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.115+0000 7f9d37fff700 1 -- 172.21.15.39:0/1159939082 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3000be70 con 0x7f9d40107500 2024-05-24T11:48:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.115+0000 7f9d37fff700 1 -- 172.21.15.39:0/1159939082 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d30011ac0 con 0x7f9d40107500 2024-05-24T11:48:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.115+0000 7f9d37fff700 1 -- 172.21.15.39:0/1159939082 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30010aa0 con 0x7f9d40107500 2024-05-24T11:48:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.116+0000 7f9d44ebf700 1 -- 172.21.15.39:0/1159939082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d401aa5f0 con 0x7f9d40107500 2024-05-24T11:48:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.116+0000 7f9d44ebf700 1 -- 172.21.15.39:0/1159939082 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d401aab00 con 0x7f9d40107500 2024-05-24T11:48:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.120+0000 7f9d5107ab80 1 Processor -- start 2024-05-24T11:48:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d5107ab80 1 -- start start 2024-05-24T11:48:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff0950 0x560779ff0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff1270 0x560779feaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779feb480 0x560779feb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560779f420f0 con 0x560779ff0950 2024-05-24T11:48:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560779f3fab0 con 0x560779ff1270 2024-05-24T11:48:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560779f3b3a0 con 0x560779feb480 2024-05-24T11:48:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d3ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff0950 0x560779ff0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d3ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff0950 0x560779ff0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37634/0 (socket says 172.21.15.39:37634) 2024-05-24T11:48:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.123+0000 7f9d3ed9d700 1 -- 172.21.15.39:0/3089521012 learned_addr learned my addr 172.21.15.39:0/3089521012 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.124+0000 7f9d3ed9d700 1 -- 172.21.15.39:0/3089521012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779feb480 msgr2=0x560779feb8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:48:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.124+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/3089521012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779feb480 0x560779feb8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.124+0000 7f9d3ed9d700 1 -- 172.21.15.39:0/3089521012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff1270 msgr2=0x560779feaf40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:48:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.124+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/3089521012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff1270 0x560779feaf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.124+0000 7f9d3ed9d700 1 -- 172.21.15.39:0/3089521012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560779dbc350 con 0x560779ff0950 2024-05-24T11:48:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.124+0000 7f9d37fff700 1 -- 172.21.15.39:0/1159939082 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9d30011c30 con 0x7f9d40107500 2024-05-24T11:48:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.124+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/3089521012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff0950 0x560779ff0d30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009f40 tx=0x7f9d3800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504c85f35eababda server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.125+0000 7f9d37fff700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9d2c0a1310 0x7f9d2c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.125+0000 7f9d34d99700 1 -- 172.21.15.39:0/3089521012 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38011070 con 0x560779ff0950 2024-05-24T11:48:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.125+0000 7f9d34d99700 1 -- 172.21.15.39:0/3089521012 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d380158a0 con 0x560779ff0950 2024-05-24T11:48:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.125+0000 7f9d34d99700 1 -- 172.21.15.39:0/3089521012 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800a870 con 0x560779ff0950 2024-05-24T11:48:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.125+0000 7f9d37fff700 1 -- 172.21.15.39:0/1159939082 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9d300cc1b0 con 0x7f9d40107500 2024-05-24T11:48:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.125+0000 7f9d3e59c700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9d2c0a1310 0x7f9d2c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.126+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3089521012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff0950 msgr2=0x560779ff0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.126+0000 7f9d3e59c700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9d2c0a1310 0x7f9d2c0a37d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x560779fe6790 tx=0x7f9d30099260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.126+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/3089521012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff0950 0x560779ff0d30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009f40 tx=0x7f9d3800c6e0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.126+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3089521012 shutdown_connections 2024-05-24T11:48:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.126+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/3089521012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779feb480 0x560779feb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.126+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/3089521012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff0950 0x560779ff0d30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.126+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/3089521012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff1270 0x560779feaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.126+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3089521012 >> 172.21.15.39:0/3089521012 conn(0x560779f491b0 msgr2=0x560779fe5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.128+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3089521012 shutdown_connections 2024-05-24T11:48:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.128+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3089521012 wait complete. 2024-05-24T11:48:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.129+0000 7f9d5107ab80 1 -- start start 2024-05-24T11:48:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.129+0000 7f9d5107ab80 1 Processor -- start 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.129+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779feb480 0x560779ff45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.129+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779ff4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.129+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff1270 0x560779ff5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.129+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560779ed7500 con 0x560779ff1270 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.129+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560779ed71a0 con 0x560779feb480 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.129+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560779dbe640 con 0x560779ff0950 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.129+0000 7f9d5107ab80 10 client.0 _ll_register_callbacks cb 0x560779ee0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779ff4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779ff4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:51454/0 (socket says 172.21.15.39:51454) 2024-05-24T11:48:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d35d9b700 1 -- 172.21.15.39:0/406678590 learned_addr learned my addr 172.21.15.39:0/406678590 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779feb480 0x560779ff45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff1270 0x560779ff5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d35d9b700 1 -- 172.21.15.39:0/406678590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779feb480 msgr2=0x560779ff45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d35d9b700 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779feb480 0x560779ff45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d35d9b700 1 -- 172.21.15.39:0/406678590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff1270 msgr2=0x560779ff5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d35d9b700 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff1270 0x560779ff5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d35d9b700 1 -- 172.21.15.39:0/406678590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d38009bf0 con 0x560779ff0950 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d35d9b700 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779ff4b30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9d28005800 tx=0x7f9d28005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fe45027cd056efe server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.130+0000 7f9d25ffb700 1 -- 172.21.15.39:0/406678590 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2801d070 con 0x560779ff0950 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d25ffb700 1 -- 172.21.15.39:0/406678590 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9d2800bc60 con 0x560779ff0950 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d5107ab80 1 -- 172.21.15.39:0/406678590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560779e94210 con 0x560779ff0950 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d25ffb700 1 -- 172.21.15.39:0/406678590 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d2800be20 con 0x560779ff0950 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d5107ab80 10 client.25726 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d5107ab80 1 -- 172.21.15.39:0/406678590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560779d68b00 con 0x560779ff0950 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d25ffb700 1 -- 172.21.15.39:0/406678590 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560779e94210 con 0x560779ff0950 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d25ffb700 1 -- 172.21.15.39:0/406678590 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9d2800f4d0 con 0x560779ff0950 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d5107ab80 10 client.25726 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d5107ab80 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d5107ab80 20 client.25726 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56077a004db0 0x56077a0071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d5107ab80 4 client.25726 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:48:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.131+0000 7f9d5107ab80 1 -- 172.21.15.39:0/406678590 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560779fba7b0 con 0x56077a004db0 2024-05-24T11:48:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.132+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56077a004db0 0x56077a0071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.132+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56077a004db0 0x56077a0071d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c1243fcd62a458 server_cookie=e708e8d2d7bac377 in_seq=0 out_seq=0 2024-05-24T11:48:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.132+0000 7f9d25ffb700 10 client.25726 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d25ffb700 1 -- 172.21.15.39:0/406678590 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560779fba7b0 con 0x56077a004db0 2024-05-24T11:48:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d25ffb700 10 client.25726 handle_command_reply: tid=0 2024-05-24T11:48:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 client.25726 shutdown 2024-05-24T11:48:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 -- 172.21.15.39:0/406678590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 msgr2=0x560779ff4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779ff4b30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9d28005800 tx=0x7f9d28005da0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 -- 172.21.15.39:0/406678590 shutdown_connections 2024-05-24T11:48:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56077a004db0 0x56077a0071d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779ff1270 0x560779ff5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779feb480 0x560779ff45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d25ffb700 0 client.25726 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/406678590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779ff4b30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 -- 172.21.15.39:0/406678590 >> 172.21.15.39:0/406678590 conn(0x560779f491b0 msgr2=0x560779fef4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 -- 172.21.15.39:0/406678590 shutdown_connections 2024-05-24T11:48:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.133+0000 7f9d5107ab80 1 -- 172.21.15.39:0/406678590 wait complete. 2024-05-24T11:48:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.134+0000 7f9d5107ab80 20 client.25726 trim_cache size 0 max 16384 2024-05-24T11:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.159+0000 7f9d5107ab80 1 Processor -- start 2024-05-24T11:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d5107ab80 1 -- start start 2024-05-24T11:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 0x56077a0071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779fe5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff1270 0x560779fefd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077a00c420 con 0x560779feb480 2024-05-24T11:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560779fea1e0 con 0x560779ff1270 2024-05-24T11:48:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077a001660 con 0x560779ff0950 2024-05-24T11:48:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779fe5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d3ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 0x56077a0071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779fe5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:51456/0 (socket says 172.21.15.39:51456) 2024-05-24T11:48:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d35d9b700 1 -- 172.21.15.39:0/3206493488 learned_addr learned my addr 172.21.15.39:0/3206493488 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/3206493488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff1270 0x560779fefd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d35d9b700 1 -- 172.21.15.39:0/3206493488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff1270 msgr2=0x560779fefd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d35d9b700 1 --2- 172.21.15.39:0/3206493488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff1270 0x560779fefd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d35d9b700 1 -- 172.21.15.39:0/3206493488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 msgr2=0x56077a0071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d35d9b700 1 --2- 172.21.15.39:0/3206493488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 0x56077a0071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.160+0000 7f9d35d9b700 1 -- 172.21.15.39:0/3206493488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560779f01500 con 0x560779ff0950 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.161+0000 7f9d35d9b700 1 --2- 172.21.15.39:0/3206493488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779fe5910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800f850 tx=0x7f9d2800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79a29d065f5d7c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.161+0000 7f9d3559a700 1 -- 172.21.15.39:0/3206493488 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2801d040 con 0x560779ff0950 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.161+0000 7f9d3559a700 1 -- 172.21.15.39:0/3206493488 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d2801e040 con 0x560779ff0950 2024-05-24T11:48:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.161+0000 7f9d3559a700 1 -- 172.21.15.39:0/3206493488 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28004a50 con 0x560779ff0950 2024-05-24T11:48:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.162+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3206493488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 msgr2=0x560779fe5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.162+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/3206493488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779fe5910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800f850 tx=0x7f9d2800fa70 comp rx=0 tx=0).stop 2024-05-24T11:48:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.162+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3206493488 shutdown_connections 2024-05-24T11:48:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.162+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/3206493488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff1270 0x560779fefd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.162+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/3206493488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 0x56077a0071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.162+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/3206493488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff0950 0x560779fe5910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.162+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3206493488 >> 172.21.15.39:0/3206493488 conn(0x560779f491b0 msgr2=0x560779fe1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.162+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3206493488 shutdown_connections 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.162+0000 7f9d5107ab80 1 -- 172.21.15.39:0/3206493488 wait complete. 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.163+0000 7f9d5107ab80 1 -- start start 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.163+0000 7f9d5107ab80 1 Processor -- start 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.163+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 0x560779ff8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.163+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff0950 0x560779ff9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.163+0000 7f9d5107ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff1270 0x56077a0592a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.163+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077a0597e0 con 0x560779feb480 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.163+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077a059920 con 0x560779ff0950 2024-05-24T11:48:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.163+0000 7f9d5107ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077a06aa70 con 0x560779ff1270 2024-05-24T11:48:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.163+0000 7f9d5107ab80 10 client.0 _ll_register_callbacks cb 0x560779ee0ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff0950 0x560779ff9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff0950 0x560779ff9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:38730/0 (socket says 172.21.15.39:38730) 2024-05-24T11:48:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d35d9b700 1 -- 172.21.15.39:0/1824716737 learned_addr learned my addr 172.21.15.39:0/1824716737 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d35d9b700 1 -- 172.21.15.39:0/1824716737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff1270 msgr2=0x56077a0592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff1270 0x56077a0592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 0x560779ff8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d35d9b700 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff1270 0x56077a0592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d35d9b700 1 -- 172.21.15.39:0/1824716737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 msgr2=0x560779ff8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d35d9b700 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 0x560779ff8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d35d9b700 1 -- 172.21.15.39:0/1824716737 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560779d68b00 con 0x560779ff0950 2024-05-24T11:48:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.164+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 0x560779ff8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:48:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.167+0000 7f9d35d9b700 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff0950 0x560779ff9470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800f9c0 tx=0x7f9d2800bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87d07d6cf2a1394 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.167+0000 7f9d25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:48:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.168+0000 7f9d25ffb700 1 -- 172.21.15.39:0/1824716737 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2801d040 con 0x560779ff0950 2024-05-24T11:48:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.168+0000 7f9d25ffb700 1 -- 172.21.15.39:0/1824716737 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9d28000e40 con 0x560779ff0950 2024-05-24T11:48:16.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.168+0000 7f9d5107ab80 1 -- 172.21.15.39:0/1824716737 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d380158a0 con 0x560779ff0950 2024-05-24T11:48:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.168+0000 7f9d25ffb700 1 -- 172.21.15.39:0/1824716737 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d280048d0 con 0x560779ff0950 2024-05-24T11:48:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.168+0000 7f9d5107ab80 10 client.25808 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.168+0000 7f9d5107ab80 1 -- 172.21.15.39:0/1824716737 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560779dc0c70 con 0x560779ff0950 2024-05-24T11:48:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.169+0000 7f9d25ffb700 1 -- 172.21.15.39:0/1824716737 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d380158a0 con 0x560779ff0950 2024-05-24T11:48:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.169+0000 7f9d25ffb700 1 -- 172.21.15.39:0/1824716737 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9d28044050 con 0x560779ff0950 2024-05-24T11:48:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.170+0000 7f9d5107ab80 10 client.25808 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.170+0000 7f9d5107ab80 10 client.25808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.170+0000 7f9d5107ab80 20 client.25808 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.170+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56077a06be10 0x56077a06e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.170+0000 7f9d5107ab80 4 client.25808 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:48:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.170+0000 7f9d5107ab80 1 -- 172.21.15.39:0/1824716737 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560779f33e60 con 0x56077a06be10 2024-05-24T11:48:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.170+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56077a06be10 0x56077a06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.170+0000 7f9d25ffb700 10 client.25808 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:16.170+0000 7f9d3ed9d700 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56077a06be10 0x56077a06e230 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b67ff5e6bbd5122a server_cookie=33f8b4724a3b7935 in_seq=0 out_seq=0 2024-05-24T11:48:16.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:16 smithi053 ceph-mon[47448]: pgmap v313: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:48:16.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:16 smithi137 ceph-mon[47907]: pgmap v313: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:48:16.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:16 smithi039 ceph-mon[42919]: pgmap v313: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:48:18.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:18 smithi053 ceph-mon[47448]: pgmap v314: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:48:18.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:18 smithi137 ceph-mon[47907]: pgmap v314: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:48:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.540+0000 7f9d25ffb700 1 -- 172.21.15.39:0/1824716737 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560779f33e60 con 0x56077a06be10 2024-05-24T11:48:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.540+0000 7f9d25ffb700 10 client.25808 handle_command_reply: tid=0 2024-05-24T11:48:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.540+0000 7f9d5107ab80 1 client.25808 shutdown 2024-05-24T11:48:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 -- 172.21.15.39:0/1824716737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff0950 msgr2=0x560779ff9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff0950 0x560779ff9470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800f9c0 tx=0x7f9d2800bb10 comp rx=0 tx=0).stop 2024-05-24T11:48:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 -- 172.21.15.39:0/1824716737 shutdown_connections 2024-05-24T11:48:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560779ff1270 0x56077a0592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x56077a06be10 0x56077a06e230 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560779feb480 0x560779ff8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 --2- 172.21.15.39:0/1824716737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560779ff0950 0x560779ff9470 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d25ffb700 0 client.25808 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 -- 172.21.15.39:0/1824716737 >> 172.21.15.39:0/1824716737 conn(0x560779f491b0 msgr2=0x560779fe7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 -- 172.21.15.39:0/1824716737 shutdown_connections 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.541+0000 7f9d5107ab80 1 -- 172.21.15.39:0/1824716737 wait complete. 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.542+0000 7f9d5107ab80 20 client.25808 trim_cache size 0 max 16384 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:48:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:48:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 -- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9d2c0a1310 msgr2=0x7f9d2c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9d2c0a1310 0x7f9d2c0a37d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x560779fe6790 tx=0x7f9d30099260 comp rx=0 tx=0).stop 2024-05-24T11:48:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 -- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40107500 msgr2=0x7f9d4019da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40107500 0x7f9d4019da30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9d4006a030 tx=0x7f9d3000b6c0 comp rx=0 tx=0).stop 2024-05-24T11:48:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 -- 172.21.15.39:0/1159939082 shutdown_connections 2024-05-24T11:48:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d401a2340 0x7f9d401a27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9d2c0a1310 0x7f9d2c0a37d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d40107500 0x7f9d4019da30 secure :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9d4006a030 tx=0x7f9d3000b6c0 comp rx=0 tx=0).stop 2024-05-24T11:48:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 --2- 172.21.15.39:0/1159939082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d40107ec0 0x7f9d4019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 -- 172.21.15.39:0/1159939082 >> 172.21.15.39:0/1159939082 conn(0x7f9d40076dc0 msgr2=0x7f9d40078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 -- 172.21.15.39:0/1159939082 shutdown_connections 2024-05-24T11:48:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:18.545+0000 7f9d44ebf700 1 -- 172.21.15.39:0/1159939082 wait complete. 2024-05-24T11:48:18.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:18 smithi039 ceph-mon[42919]: pgmap v314: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:48:19.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 535.0 s, 950.0 tps, lat 104.511 ms stddev 33.745 2024-05-24T11:48:20.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:20 smithi053 ceph-mon[47448]: pgmap v315: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-05-24T11:48:20.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:20 smithi137 ceph-mon[47907]: pgmap v315: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-05-24T11:48:20.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:20 smithi039 ceph-mon[42919]: pgmap v315: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-05-24T11:48:22.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:22 smithi053 ceph-mon[47448]: pgmap v316: 97 pgs: 97 active+clean; 5.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 16 MiB/s wr, 1.22k op/s 2024-05-24T11:48:22.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:22 smithi137 ceph-mon[47907]: pgmap v316: 97 pgs: 97 active+clean; 5.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 16 MiB/s wr, 1.22k op/s 2024-05-24T11:48:22.581 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:22 smithi039 ceph-mon[42919]: pgmap v316: 97 pgs: 97 active+clean; 5.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 16 MiB/s wr, 1.22k op/s 2024-05-24T11:48:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 540.0 s, 1012.6 tps, lat 99.028 ms stddev 36.518 2024-05-24T11:48:24.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:24 smithi053 ceph-mon[47448]: pgmap v317: 97 pgs: 97 active+clean; 5.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 14 MiB/s wr, 1.08k op/s 2024-05-24T11:48:24.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:24 smithi137 ceph-mon[47907]: pgmap v317: 97 pgs: 97 active+clean; 5.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 14 MiB/s wr, 1.08k op/s 2024-05-24T11:48:24.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:24 smithi039 ceph-mon[42919]: pgmap v317: 97 pgs: 97 active+clean; 5.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 14 MiB/s wr, 1.08k op/s 2024-05-24T11:48:25.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:25 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:48:25.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:25 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:48:25.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:25 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:48:26.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: pgmap v318: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 25 MiB/s wr, 1.66k op/s 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.491 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: pgmap v318: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 25 MiB/s wr, 1.66k op/s 2024-05-24T11:48:26.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: pgmap v318: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 25 MiB/s wr, 1.66k op/s 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:26.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi137 to 182.4M 2024-05-24T11:48:27.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: Unable to set osd_memory_target on smithi137 to 191356211: error parsing value: Value '191356211' is below minimum 939524096 2024-05-24T11:48:27.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-24T11:48:27.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-24T11:48:27.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:48:27.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:48:27.490 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:27.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi137 to 182.4M 2024-05-24T11:48:27.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: Unable to set osd_memory_target on smithi137 to 191356211: error parsing value: Value '191356211' is below minimum 939524096 2024-05-24T11:48:27.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-24T11:48:27.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-24T11:48:27.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:48:27.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:48:27.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:27.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi137 to 182.4M 2024-05-24T11:48:27.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: Unable to set osd_memory_target on smithi137 to 191356211: error parsing value: Value '191356211' is below minimum 939524096 2024-05-24T11:48:27.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi053 to 183.5M 2024-05-24T11:48:27.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: Unable to set osd_memory_target on smithi053 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-24T11:48:27.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-24T11:48:27.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:48:27.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:48:27.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:48:28.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:28 smithi053 ceph-mon[47448]: pgmap v319: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-05-24T11:48:28.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:28 smithi053 ceph-mon[47448]: Adjusting osd_memory_target on smithi039 to 182.5M 2024-05-24T11:48:28.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:28 smithi053 ceph-mon[47448]: Unable to set osd_memory_target on smithi039 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-05-24T11:48:28.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:28 smithi137 ceph-mon[47907]: pgmap v319: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-05-24T11:48:28.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:28 smithi137 ceph-mon[47907]: Adjusting osd_memory_target on smithi039 to 182.5M 2024-05-24T11:48:28.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:28 smithi137 ceph-mon[47907]: Unable to set osd_memory_target on smithi039 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-05-24T11:48:28.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:28 smithi039 ceph-mon[42919]: pgmap v319: 97 pgs: 97 active+clean; 5.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-05-24T11:48:28.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:28 smithi039 ceph-mon[42919]: Adjusting osd_memory_target on smithi039 to 182.5M 2024-05-24T11:48:28.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:28 smithi039 ceph-mon[42919]: Unable to set osd_memory_target on smithi039 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-05-24T11:48:29.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 545.0 s, 940.6 tps, lat 106.652 ms stddev 38.848 2024-05-24T11:48:30.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:30 smithi053 ceph-mon[47448]: pgmap v320: 97 pgs: 97 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 27 MiB/s wr, 1.69k op/s 2024-05-24T11:48:30.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:30 smithi137 ceph-mon[47907]: pgmap v320: 97 pgs: 97 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 27 MiB/s wr, 1.69k op/s 2024-05-24T11:48:30.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:30 smithi039 ceph-mon[42919]: pgmap v320: 97 pgs: 97 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 27 MiB/s wr, 1.69k op/s 2024-05-24T11:48:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.731+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2437033993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64108430 msgr2=0x7f4e6410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.731+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2437033993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64108430 0x7f4e6410ffb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0098f0 tx=0x7f4e5c009c00 comp rx=0 tx=0).stop 2024-05-24T11:48:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.731+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2437033993 shutdown_connections 2024-05-24T11:48:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.731+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2437033993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64108430 0x7f4e6410ffb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.731+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2437033993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e64103310 0x7f4e641036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.731+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2437033993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e64103cc0 0x7f4e64107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.731+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2437033993 >> 172.21.15.39:0/2437033993 conn(0x7f4e640feb50 msgr2=0x7f4e64100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.731+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2437033993 shutdown_connections 2024-05-24T11:48:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.732+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2437033993 wait complete. 2024-05-24T11:48:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e68ed0700 1 Processor -- start 2024-05-24T11:48:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e68ed0700 1 -- start start 2024-05-24T11:48:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e68ed0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64103310 0x7f4e6406ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e68ed0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e64108430 0x7f4e6406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e68ed0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e6406f6f0 0x7f4e6410eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e68ed0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6410f0d0 con 0x7f4e6406f6f0 2024-05-24T11:48:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e68ed0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6410f240 con 0x7f4e64103310 2024-05-24T11:48:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e68ed0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6410f400 con 0x7f4e64108430 2024-05-24T11:48:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e6259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64103310 0x7f4e6406ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e6259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64103310 0x7f4e6406ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:37984/0 (socket says 172.21.15.39:37984) 2024-05-24T11:48:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.733+0000 7f4e6259c700 1 -- 172.21.15.39:0/2463020983 learned_addr learned my addr 172.21.15.39:0/2463020983 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.734+0000 7f4e62d9d700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e6406f6f0 0x7f4e6410eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.734+0000 7f4e6259c700 1 -- 172.21.15.39:0/2463020983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e64108430 msgr2=0x7f4e6406f1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:48:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.734+0000 7f4e6259c700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e64108430 0x7f4e6406f1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.734+0000 7f4e6259c700 1 -- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e6406f6f0 msgr2=0x7f4e6410eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.734+0000 7f4e6259c700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e6406f6f0 0x7f4e6410eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.734+0000 7f4e6259c700 1 -- 172.21.15.39:0/2463020983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e54009d00 con 0x7f4e64103310 2024-05-24T11:48:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.735+0000 7f4e6259c700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64103310 0x7f4e6406ec70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400e3c0 tx=0x7f4e5400e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.735+0000 7f4e5b7fe700 1 -- 172.21.15.39:0/2463020983 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54018070 con 0x7f4e64103310 2024-05-24T11:48:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.735+0000 7f4e5b7fe700 1 -- 172.21.15.39:0/2463020983 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e5400f040 con 0x7f4e64103310 2024-05-24T11:48:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.735+0000 7f4e5b7fe700 1 -- 172.21.15.39:0/2463020983 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540043a0 con 0x7f4e64103310 2024-05-24T11:48:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.735+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2463020983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c009520 con 0x7f4e64103310 2024-05-24T11:48:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.735+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2463020983 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6410f9b0 con 0x7f4e64103310 2024-05-24T11:48:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.738+0000 7f4e5b7fe700 1 -- 172.21.15.39:0/2463020983 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4e540090d0 con 0x7f4e64103310 2024-05-24T11:48:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.739+0000 7f4e5b7fe700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4e500a1390 0x7f4e500a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.739+0000 7f4e5b7fe700 1 -- 172.21.15.39:0/2463020983 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4e540ccb40 con 0x7f4e64103310 2024-05-24T11:48:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.739+0000 7f4e7508bb80 1 Processor -- start 2024-05-24T11:48:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e7508bb80 1 -- start start 2024-05-24T11:48:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4e500a1390 0x7f4e500a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e9645cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e96456f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e964578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e963ae0f0 con 0x556e9645c950 2024-05-24T11:48:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e963abab0 con 0x556e96457480 2024-05-24T11:48:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e963a73a0 con 0x556e9645d270 2024-05-24T11:48:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e62d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e964578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e62d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e964578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:37996/0 (socket says 172.21.15.39:37996) 2024-05-24T11:48:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e62d9d700 1 -- 172.21.15.39:0/3237946736 learned_addr learned my addr 172.21.15.39:0/3237946736 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.740+0000 7f4e6259c700 1 --2- 172.21.15.39:0/3237946736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e9645cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/3237946736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e96456f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e62d9d700 1 -- 172.21.15.39:0/3237946736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 msgr2=0x556e96456f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e62d9d700 1 --2- 172.21.15.39:0/3237946736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e96456f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e62d9d700 1 -- 172.21.15.39:0/3237946736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 msgr2=0x556e9645cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e62d9d700 1 --2- 172.21.15.39:0/3237946736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e9645cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e62d9d700 1 -- 172.21.15.39:0/3237946736 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e96228350 con 0x556e96457480 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e62d9d700 1 --2- 172.21.15.39:0/3237946736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e964578a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c016000 tx=0x7f4e5c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68049593e9dfabb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4e500a1390 0x7f4e500a3850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e64110220 tx=0x7f4e4c00b370 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e58ff9700 1 -- 172.21.15.39:0/3237946736 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c01c040 con 0x556e96457480 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e58ff9700 1 -- 172.21.15.39:0/3237946736 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e5c00f8f0 con 0x556e96457480 2024-05-24T11:48:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.741+0000 7f4e58ff9700 1 -- 172.21.15.39:0/3237946736 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0052a0 con 0x556e96457480 2024-05-24T11:48:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.742+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3237946736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 msgr2=0x556e964578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.742+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3237946736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e964578a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c016000 tx=0x7f4e5c0058e0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.742+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3237946736 shutdown_connections 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.742+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3237946736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e964578a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.742+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3237946736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e9645cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.742+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3237946736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e96456f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.742+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3237946736 >> 172.21.15.39:0/3237946736 conn(0x556e963b51b0 msgr2=0x556e96451840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.742+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3237946736 shutdown_connections 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.742+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3237946736 wait complete. 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e7508bb80 1 -- start start 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e7508bb80 1 Processor -- start 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e96457480 0x556e96463260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645c950 0x556e96460550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 0x556e96460c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96343500 con 0x556e96457480 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e963431a0 con 0x556e9645d270 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9622a640 con 0x556e9645c950 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e7508bb80 10 client.0 _ll_register_callbacks cb 0x556e9634cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e6259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e96457480 0x556e96463260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e6259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e96457480 0x556e96463260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60416/0 (socket says 172.21.15.39:60416) 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e62d9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 0x556e96460c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.743+0000 7f4e6259c700 1 -- 172.21.15.39:0/3089842919 learned_addr learned my addr 172.21.15.39:0/3089842919 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.744+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645c950 0x556e96460550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.744+0000 7f4e6259c700 1 -- 172.21.15.39:0/3089842919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645c950 msgr2=0x556e96460550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.744+0000 7f4e6259c700 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645c950 0x556e96460550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.744+0000 7f4e6259c700 1 -- 172.21.15.39:0/3089842919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 msgr2=0x556e96460c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.744+0000 7f4e6259c700 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 0x556e96460c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.744+0000 7f4e6259c700 1 -- 172.21.15.39:0/3089842919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e64110820 con 0x556e96457480 2024-05-24T11:48:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.744+0000 7f4e62d9d700 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 0x556e96460c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:48:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.744+0000 7f4e6259c700 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e96457480 0x556e96463260 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009fd0 tx=0x7f4e540ca7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2ff75547715e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.744+0000 7f4e49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.745+0000 7f4e49ffb700 1 -- 172.21.15.39:0/3089842919 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54099380 con 0x556e96457480 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.745+0000 7f4e49ffb700 1 -- 172.21.15.39:0/3089842919 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4e54099c90 con 0x556e96457480 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.745+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3089842919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e964267b0 con 0x556e96457480 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.745+0000 7f4e49ffb700 1 -- 172.21.15.39:0/3089842919 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e5409dac0 con 0x556e96457480 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.745+0000 7f4e7508bb80 10 client.16110 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.745+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3089842919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556e961d4b00 con 0x556e96457480 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.746+0000 7f4e49ffb700 1 -- 172.21.15.39:0/3089842919 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556e964267b0 con 0x556e96457480 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.746+0000 7f4e49ffb700 1 -- 172.21.15.39:0/3089842919 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4e54004bb0 con 0x556e96457480 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.746+0000 7f4e7508bb80 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.746+0000 7f4e7508bb80 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.746+0000 7f4e7508bb80 20 client.16110 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.746+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556e96470a00 0x556e96472e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.746+0000 7f4e7508bb80 4 client.16110 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.746+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3089842919 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e9639fe60 con 0x556e96470a00 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.747+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556e96470a00 0x556e96472e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.747+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556e96470a00 0x556e96472e20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674f799208308c6b server_cookie=9ce42af99bf4c6be in_seq=0 out_seq=0 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.747+0000 7f4e49ffb700 10 client.16110 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e49ffb700 1 -- 172.21.15.39:0/3089842919 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e9639fe60 con 0x556e96470a00 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e49ffb700 10 client.16110 handle_command_reply: tid=0 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e7508bb80 1 client.16110 shutdown 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3089842919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e96457480 msgr2=0x556e96463260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e96457480 0x556e96463260 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009fd0 tx=0x7f4e540ca7a0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3089842919 shutdown_connections 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556e96470a00 0x556e96472e20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 0x556e96460c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e96457480 0x556e96463260 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/3089842919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645c950 0x556e96460550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.748+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3089842919 >> 172.21.15.39:0/3089842919 conn(0x556e963b51b0 msgr2=0x556e9645b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.749+0000 7f4e49ffb700 0 client.16110 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.749+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3089842919 shutdown_connections 2024-05-24T11:48:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.749+0000 7f4e7508bb80 1 -- 172.21.15.39:0/3089842919 wait complete. 2024-05-24T11:48:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.749+0000 7f4e7508bb80 20 client.16110 trim_cache size 0 max 16384 2024-05-24T11:48:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.768+0000 7f4e7508bb80 1 Processor -- start 2024-05-24T11:48:31.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.768+0000 7f4e7508bb80 1 -- start start 2024-05-24T11:48:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.768+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e96457860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.768+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e96451910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.768+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e964534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.768+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e964539f0 con 0x556e9645c950 2024-05-24T11:48:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.768+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96461b20 con 0x556e96457480 2024-05-24T11:48:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.768+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96453290 con 0x556e9645d270 2024-05-24T11:48:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e6259c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e96457860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e62d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e964534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e62d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e964534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38466/0 (socket says 172.21.15.39:38466) 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e62d9d700 1 -- 172.21.15.39:0/67384631 learned_addr learned my addr 172.21.15.39:0/67384631 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/67384631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e96451910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e62d9d700 1 -- 172.21.15.39:0/67384631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 msgr2=0x556e96457860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e62d9d700 1 --2- 172.21.15.39:0/67384631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e96457860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e62d9d700 1 -- 172.21.15.39:0/67384631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 msgr2=0x556e96451910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e62d9d700 1 --2- 172.21.15.39:0/67384631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e96451910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e62d9d700 1 -- 172.21.15.39:0/67384631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e9636d500 con 0x556e9645d270 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.769+0000 7f4e62d9d700 1 --2- 172.21.15.39:0/67384631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e964534b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00bec0 tx=0x7f4e5c009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd4b69f58f597d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.770+0000 7f4e597fa700 1 -- 172.21.15.39:0/67384631 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c01c040 con 0x556e9645d270 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.770+0000 7f4e597fa700 1 -- 172.21.15.39:0/67384631 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e5c00b760 con 0x556e9645d270 2024-05-24T11:48:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.770+0000 7f4e597fa700 1 -- 172.21.15.39:0/67384631 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0055a0 con 0x556e9645d270 2024-05-24T11:48:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.770+0000 7f4e7508bb80 1 -- 172.21.15.39:0/67384631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 msgr2=0x556e964534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.770+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/67384631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e964534b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00bec0 tx=0x7f4e5c009480 comp rx=0 tx=0).stop 2024-05-24T11:48:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 -- 172.21.15.39:0/67384631 shutdown_connections 2024-05-24T11:48:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/67384631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e9645d270 0x556e964534b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/67384631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e96457480 0x556e96457860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/67384631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e96451910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 -- 172.21.15.39:0/67384631 >> 172.21.15.39:0/67384631 conn(0x556e963b51b0 msgr2=0x556e96455630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 -- 172.21.15.39:0/67384631 shutdown_connections 2024-05-24T11:48:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 -- 172.21.15.39:0/67384631 wait complete. 2024-05-24T11:48:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 -- start start 2024-05-24T11:48:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 Processor -- start 2024-05-24T11:48:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.771+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e96457480 0x556e96468070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e9645b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e7508bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 0x556e9645bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e964623b0 con 0x556e9645c950 2024-05-24T11:48:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96462520 con 0x556e9645d270 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e7508bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e964626e0 con 0x556e96457480 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e7508bb80 10 client.0 _ll_register_callbacks cb 0x556e9634cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e6259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e96457480 0x556e96468070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e6259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e96457480 0x556e96468070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38480/0 (socket says 172.21.15.39:38480) 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e6259c700 1 -- 172.21.15.39:0/37152602 learned_addr learned my addr 172.21.15.39:0/37152602 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e62d9d700 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 0x556e9645bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e9645b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e6259c700 1 -- 172.21.15.39:0/37152602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 msgr2=0x556e9645bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e6259c700 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 0x556e9645bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e6259c700 1 -- 172.21.15.39:0/37152602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 msgr2=0x556e9645b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e6259c700 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e9645b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e6259c700 1 -- 172.21.15.39:0/37152602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e96228350 con 0x556e96457480 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e6259c700 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e96457480 0x556e96468070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e540a1b50 tx=0x7f4e54004de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c6b84d29c0d7155 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.772+0000 7f4e49ffb700 1 -- 172.21.15.39:0/37152602 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540b27b0 con 0x556e96457480 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e49ffb700 1 -- 172.21.15.39:0/37152602 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4e54099c90 con 0x556e96457480 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e7508bb80 1 -- 172.21.15.39:0/37152602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e96300210 con 0x556e96457480 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e49ffb700 1 -- 172.21.15.39:0/37152602 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e54095fd0 con 0x556e96457480 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e7508bb80 10 client.25741 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e7508bb80 1 -- 172.21.15.39:0/37152602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556e9622cc70 con 0x556e96457480 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e49ffb700 1 -- 172.21.15.39:0/37152602 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556e96300210 con 0x556e96457480 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e49ffb700 1 -- 172.21.15.39:0/37152602 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4e540830d0 con 0x556e96457480 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e7508bb80 10 client.25741 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e7508bb80 10 client.25741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e7508bb80 20 client.25741 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556e964c6640 0x556e964c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e7508bb80 4 client.25741 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:48:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.773+0000 7f4e7508bb80 1 -- 172.21.15.39:0/37152602 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e961f5c90 con 0x556e964c6640 2024-05-24T11:48:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.774+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556e964c6640 0x556e964c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.774+0000 7f4e49ffb700 10 client.25741 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:31.774+0000 7f4e61d9b700 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556e964c6640 0x556e964c8a60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6041aa616d2c8e04 server_cookie=c0d8531e7116478 in_seq=0 out_seq=0 2024-05-24T11:48:32.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:32 smithi053 ceph-mon[47448]: pgmap v321: 97 pgs: 97 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 20 MiB/s wr, 1.21k op/s 2024-05-24T11:48:32.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:32 smithi137 ceph-mon[47907]: pgmap v321: 97 pgs: 97 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 20 MiB/s wr, 1.21k op/s 2024-05-24T11:48:32.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:32 smithi039 ceph-mon[42919]: pgmap v321: 97 pgs: 97 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 20 MiB/s wr, 1.21k op/s 2024-05-24T11:48:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.941+0000 7f4e49ffb700 1 -- 172.21.15.39:0/37152602 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e961f5c90 con 0x556e964c6640 2024-05-24T11:48:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.941+0000 7f4e49ffb700 10 client.25741 handle_command_reply: tid=0 2024-05-24T11:48:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.941+0000 7f4e7508bb80 1 client.25741 shutdown 2024-05-24T11:48:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 -- 172.21.15.39:0/37152602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e96457480 msgr2=0x556e96468070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e96457480 0x556e96468070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e540a1b50 tx=0x7f4e54004de0 comp rx=0 tx=0).stop 2024-05-24T11:48:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 -- 172.21.15.39:0/37152602 shutdown_connections 2024-05-24T11:48:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556e96457480 0x556e96468070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556e964c6640 0x556e964c8a60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e9645d270 0x556e9645bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 --2- 172.21.15.39:0/37152602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e9645c950 0x556e9645b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 -- 172.21.15.39:0/37152602 >> 172.21.15.39:0/37152602 conn(0x556e963b51b0 msgr2=0x556e9645fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e49ffb700 0 client.25741 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 -- 172.21.15.39:0/37152602 shutdown_connections 2024-05-24T11:48:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.942+0000 7f4e7508bb80 1 -- 172.21.15.39:0/37152602 wait complete. 2024-05-24T11:48:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e7508bb80 20 client.25741 trim_cache size 0 max 16384 2024-05-24T11:48:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:48:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:48:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:48:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4e500a1390 msgr2=0x7f4e500a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4e500a1390 0x7f4e500a3850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e64110220 tx=0x7f4e4c00b370 comp rx=0 tx=0).stop 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2463020983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64103310 msgr2=0x7f4e6406ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64103310 0x7f4e6406ec70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400e3c0 tx=0x7f4e5400e780 comp rx=0 tx=0).stop 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2463020983 shutdown_connections 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4e500a1390 0x7f4e500a3850 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e6406f6f0 0x7f4e6410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e64103310 0x7f4e6406ec70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 --2- 172.21.15.39:0/2463020983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e64108430 0x7f4e6406f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.943+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2463020983 >> 172.21.15.39:0/2463020983 conn(0x7f4e640feb50 msgr2=0x7f4e640ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.944+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2463020983 shutdown_connections 2024-05-24T11:48:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:32.944+0000 7f4e68ed0700 1 -- 172.21.15.39:0/2463020983 wait complete. 2024-05-24T11:48:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 550.0 s, 846.0 tps, lat 118.326 ms stddev 182.429 2024-05-24T11:48:34.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:34 smithi053 ceph-mon[47448]: pgmap v322: 97 pgs: 97 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-05-24T11:48:34.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:34 smithi137 ceph-mon[47907]: pgmap v322: 97 pgs: 97 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-05-24T11:48:34.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:34 smithi039 ceph-mon[42919]: pgmap v322: 97 pgs: 97 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-05-24T11:48:36.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:36 smithi053 ceph-mon[47448]: pgmap v323: 97 pgs: 97 active+clean; 5.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2024-05-24T11:48:36.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:36 smithi137 ceph-mon[47907]: pgmap v323: 97 pgs: 97 active+clean; 5.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2024-05-24T11:48:36.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:36 smithi039 ceph-mon[42919]: pgmap v323: 97 pgs: 97 active+clean; 5.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2024-05-24T11:48:38.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:38 smithi053 ceph-mon[47448]: pgmap v324: 97 pgs: 97 active+clean; 5.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2024-05-24T11:48:38.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:38 smithi137 ceph-mon[47907]: pgmap v324: 97 pgs: 97 active+clean; 5.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2024-05-24T11:48:38.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:38 smithi039 ceph-mon[42919]: pgmap v324: 97 pgs: 97 active+clean; 5.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2024-05-24T11:48:39.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 555.0 s, 961.6 tps, lat 103.903 ms stddev 37.017 2024-05-24T11:48:40.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:40 smithi053 ceph-mon[47448]: pgmap v325: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2024-05-24T11:48:40.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:40 smithi137 ceph-mon[47907]: pgmap v325: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2024-05-24T11:48:40.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:40 smithi039 ceph-mon[42919]: pgmap v325: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2024-05-24T11:48:42.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:42 smithi053 ceph-mon[47448]: pgmap v326: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-05-24T11:48:42.496 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:42 smithi137 ceph-mon[47907]: pgmap v326: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-05-24T11:48:42.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:42 smithi039 ceph-mon[42919]: pgmap v326: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-05-24T11:48:44.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 560.0 s, 999.0 tps, lat 100.386 ms stddev 31.286 2024-05-24T11:48:44.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:44 smithi053 ceph-mon[47448]: pgmap v327: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-05-24T11:48:44.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:44 smithi137 ceph-mon[47907]: pgmap v327: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-05-24T11:48:44.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:44 smithi039 ceph-mon[42919]: pgmap v327: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-05-24T11:48:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.123+0000 7f58d12f2700 1 -- 172.21.15.39:0/2702734868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc10d5a0 msgr2=0x7f58cc10d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.123+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2702734868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc10d5a0 0x7f58cc10d980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009990 tx=0x7f58bc009ca0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.123+0000 7f58d12f2700 1 -- 172.21.15.39:0/2702734868 shutdown_connections 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.123+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2702734868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc10d5a0 0x7f58cc10d980 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.123+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2702734868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58cc107ea0 0x7f58cc114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.123+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2702734868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58cc1074e0 0x7f58cc107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.123+0000 7f58d12f2700 1 -- 172.21.15.39:0/2702734868 >> 172.21.15.39:0/2702734868 conn(0x7f58cc076dc0 msgr2=0x7f58cc105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.123+0000 7f58d12f2700 1 -- 172.21.15.39:0/2702734868 shutdown_connections 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.123+0000 7f58d12f2700 1 -- 172.21.15.39:0/2702734868 wait complete. 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.124+0000 7f58d12f2700 1 Processor -- start 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.124+0000 7f58d12f2700 1 -- start start 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.124+0000 7f58d12f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc1074e0 0x7f58cc0812c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.124+0000 7f58d12f2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58cc107ea0 0x7f58cc081800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.124+0000 7f58d12f2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58cc10d5a0 0x7f58cc07b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.124+0000 7f58d12f2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc07baf0 con 0x7f58cc107ea0 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.124+0000 7f58d12f2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc07bc60 con 0x7f58cc10d5a0 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.124+0000 7f58d12f2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc07be20 con 0x7f58cc1074e0 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58caffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc1074e0 0x7f58cc0812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58caffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc1074e0 0x7f58cc0812c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:39760/0 (socket says 172.21.15.39:39760) 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58caffd700 1 -- 172.21.15.39:0/2645012577 learned_addr learned my addr 172.21.15.39:0/2645012577 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58cb7fe700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58cc10d5a0 0x7f58cc07b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58caffd700 1 -- 172.21.15.39:0/2645012577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58cc10d5a0 msgr2=0x7f58cc07b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58caffd700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58cc10d5a0 0x7f58cc07b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58caffd700 1 -- 172.21.15.39:0/2645012577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58cc107ea0 msgr2=0x7f58cc081800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58caffd700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58cc107ea0 0x7f58cc081800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58caffd700 1 -- 172.21.15.39:0/2645012577 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc0095f0 con 0x7f58cc1074e0 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.125+0000 7f58caffd700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc1074e0 0x7f58cc0812c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0098e0 tx=0x7f58bc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.127+0000 7f58b3fff700 1 -- 172.21.15.39:0/2645012577 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc01d070 con 0x7f58cc1074e0 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.127+0000 7f58b3fff700 1 -- 172.21.15.39:0/2645012577 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58bc00fb80 con 0x7f58cc1074e0 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.127+0000 7f58d12f2700 1 -- 172.21.15.39:0/2645012577 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc07c0a0 con 0x7f58cc1074e0 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.127+0000 7f58d12f2700 1 -- 172.21.15.39:0/2645012577 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc070bd0 con 0x7f58cc1074e0 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.129+0000 7f58b3fff700 1 -- 172.21.15.39:0/2645012577 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc022c40 con 0x7f58cc1074e0 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.132+0000 7f58dd4adb80 1 Processor -- start 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.133+0000 7f58b3fff700 1 -- 172.21.15.39:0/2645012577 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f58bc022470 con 0x7f58cc1074e0 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58b3fff700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f58b40a13e0 0x7f58b40a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58b3fff700 1 -- 172.21.15.39:0/2645012577 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f58bc0ce060 con 0x7f58cc1074e0 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58dd4adb80 1 -- start start 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58cb7fe700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f58b40a13e0 0x7f58b40a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb2950 0x561cfafb2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafb3270 0x561cfafacf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafad480 0x561cfafad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfaf040f0 con 0x561cfafb2950 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfaf01ab0 con 0x561cfafad480 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.134+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfaefd3a0 con 0x561cfafb3270 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58cb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafad480 0x561cfafad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58cb7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafad480 0x561cfafad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:45844/0 (socket says 172.21.15.39:45844) 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58cb7fe700 1 -- 172.21.15.39:0/125298490 learned_addr learned my addr 172.21.15.39:0/125298490 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafb3270 0x561cfafacf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58caffd700 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb2950 0x561cfafb2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58ca7fc700 1 -- 172.21.15.39:0/125298490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafad480 msgr2=0x561cfafad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafad480 0x561cfafad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58ca7fc700 1 -- 172.21.15.39:0/125298490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb2950 msgr2=0x561cfafb2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb2950 0x561cfafb2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58ca7fc700 1 -- 172.21.15.39:0/125298490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cfad7e350 con 0x561cfafb3270 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.135+0000 7f58caffd700 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb2950 0x561cfafb2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.136+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafb3270 0x561cfafacf40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0773e0 tx=0x7f58c400c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=731bfede99854622 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.136+0000 7f58cb7fe700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f58b40a13e0 0x7f58b40a38a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58c00046e0 tx=0x7f58c000be30 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.136+0000 7f58b17fa700 1 -- 172.21.15.39:0/125298490 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c400ce40 con 0x561cfafb3270 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.136+0000 7f58b17fa700 1 -- 172.21.15.39:0/125298490 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58c4007aa0 con 0x561cfafb3270 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.136+0000 7f58b17fa700 1 -- 172.21.15.39:0/125298490 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c4011760 con 0x561cfafb3270 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.137+0000 7f58dd4adb80 1 -- 172.21.15.39:0/125298490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafb3270 msgr2=0x561cfafacf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.137+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafb3270 0x561cfafacf40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0773e0 tx=0x7f58c400c3c0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.137+0000 7f58dd4adb80 1 -- 172.21.15.39:0/125298490 shutdown_connections 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.137+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafad480 0x561cfafad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.137+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb2950 0x561cfafb2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.137+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/125298490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafb3270 0x561cfafacf40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.137+0000 7f58dd4adb80 1 -- 172.21.15.39:0/125298490 >> 172.21.15.39:0/125298490 conn(0x561cfaf0b1b0 msgr2=0x561cfafa7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.137+0000 7f58dd4adb80 1 -- 172.21.15.39:0/125298490 shutdown_connections 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.137+0000 7f58dd4adb80 1 -- 172.21.15.39:0/125298490 wait complete. 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.138+0000 7f58dd4adb80 1 -- start start 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.138+0000 7f58dd4adb80 1 Processor -- start 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.138+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafb9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.138+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafb6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.138+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafb6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.138+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfae99500 con 0x561cfafb3270 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.138+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfae991a0 con 0x561cfafb2950 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.138+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfad80640 con 0x561cfafad480 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.138+0000 7f58dd4adb80 10 client.0 _ll_register_callbacks cb 0x561cfaea2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58caffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafb9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58caffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafb9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:39784/0 (socket says 172.21.15.39:39784) 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58caffd700 1 -- 172.21.15.39:0/2472603719 learned_addr learned my addr 172.21.15.39:0/2472603719 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafb6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58caffd700 1 -- 172.21.15.39:0/2472603719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 msgr2=0x561cfafb6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58caffd700 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafb6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58caffd700 1 -- 172.21.15.39:0/2472603719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 msgr2=0x561cfafb6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58caffd700 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafb6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58caffd700 1 -- 172.21.15.39:0/2472603719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c4007c20 con 0x561cfafad480 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58caffd700 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafb9220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58bc097830 tx=0x7f58bc09bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2091e54b12824d63 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.139+0000 7f58aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58aeff5700 1 -- 172.21.15.39:0/2472603719 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc09aa20 con 0x561cfafad480 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58aeff5700 1 -- 172.21.15.39:0/2472603719 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f58bc09f160 con 0x561cfafad480 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2472603719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cfaf7c7b0 con 0x561cfafad480 2024-05-24T11:48:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58aeff5700 1 -- 172.21.15.39:0/2472603719 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58bc0a3720 con 0x561cfafad480 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58dd4adb80 10 client.25753 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2472603719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561cfad2ab00 con 0x561cfafad480 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58aeff5700 1 -- 172.21.15.39:0/2472603719 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561cfaf7c7b0 con 0x561cfafad480 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58aeff5700 1 -- 172.21.15.39:0/2472603719 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f58bc0b2030 con 0x561cfafad480 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58dd4adb80 10 client.25753 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58dd4adb80 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58dd4adb80 20 client.25753 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561cfafc6890 0x561cfafc8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58dd4adb80 4 client.25753 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.140+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2472603719 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cfaef5e60 con 0x561cfafc6890 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.141+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561cfafc6890 0x561cfafc8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.141+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561cfafc6890 0x561cfafc8cb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336503978c0a484f server_cookie=fd28c5480ac0b4a0 in_seq=0 out_seq=0 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.141+0000 7f58aeff5700 10 client.25753 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58aeff5700 1 -- 172.21.15.39:0/2472603719 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cfaef5e60 con 0x561cfafc6890 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58aeff5700 10 client.25753 handle_command_reply: tid=0 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58dd4adb80 1 client.25753 shutdown 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2472603719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 msgr2=0x561cfafb9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafb9220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58bc097830 tx=0x7f58bc09bb20 comp rx=0 tx=0).stop 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2472603719 shutdown_connections 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafb9220 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561cfafc6890 0x561cfafc8cb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafb6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2472603719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafb6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2472603719 >> 172.21.15.39:0/2472603719 conn(0x561cfaf0b1b0 msgr2=0x561cfafb14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.142+0000 7f58aeff5700 0 client.25753 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.143+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2472603719 shutdown_connections 2024-05-24T11:48:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.143+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2472603719 wait complete. 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.143+0000 7f58dd4adb80 20 client.25753 trim_cache size 0 max 16384 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.169+0000 7f58dd4adb80 1 Processor -- start 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.169+0000 7f58dd4adb80 1 -- start start 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafc2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafa7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafb5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfafa8440 con 0x561cfafb3270 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfaf9fa40 con 0x561cfafb2950 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfafa9290 con 0x561cfafad480 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58caffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafc2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58caffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafc2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:39786/0 (socket says 172.21.15.39:39786) 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58caffd700 1 -- 172.21.15.39:0/2576076672 learned_addr learned my addr 172.21.15.39:0/2576076672 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58cb7fe700 1 --2- 172.21.15.39:0/2576076672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafb5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.170+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/2576076672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafa7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.171+0000 7f58caffd700 1 -- 172.21.15.39:0/2576076672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 msgr2=0x561cfafa7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.171+0000 7f58caffd700 1 --2- 172.21.15.39:0/2576076672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafa7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.171+0000 7f58caffd700 1 -- 172.21.15.39:0/2576076672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 msgr2=0x561cfafb5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.171+0000 7f58caffd700 1 --2- 172.21.15.39:0/2576076672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafb5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.171+0000 7f58caffd700 1 -- 172.21.15.39:0/2576076672 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cfaec3500 con 0x561cfafad480 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.171+0000 7f58caffd700 1 --2- 172.21.15.39:0/2576076672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafc2720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58bc099680 tx=0x7f58bc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece97afef571cfdc server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.172+0000 7f58b1ffb700 1 -- 172.21.15.39:0/2576076672 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc09f160 con 0x561cfafad480 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.172+0000 7f58b1ffb700 1 -- 172.21.15.39:0/2576076672 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58bc0a13b0 con 0x561cfafad480 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.172+0000 7f58b1ffb700 1 -- 172.21.15.39:0/2576076672 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc09a820 con 0x561cfafad480 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.172+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2576076672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 msgr2=0x561cfafc2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.172+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2576076672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafc2720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58bc099680 tx=0x7f58bc009ca0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.173+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2576076672 shutdown_connections 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.173+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2576076672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafc2720 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.173+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2576076672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafb5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.173+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/2576076672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafa7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.173+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2576076672 >> 172.21.15.39:0/2576076672 conn(0x561cfaf0b1b0 msgr2=0x561cfafa3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.173+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2576076672 shutdown_connections 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.173+0000 7f58dd4adb80 1 -- 172.21.15.39:0/2576076672 wait complete. 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.173+0000 7f58dd4adb80 1 -- start start 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.173+0000 7f58dd4adb80 1 Processor -- start 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafbdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafbe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58dd4adb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafbbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfafbc1d0 con 0x561cfafb3270 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfafbc310 con 0x561cfafb2950 2024-05-24T11:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58dd4adb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfafbc4d0 con 0x561cfafad480 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58dd4adb80 10 client.0 _ll_register_callbacks cb 0x561cfaea2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58ca7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafbe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58ca7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafbe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:45864/0 (socket says 172.21.15.39:45864) 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.174+0000 7f58ca7fc700 1 -- 172.21.15.39:0/3946732004 learned_addr learned my addr 172.21.15.39:0/3946732004 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.175+0000 7f58cb7fe700 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafbbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.175+0000 7f58ca7fc700 1 -- 172.21.15.39:0/3946732004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 msgr2=0x561cfafbdc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.175+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafbdc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.175+0000 7f58ca7fc700 1 -- 172.21.15.39:0/3946732004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 msgr2=0x561cfafbbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.175+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafbbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.175+0000 7f58ca7fc700 1 -- 172.21.15.39:0/3946732004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cfad2ab00 con 0x561cfafb2950 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.175+0000 7f58ca7fc700 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafbe1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x561cfafa8790 tx=0x7f58c4004b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c3f38eba48a878 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.175+0000 7f58aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:48:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.176+0000 7f58aeff5700 1 -- 172.21.15.39:0/3946732004 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c4021040 con 0x561cfafb2950 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.176+0000 7f58aeff5700 1 -- 172.21.15.39:0/3946732004 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f58c4020070 con 0x561cfafb2950 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.176+0000 7f58dd4adb80 1 -- 172.21.15.39:0/3946732004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cfae56210 con 0x561cfafb2950 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.176+0000 7f58aeff5700 1 -- 172.21.15.39:0/3946732004 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58c4003c00 con 0x561cfafb2950 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.176+0000 7f58dd4adb80 10 client.25844 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.176+0000 7f58dd4adb80 1 -- 172.21.15.39:0/3946732004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561cfad82c70 con 0x561cfafb2950 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.177+0000 7f58aeff5700 1 -- 172.21.15.39:0/3946732004 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561cfae56210 con 0x561cfafb2950 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.177+0000 7f58aeff5700 1 -- 172.21.15.39:0/3946732004 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f58c4012910 con 0x561cfafb2950 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.177+0000 7f58dd4adb80 10 client.25844 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.177+0000 7f58dd4adb80 10 client.25844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.177+0000 7f58dd4adb80 20 client.25844 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.177+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561cfb01cdc0 0x561cfb01f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.177+0000 7f58dd4adb80 4 client.25844 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.177+0000 7f58dd4adb80 1 -- 172.21.15.39:0/3946732004 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561cfad4bc90 con 0x561cfb01cdc0 2024-05-24T11:48:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.177+0000 7f58caffd700 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561cfb01cdc0 0x561cfb01f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.178+0000 7f58caffd700 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561cfb01cdc0 0x561cfb01f1e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3cadec868a81722 server_cookie=6f43ddbc6417485d in_seq=0 out_seq=0 2024-05-24T11:48:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.178+0000 7f58aeff5700 10 client.25844 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.260+0000 7f58aeff5700 1 -- 172.21.15.39:0/3946732004 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561cfad4bc90 con 0x561cfb01cdc0 2024-05-24T11:48:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.260+0000 7f58aeff5700 10 client.25844 handle_command_reply: tid=0 2024-05-24T11:48:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.260+0000 7f58dd4adb80 1 client.25844 shutdown 2024-05-24T11:48:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.260+0000 7f58dd4adb80 1 -- 172.21.15.39:0/3946732004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 msgr2=0x561cfafbe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.260+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafbe1b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x561cfafa8790 tx=0x7f58c4004b20 comp rx=0 tx=0).stop 2024-05-24T11:48:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.260+0000 7f58dd4adb80 1 -- 172.21.15.39:0/3946732004 shutdown_connections 2024-05-24T11:48:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.260+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cfafad480 0x561cfafbdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.260+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x561cfb01cdc0 0x561cfb01f1e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.261+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cfafb3270 0x561cfafbbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.261+0000 7f58dd4adb80 1 --2- 172.21.15.39:0/3946732004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cfafb2950 0x561cfafbe1b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.261+0000 7f58dd4adb80 1 -- 172.21.15.39:0/3946732004 >> 172.21.15.39:0/3946732004 conn(0x561cfaf0b1b0 msgr2=0x561cfafcd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.261+0000 7f58aeff5700 0 client.25844 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.261+0000 7f58dd4adb80 1 -- 172.21.15.39:0/3946732004 shutdown_connections 2024-05-24T11:48:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.261+0000 7f58dd4adb80 1 -- 172.21.15.39:0/3946732004 wait complete. 2024-05-24T11:48:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.261+0000 7f58dd4adb80 20 client.25844 trim_cache size 0 max 16384 2024-05-24T11:48:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:48:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:48:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:48:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 -- 172.21.15.39:0/2645012577 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f58b40a13e0 msgr2=0x7f58b40a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f58b40a13e0 0x7f58b40a38a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58c00046e0 tx=0x7f58c000be30 comp rx=0 tx=0).stop 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 -- 172.21.15.39:0/2645012577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc1074e0 msgr2=0x7f58cc0812c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc1074e0 0x7f58cc0812c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0098e0 tx=0x7f58bc005740 comp rx=0 tx=0).stop 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 -- 172.21.15.39:0/2645012577 shutdown_connections 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58cc1074e0 0x7f58cc0812c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f58b40a13e0 0x7f58b40a38a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58cc10d5a0 0x7f58cc07b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 --2- 172.21.15.39:0/2645012577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58cc107ea0 0x7f58cc081800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 -- 172.21.15.39:0/2645012577 >> 172.21.15.39:0/2645012577 conn(0x7f58cc076dc0 msgr2=0x7f58cc077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.263+0000 7f58d12f2700 1 -- 172.21.15.39:0/2645012577 shutdown_connections 2024-05-24T11:48:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:46.264+0000 7f58d12f2700 1 -- 172.21.15.39:0/2645012577 wait complete. 2024-05-24T11:48:46.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:46 smithi053 ceph-mon[47448]: pgmap v328: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 24 MiB/s wr, 1.72k op/s 2024-05-24T11:48:46.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:46 smithi137 ceph-mon[47907]: pgmap v328: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 24 MiB/s wr, 1.72k op/s 2024-05-24T11:48:46.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:46 smithi039 ceph-mon[42919]: pgmap v328: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 24 MiB/s wr, 1.72k op/s 2024-05-24T11:48:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:48 smithi053 ceph-mon[47448]: pgmap v329: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-05-24T11:48:48.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:48 smithi137 ceph-mon[47907]: pgmap v329: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-05-24T11:48:48.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:48 smithi039 ceph-mon[42919]: pgmap v329: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-05-24T11:48:49.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 565.0 s, 889.6 tps, lat 111.813 ms stddev 36.878 2024-05-24T11:48:50.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:50 smithi053 ceph-mon[47448]: pgmap v330: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 22 MiB/s wr, 1.66k op/s 2024-05-24T11:48:50.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:50 smithi137 ceph-mon[47907]: pgmap v330: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 22 MiB/s wr, 1.66k op/s 2024-05-24T11:48:50.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:50 smithi039 ceph-mon[42919]: pgmap v330: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 22 MiB/s wr, 1.66k op/s 2024-05-24T11:48:52.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:52 smithi053 ceph-mon[47448]: pgmap v331: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-05-24T11:48:52.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:52 smithi137 ceph-mon[47907]: pgmap v331: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-05-24T11:48:52.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:52 smithi039 ceph-mon[42919]: pgmap v331: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-05-24T11:48:54.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 570.0 s, 914.4 tps, lat 109.493 ms stddev 37.167 2024-05-24T11:48:54.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:54 smithi053 ceph-mon[47448]: pgmap v332: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-05-24T11:48:54.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:54 smithi137 ceph-mon[47907]: pgmap v332: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-05-24T11:48:54.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:54 smithi039 ceph-mon[42919]: pgmap v332: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-05-24T11:48:56.493 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:56 smithi053 ceph-mon[47448]: pgmap v333: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 21 MiB/s wr, 1.67k op/s 2024-05-24T11:48:56.495 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:56 smithi137 ceph-mon[47907]: pgmap v333: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 21 MiB/s wr, 1.67k op/s 2024-05-24T11:48:56.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:56 smithi039 ceph-mon[42919]: pgmap v333: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 21 MiB/s wr, 1.67k op/s 2024-05-24T11:48:57.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:57 smithi039 ceph-mon[42919]: pgmap v334: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2024-05-24T11:48:57.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:57 smithi053 ceph-mon[47448]: pgmap v334: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2024-05-24T11:48:57.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:57 smithi137 ceph-mon[47907]: pgmap v334: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2024-05-24T11:48:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 575.0 s, 869.4 tps, lat 115.374 ms stddev 58.853 2024-05-24T11:48:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.437+0000 7f1e6f126700 1 -- 172.21.15.39:0/1912707148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e68105e60 msgr2=0x7f1e68106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.437+0000 7f1e6f126700 1 --2- 172.21.15.39:0/1912707148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e68105e60 0x7f1e68106240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c01a9a0 tx=0x7f1e5c01acb0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.438+0000 7f1e6f126700 1 -- 172.21.15.39:0/1912707148 shutdown_connections 2024-05-24T11:48:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.438+0000 7f1e6f126700 1 --2- 172.21.15.39:0/1912707148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e68105e60 0x7f1e68106240 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.438+0000 7f1e6f126700 1 --2- 172.21.15.39:0/1912707148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e68100760 0x7f1e6810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.438+0000 7f1e6f126700 1 --2- 172.21.15.39:0/1912707148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e680ffda0 0x7f1e68100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.438+0000 7f1e6f126700 1 -- 172.21.15.39:0/1912707148 >> 172.21.15.39:0/1912707148 conn(0x7f1e68076f10 msgr2=0x7f1e68077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.438+0000 7f1e6f126700 1 -- 172.21.15.39:0/1912707148 shutdown_connections 2024-05-24T11:48:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.438+0000 7f1e6f126700 1 -- 172.21.15.39:0/1912707148 wait complete. 2024-05-24T11:48:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e6f126700 1 Processor -- start 2024-05-24T11:48:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e6f126700 1 -- start start 2024-05-24T11:48:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e6f126700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e680ffda0 0x7f1e6810c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e6f126700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e68100760 0x7f1e6810c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e6f126700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e68105e60 0x7f1e6806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e6f126700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6806dd30 con 0x7f1e68105e60 2024-05-24T11:48:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e6f126700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6806dea0 con 0x7f1e68100760 2024-05-24T11:48:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e6f126700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6806e060 con 0x7f1e680ffda0 2024-05-24T11:48:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e67fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e68100760 0x7f1e6810c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e67fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e68100760 0x7f1e6810c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:53572/0 (socket says 172.21.15.39:53572) 2024-05-24T11:48:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.439+0000 7f1e67fff700 1 -- 172.21.15.39:0/538607355 learned_addr learned my addr 172.21.15.39:0/538607355 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.440+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e680ffda0 0x7f1e6810c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.440+0000 7f1e6d6c3700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e68105e60 0x7f1e6806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.440+0000 7f1e6d6c3700 1 -- 172.21.15.39:0/538607355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e680ffda0 msgr2=0x7f1e6810c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.440+0000 7f1e6d6c3700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e680ffda0 0x7f1e6810c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.440+0000 7f1e6d6c3700 1 -- 172.21.15.39:0/538607355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e68100760 msgr2=0x7f1e6810c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.440+0000 7f1e6d6c3700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e68100760 0x7f1e6810c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.440+0000 7f1e6d6c3700 1 -- 172.21.15.39:0/538607355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c01a600 con 0x7f1e68105e60 2024-05-24T11:48:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.441+0000 7f1e6d6c3700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e68105e60 0x7f1e6806d7f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000efc0 tx=0x7f1e6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.442+0000 7f1e65ffb700 1 -- 172.21.15.39:0/538607355 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60018070 con 0x7f1e68105e60 2024-05-24T11:48:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.442+0000 7f1e65ffb700 1 -- 172.21.15.39:0/538607355 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e6000f040 con 0x7f1e68105e60 2024-05-24T11:48:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.442+0000 7f1e65ffb700 1 -- 172.21.15.39:0/538607355 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000cc70 con 0x7f1e68105e60 2024-05-24T11:48:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.443+0000 7f1e6f126700 1 -- 172.21.15.39:0/538607355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6806e2e0 con 0x7f1e68105e60 2024-05-24T11:48:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.443+0000 7f1e6f126700 1 -- 172.21.15.39:0/538607355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e681bf8b0 con 0x7f1e68105e60 2024-05-24T11:48:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.448+0000 7f1e7b2e1b80 1 Processor -- start 2024-05-24T11:48:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.450+0000 7f1e65ffb700 1 -- 172.21.15.39:0/538607355 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1e60007500 con 0x7f1e68105e60 2024-05-24T11:48:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.451+0000 7f1e65ffb700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1e580a13e0 0x7f1e580a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.451+0000 7f1e65ffb700 1 -- 172.21.15.39:0/538607355 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1e600cbfa0 con 0x7f1e68105e60 2024-05-24T11:48:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.451+0000 7f1e67fff700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1e580a13e0 0x7f1e580a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.451+0000 7f1e7b2e1b80 1 -- start start 2024-05-24T11:48:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 0x55e6283dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283e0270 0x55e6283d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e67fff700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1e580a13e0 0x7f1e580a38a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e6806eed0 tx=0x7f1e54009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 0x55e6283da8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6283310f0 con 0x55e6283da480 2024-05-24T11:48:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62832eab0 con 0x55e6283df950 2024-05-24T11:48:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62832a3a0 con 0x55e6283e0270 2024-05-24T11:48:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e67fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283e0270 0x55e6283d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e67fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283e0270 0x55e6283d9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38206/0 (socket says 172.21.15.39:38206) 2024-05-24T11:48:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e67fff700 1 -- 172.21.15.39:0/1884202826 learned_addr learned my addr 172.21.15.39:0/1884202826 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/1884202826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 0x55e6283dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e67fff700 1 -- 172.21.15.39:0/1884202826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 msgr2=0x55e6283dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e67fff700 1 --2- 172.21.15.39:0/1884202826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 0x55e6283dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e67fff700 1 -- 172.21.15.39:0/1884202826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 msgr2=0x55e6283da8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e67fff700 1 --2- 172.21.15.39:0/1884202826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 0x55e6283da8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.452+0000 7f1e67fff700 1 -- 172.21.15.39:0/1884202826 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6281ab350 con 0x55e6283e0270 2024-05-24T11:48:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.453+0000 7f1e67fff700 1 --2- 172.21.15.39:0/1884202826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283e0270 0x55e6283d9f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400e570 tx=0x7f1e54014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e57c3a82e7e28fc server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.453+0000 7f1e52ffd700 1 -- 172.21.15.39:0/1884202826 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54026430 con 0x55e6283e0270 2024-05-24T11:48:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.453+0000 7f1e52ffd700 1 -- 172.21.15.39:0/1884202826 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e54021070 con 0x55e6283e0270 2024-05-24T11:48:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.453+0000 7f1e52ffd700 1 -- 172.21.15.39:0/1884202826 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5401d790 con 0x55e6283e0270 2024-05-24T11:48:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.454+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/1884202826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283e0270 msgr2=0x55e6283d9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.454+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/1884202826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283e0270 0x55e6283d9f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400e570 tx=0x7f1e54014000 comp rx=0 tx=0).stop 2024-05-24T11:48:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.455+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/1884202826 shutdown_connections 2024-05-24T11:48:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.455+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/1884202826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 0x55e6283da8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.455+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/1884202826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 0x55e6283dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.455+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/1884202826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283e0270 0x55e6283d9f40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.455+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/1884202826 >> 172.21.15.39:0/1884202826 conn(0x55e6283381b0 msgr2=0x55e6283d4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.455+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/1884202826 shutdown_connections 2024-05-24T11:48:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.455+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/1884202826 wait complete. 2024-05-24T11:48:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.456+0000 7f1e7b2e1b80 1 -- start start 2024-05-24T11:48:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.456+0000 7f1e7b2e1b80 1 Processor -- start 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.456+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283da480 0x55e6283e6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.456+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 0x55e6283e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 0x55e6283e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6282c6500 con 0x55e6283e0270 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6282c61a0 con 0x55e6283da480 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6281ad640 con 0x55e6283df950 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e7b2e1b80 10 client.0 _ll_register_callbacks cb 0x55e6282cfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6cec2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283da480 0x55e6283e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6cec2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283da480 0x55e6283e6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:53592/0 (socket says 172.21.15.39:53592) 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6cec2700 1 -- 172.21.15.39:0/3527232274 learned_addr learned my addr 172.21.15.39:0/3527232274 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e67fff700 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 0x55e6283e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6d6c3700 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 0x55e6283e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6cec2700 1 -- 172.21.15.39:0/3527232274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 msgr2=0x55e6283e3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 0x55e6283e3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6cec2700 1 -- 172.21.15.39:0/3527232274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 msgr2=0x55e6283e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 0x55e6283e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6cec2700 1 -- 172.21.15.39:0/3527232274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e54017070 con 0x55e6283da480 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.457+0000 7f1e6d6c3700 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 0x55e6283e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.458+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283da480 0x55e6283e6280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c026000 tx=0x7f1e5c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f008fd91fc7fdf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.458+0000 7f1e47fff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.458+0000 7f1e47fff700 1 -- 172.21.15.39:0/3527232274 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c01cee0 con 0x55e6283da480 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.458+0000 7f1e47fff700 1 -- 172.21.15.39:0/3527232274 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1e5c0050c0 con 0x55e6283da480 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.458+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3527232274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e628283210 con 0x55e6283da480 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e47fff700 1 -- 172.21.15.39:0/3527232274 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e5c005280 con 0x55e6283da480 2024-05-24T11:48:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e7b2e1b80 10 client.25856 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3527232274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e628157b00 con 0x55e6283da480 2024-05-24T11:48:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e47fff700 1 -- 172.21.15.39:0/3527232274 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e5c02c030 con 0x55e6283da480 2024-05-24T11:48:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e47fff700 1 -- 172.21.15.39:0/3527232274 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1e5c010460 con 0x55e6283da480 2024-05-24T11:48:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e7b2e1b80 10 client.25856 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e7b2e1b80 10 client.25856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e7b2e1b80 20 client.25856 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e6283f3be0 0x55e6283f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e7b2e1b80 4 client.25856 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:48:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.459+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3527232274 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6283a97b0 con 0x55e6283f3be0 2024-05-24T11:48:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.460+0000 7f1e67fff700 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e6283f3be0 0x55e6283f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.461+0000 7f1e47fff700 10 client.25856 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.461+0000 7f1e67fff700 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e6283f3be0 0x55e6283f6000 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c998cc7d7822e6 server_cookie=ecd069439a1fbd0f in_seq=0 out_seq=0 2024-05-24T11:48:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.461+0000 7f1e47fff700 1 -- 172.21.15.39:0/3527232274 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6283a97b0 con 0x55e6283f3be0 2024-05-24T11:48:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.461+0000 7f1e47fff700 10 client.25856 handle_command_reply: tid=0 2024-05-24T11:48:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.461+0000 7f1e7b2e1b80 1 client.25856 shutdown 2024-05-24T11:48:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3527232274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283da480 msgr2=0x55e6283e6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283da480 0x55e6283e6280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c026000 tx=0x7f1e5c005970 comp rx=0 tx=0).stop 2024-05-24T11:48:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3527232274 shutdown_connections 2024-05-24T11:48:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e6283f3be0 0x55e6283f6000 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 0x55e6283e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283da480 0x55e6283e6280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3527232274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 0x55e6283e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3527232274 >> 172.21.15.39:0/3527232274 conn(0x55e6283381b0 msgr2=0x55e6283de4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e47fff700 0 client.25856 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3527232274 shutdown_connections 2024-05-24T11:48:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3527232274 wait complete. 2024-05-24T11:48:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.462+0000 7f1e7b2e1b80 20 client.25856 trim_cache size 0 max 16384 2024-05-24T11:48:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.492+0000 7f1e7b2e1b80 1 Processor -- start 2024-05-24T11:48:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.492+0000 7f1e7b2e1b80 1 -- start start 2024-05-24T11:48:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.492+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 0x55e6283e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.492+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 0x55e6283da880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.492+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283e0270 0x55e6283e3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.492+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6283fb1f0 con 0x55e6283da480 2024-05-24T11:48:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.492+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6283d9160 con 0x55e6283e0270 2024-05-24T11:48:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.492+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6283dfd30 con 0x55e6283df950 2024-05-24T11:48:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6cec2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 0x55e6283e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6cec2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 0x55e6283e3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52846/0 (socket says 172.21.15.39:52846) 2024-05-24T11:48:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6cec2700 1 -- 172.21.15.39:0/3357059123 learned_addr learned my addr 172.21.15.39:0/3357059123 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e67fff700 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 0x55e6283da880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6d6c3700 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283e0270 0x55e6283e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6cec2700 1 -- 172.21.15.39:0/3357059123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 msgr2=0x55e6283da880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 0x55e6283da880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6cec2700 1 -- 172.21.15.39:0/3357059123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283e0270 msgr2=0x55e6283e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283e0270 0x55e6283e3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6cec2700 1 -- 172.21.15.39:0/3357059123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6282f0500 con 0x55e6283da480 2024-05-24T11:48:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.493+0000 7f1e6d6c3700 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283e0270 0x55e6283e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:48:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.494+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 0x55e6283e3cb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c01a8f0 tx=0x7f1e5c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6cf12934e9161b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.497+0000 7f1e537fe700 1 -- 172.21.15.39:0/3357059123 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c011ed0 con 0x55e6283da480 2024-05-24T11:48:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.497+0000 7f1e537fe700 1 -- 172.21.15.39:0/3357059123 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e5c02c030 con 0x55e6283da480 2024-05-24T11:48:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.497+0000 7f1e537fe700 1 -- 172.21.15.39:0/3357059123 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c003980 con 0x55e6283da480 2024-05-24T11:48:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.498+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3357059123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 msgr2=0x55e6283e3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:48:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.498+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 0x55e6283e3cb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c01a8f0 tx=0x7f1e5c01acb0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.498+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3357059123 shutdown_connections 2024-05-24T11:48:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.498+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283e0270 0x55e6283e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.498+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283da480 0x55e6283e3cb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.498+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/3357059123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283df950 0x55e6283da880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.498+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3357059123 >> 172.21.15.39:0/3357059123 conn(0x55e6283381b0 msgr2=0x55e6283e2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:48:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.499+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3357059123 shutdown_connections 2024-05-24T11:48:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.499+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/3357059123 wait complete. 2024-05-24T11:48:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.499+0000 7f1e7b2e1b80 1 -- start start 2024-05-24T11:48:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e7b2e1b80 1 Processor -- start 2024-05-24T11:48:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283da480 0x55e6283eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 0x55e6283de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e7b2e1b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 0x55e6283e2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6283df020 con 0x55e6283e0270 2024-05-24T11:48:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6283df190 con 0x55e6283df950 2024-05-24T11:48:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e7b2e1b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6283e2cd0 con 0x55e6283da480 2024-05-24T11:48:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e7b2e1b80 10 client.0 _ll_register_callbacks cb 0x55e6282cfad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:48:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e6cec2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283da480 0x55e6283eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e6cec2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283da480 0x55e6283eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38232/0 (socket says 172.21.15.39:38232) 2024-05-24T11:48:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.500+0000 7f1e6cec2700 1 -- 172.21.15.39:0/370452800 learned_addr learned my addr 172.21.15.39:0/370452800 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:48:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e6d6c3700 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 0x55e6283e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e67fff700 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 0x55e6283de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e6cec2700 1 -- 172.21.15.39:0/370452800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 msgr2=0x55e6283de7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 0x55e6283de7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e6cec2700 1 -- 172.21.15.39:0/370452800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 msgr2=0x55e6283e2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:48:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 0x55e6283e2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:48:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e6cec2700 1 -- 172.21.15.39:0/370452800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e628157b00 con 0x55e6283da480 2024-05-24T11:48:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e6cec2700 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283da480 0x55e6283eb0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c010910 tx=0x7f1e5c010c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2efc8855cec41e5b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:48:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e47fff700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:48:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e47fff700 1 -- 172.21.15.39:0/370452800 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c01cee0 con 0x55e6283da480 2024-05-24T11:48:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e47fff700 1 -- 172.21.15.39:0/370452800 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1e5c000e40 con 0x55e6283da480 2024-05-24T11:48:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.501+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/370452800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e54021070 con 0x55e6283da480 2024-05-24T11:48:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.502+0000 7f1e47fff700 1 -- 172.21.15.39:0/370452800 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e5c0050b0 con 0x55e6283da480 2024-05-24T11:48:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.502+0000 7f1e7b2e1b80 10 client.25771 fetch_fsmap learned FSMap version 29 2024-05-24T11:48:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.502+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/370452800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e6281afc70 con 0x55e6283da480 2024-05-24T11:48:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.502+0000 7f1e47fff700 1 -- 172.21.15.39:0/370452800 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e54021070 con 0x55e6283da480 2024-05-24T11:48:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.502+0000 7f1e47fff700 1 -- 172.21.15.39:0/370452800 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1e5c003c60 con 0x55e6283da480 2024-05-24T11:48:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.503+0000 7f1e7b2e1b80 10 client.25771 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:48:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.503+0000 7f1e7b2e1b80 10 client.25771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:48:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.503+0000 7f1e7b2e1b80 20 client.25771 populate_metadata read hostname 'smithi039' 2024-05-24T11:48:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.503+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e628452150 0x55e628454540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:48:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.503+0000 7f1e7b2e1b80 4 client.25771 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:48:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.503+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/370452800 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e628322e60 con 0x55e628452150 2024-05-24T11:48:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.503+0000 7f1e67fff700 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e628452150 0x55e628454540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:48:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.504+0000 7f1e67fff700 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e628452150 0x55e628454540 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff40a2a3dc02938 server_cookie=99856b9426a07f72 in_seq=0 out_seq=0 2024-05-24T11:48:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:48:59.504+0000 7f1e47fff700 10 client.25771 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:48:59.739 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:48:59 smithi053 ceph-mon[47448]: pgmap v335: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 20 MiB/s wr, 1.65k op/s 2024-05-24T11:48:59.745 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:48:59 smithi137 ceph-mon[47907]: pgmap v335: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 20 MiB/s wr, 1.65k op/s 2024-05-24T11:48:59.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:48:59 smithi039 ceph-mon[42919]: pgmap v335: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 20 MiB/s wr, 1.65k op/s 2024-05-24T11:49:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.377+0000 7f1e47fff700 1 -- 172.21.15.39:0/370452800 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e628322e60 con 0x55e628452150 2024-05-24T11:49:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.377+0000 7f1e47fff700 10 client.25771 handle_command_reply: tid=0 2024-05-24T11:49:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.377+0000 7f1e7b2e1b80 1 client.25771 shutdown 2024-05-24T11:49:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/370452800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283da480 msgr2=0x55e6283eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283da480 0x55e6283eb0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c010910 tx=0x7f1e5c010c40 comp rx=0 tx=0).stop 2024-05-24T11:49:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/370452800 shutdown_connections 2024-05-24T11:49:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6283da480 0x55e6283eb0a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55e628452150 0x55e628454540 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6283e0270 0x55e6283e2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 --2- 172.21.15.39:0/370452800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e6283df950 0x55e6283de7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/370452800 >> 172.21.15.39:0/370452800 conn(0x55e6283381b0 msgr2=0x55e6283d9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e47fff700 0 client.25771 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/370452800 shutdown_connections 2024-05-24T11:49:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 1 -- 172.21.15.39:0/370452800 wait complete. 2024-05-24T11:49:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.379+0000 7f1e7b2e1b80 20 client.25771 trim_cache size 0 max 16384 2024-05-24T11:49:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:49:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:49:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:49:00.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 -- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1e580a13e0 msgr2=0x7f1e580a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1e580a13e0 0x7f1e580a38a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e6806eed0 tx=0x7f1e54009260 comp rx=0 tx=0).stop 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 -- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e68105e60 msgr2=0x7f1e6806d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e68105e60 0x7f1e6806d7f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000efc0 tx=0x7f1e6000c3a0 comp rx=0 tx=0).stop 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 -- 172.21.15.39:0/538607355 shutdown_connections 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e680ffda0 0x7f1e6810c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f1e580a13e0 0x7f1e580a38a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e68105e60 0x7f1e6806d7f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 --2- 172.21.15.39:0/538607355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e68100760 0x7f1e6810c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 -- 172.21.15.39:0/538607355 >> 172.21.15.39:0/538607355 conn(0x7f1e68076f10 msgr2=0x7f1e680fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.380+0000 7f1e6f126700 1 -- 172.21.15.39:0/538607355 shutdown_connections 2024-05-24T11:49:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:00.381+0000 7f1e6f126700 1 -- 172.21.15.39:0/538607355 wait complete. 2024-05-24T11:49:01.740 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:01 smithi053 ceph-mon[47448]: pgmap v336: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-05-24T11:49:01.748 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:01 smithi137 ceph-mon[47907]: pgmap v336: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-05-24T11:49:01.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:01 smithi039 ceph-mon[42919]: pgmap v336: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-05-24T11:49:03.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:03 smithi039 ceph-mon[42919]: pgmap v337: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-05-24T11:49:03.990 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:03 smithi053 ceph-mon[47448]: pgmap v337: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-05-24T11:49:03.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:03 smithi137 ceph-mon[47907]: pgmap v337: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-05-24T11:49:04.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 580.0 s, 858.6 tps, lat 113.429 ms stddev 57.381 2024-05-24T11:49:04.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:04 smithi039 ceph-mon[42919]: osdmap e90: 12 total, 12 up, 12 in 2024-05-24T11:49:04.989 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:04 smithi053 ceph-mon[47448]: osdmap e90: 12 total, 12 up, 12 in 2024-05-24T11:49:04.995 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:04 smithi137 ceph-mon[47907]: osdmap e90: 12 total, 12 up, 12 in 2024-05-24T11:49:06.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:05 smithi039 ceph-mon[42919]: pgmap v339: 97 pgs: 97 active+clean; 5.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-05-24T11:49:06.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:05 smithi053 ceph-mon[47448]: pgmap v339: 97 pgs: 97 active+clean; 5.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-05-24T11:49:06.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:05 smithi137 ceph-mon[47907]: pgmap v339: 97 pgs: 97 active+clean; 5.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-05-24T11:49:08.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:07 smithi039 ceph-mon[42919]: pgmap v340: 97 pgs: 97 active+clean; 5.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-05-24T11:49:08.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:07 smithi053 ceph-mon[47448]: pgmap v340: 97 pgs: 97 active+clean; 5.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-05-24T11:49:08.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:07 smithi137 ceph-mon[47907]: pgmap v340: 97 pgs: 97 active+clean; 5.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2024-05-24T11:49:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 585.0 s, 869.2 tps, lat 117.572 ms stddev 244.382 2024-05-24T11:49:10.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:09 smithi039 ceph-mon[42919]: pgmap v341: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 16 MiB/s wr, 1.34k op/s 2024-05-24T11:49:10.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:09 smithi039 ceph-mon[42919]: osdmap e91: 12 total, 12 up, 12 in 2024-05-24T11:49:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:09 smithi053 ceph-mon[47448]: pgmap v341: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 16 MiB/s wr, 1.34k op/s 2024-05-24T11:49:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:09 smithi053 ceph-mon[47448]: osdmap e91: 12 total, 12 up, 12 in 2024-05-24T11:49:10.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:09 smithi137 ceph-mon[47907]: pgmap v341: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 16 MiB/s wr, 1.34k op/s 2024-05-24T11:49:10.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:09 smithi137 ceph-mon[47907]: osdmap e91: 12 total, 12 up, 12 in 2024-05-24T11:49:12.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:11 smithi039 ceph-mon[42919]: pgmap v343: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-05-24T11:49:12.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:11 smithi053 ceph-mon[47448]: pgmap v343: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-05-24T11:49:12.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:11 smithi137 ceph-mon[47907]: pgmap v343: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-05-24T11:49:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.563+0000 7f7526507700 1 -- 172.21.15.39:0/3645936142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520105e60 msgr2=0x7f7520106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.563+0000 7f7526507700 1 --2- 172.21.15.39:0/3645936142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520105e60 0x7f7520106240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7518009f90 tx=0x7f751800b330 comp rx=0 tx=0).stop 2024-05-24T11:49:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.563+0000 7f7526507700 1 -- 172.21.15.39:0/3645936142 shutdown_connections 2024-05-24T11:49:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.563+0000 7f7526507700 1 --2- 172.21.15.39:0/3645936142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520105e60 0x7f7520106240 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7518009f90 tx=0x7f751800b330 comp rx=0 tx=0).stop 2024-05-24T11:49:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.563+0000 7f7526507700 1 --2- 172.21.15.39:0/3645936142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7520100760 0x7f752010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.563+0000 7f7526507700 1 --2- 172.21.15.39:0/3645936142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75200ffda0 0x7f7520100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.563+0000 7f7526507700 1 -- 172.21.15.39:0/3645936142 >> 172.21.15.39:0/3645936142 conn(0x7f7520076f10 msgr2=0x7f7520077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.563+0000 7f7526507700 1 -- 172.21.15.39:0/3645936142 shutdown_connections 2024-05-24T11:49:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.564+0000 7f7526507700 1 -- 172.21.15.39:0/3645936142 wait complete. 2024-05-24T11:49:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.564+0000 7f7526507700 1 Processor -- start 2024-05-24T11:49:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.564+0000 7f7526507700 1 -- start start 2024-05-24T11:49:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f7526507700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75200ffda0 0x7f7520072a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f7526507700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520100760 0x7f752006bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f7526507700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f752006c090 0x7f752006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f7526507700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7520070770 con 0x7f752006c090 2024-05-24T11:49:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f7526507700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75200708e0 con 0x7f75200ffda0 2024-05-24T11:49:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f7526507700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7520070aa0 con 0x7f7520100760 2024-05-24T11:49:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f751f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520100760 0x7f752006bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f751f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520100760 0x7f752006bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59390/0 (socket says 172.21.15.39:59390) 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f7524aa4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f752006c090 0x7f752006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f751f7fe700 1 -- 172.21.15.39:0/4223601858 learned_addr learned my addr 172.21.15.39:0/4223601858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f751f7fe700 1 -- 172.21.15.39:0/4223601858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75200ffda0 msgr2=0x7f7520072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f751ffff700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75200ffda0 0x7f7520072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f751f7fe700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75200ffda0 0x7f7520072a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f751f7fe700 1 -- 172.21.15.39:0/4223601858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f752006c090 msgr2=0x7f752006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f751f7fe700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f752006c090 0x7f752006c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.565+0000 7f751f7fe700 1 -- 172.21.15.39:0/4223601858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7518009c40 con 0x7f7520100760 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.566+0000 7f751f7fe700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520100760 0x7f752006bb50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f751400e9c0 tx=0x7f751400ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.566+0000 7f751d7fa700 1 -- 172.21.15.39:0/4223601858 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751400cbd0 con 0x7f7520100760 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.566+0000 7f751d7fa700 1 -- 172.21.15.39:0/4223601858 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f751400cd30 con 0x7f7520100760 2024-05-24T11:49:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.566+0000 7f751d7fa700 1 -- 172.21.15.39:0/4223601858 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514005520 con 0x7f7520100760 2024-05-24T11:49:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.568+0000 7f7526507700 1 -- 172.21.15.39:0/4223601858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7520070d80 con 0x7f7520100760 2024-05-24T11:49:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.568+0000 7f7526507700 1 -- 172.21.15.39:0/4223601858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7520071210 con 0x7f7520100760 2024-05-24T11:49:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.571+0000 7f75326c2b80 1 Processor -- start 2024-05-24T11:49:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.571+0000 7f75326c2b80 1 -- start start 2024-05-24T11:49:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.571+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b1d950 0x558df5b1dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.571+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1e270 0x558df5b17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.571+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 0x558df5b188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.571+0000 7f75326c2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5a6f0f0 con 0x558df5b1d950 2024-05-24T11:49:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.571+0000 7f75326c2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5a6cab0 con 0x558df5b1e270 2024-05-24T11:49:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.571+0000 7f75326c2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5a683a0 con 0x558df5b18480 2024-05-24T11:49:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1e270 0x558df5b17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1e270 0x558df5b17f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:48882/0 (socket says 172.21.15.39:48882) 2024-05-24T11:49:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751ffff700 1 -- 172.21.15.39:0/3321504011 learned_addr learned my addr 172.21.15.39:0/3321504011 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751ffff700 1 -- 172.21.15.39:0/3321504011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 msgr2=0x558df5b188a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:49:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751ffff700 1 --2- 172.21.15.39:0/3321504011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 0x558df5b188a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751ffff700 1 -- 172.21.15.39:0/3321504011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b1d950 msgr2=0x558df5b1dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751ffff700 1 --2- 172.21.15.39:0/3321504011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b1d950 0x558df5b1dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751ffff700 1 -- 172.21.15.39:0/3321504011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558df58e9350 con 0x558df5b1e270 2024-05-24T11:49:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751d7fa700 1 -- 172.21.15.39:0/4223601858 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7514004750 con 0x7f7520100760 2024-05-24T11:49:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.572+0000 7f751ffff700 1 --2- 172.21.15.39:0/3321504011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1e270 0x558df5b17f40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7518009040 tx=0x7f7518009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3871c1088965b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.573+0000 7f75067fc700 1 -- 172.21.15.39:0/3321504011 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751800e050 con 0x558df5b1e270 2024-05-24T11:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.573+0000 7f75067fc700 1 -- 172.21.15.39:0/3321504011 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7518009ac0 con 0x558df5b1e270 2024-05-24T11:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.573+0000 7f75067fc700 1 -- 172.21.15.39:0/3321504011 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518007bc0 con 0x558df5b1e270 2024-05-24T11:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.573+0000 7f751d7fa700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f75080a1290 0x7f75080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.573+0000 7f751d7fa700 1 -- 172.21.15.39:0/4223601858 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7514014070 con 0x7f7520100760 2024-05-24T11:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 -- 172.21.15.39:0/3321504011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1e270 msgr2=0x558df5b17f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3321504011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1e270 0x558df5b17f40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7518009040 tx=0x7f7518009360 comp rx=0 tx=0).stop 2024-05-24T11:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 -- 172.21.15.39:0/3321504011 shutdown_connections 2024-05-24T11:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3321504011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 0x558df5b188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3321504011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b1d950 0x558df5b1dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3321504011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1e270 0x558df5b17f40 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7518009040 tx=0x7f7518009360 comp rx=0 tx=0).stop 2024-05-24T11:49:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 -- 172.21.15.39:0/3321504011 >> 172.21.15.39:0/3321504011 conn(0x558df5a761b0 msgr2=0x558df5b12840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 -- 172.21.15.39:0/3321504011 shutdown_connections 2024-05-24T11:49:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f7524aa4700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f75080a1290 0x7f75080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 -- 172.21.15.39:0/3321504011 wait complete. 2024-05-24T11:49:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.574+0000 7f75326c2b80 1 -- start start 2024-05-24T11:49:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.575+0000 7f75326c2b80 1 Processor -- start 2024-05-24T11:49:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b18480 0x558df5b21840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b21d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b2dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f75326c2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5a04500 con 0x558df5b222c0 2024-05-24T11:49:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f75326c2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5a041a0 con 0x558df5b18480 2024-05-24T11:49:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f75326c2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df58eb640 con 0x558df5b1d950 2024-05-24T11:49:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f75326c2b80 10 client.0 _ll_register_callbacks cb 0x558df5a0dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f751f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b21d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f751f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b21d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59402/0 (socket says 172.21.15.39:59402) 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.576+0000 7f751f7fe700 1 -- 172.21.15.39:0/3919749677 learned_addr learned my addr 172.21.15.39:0/3919749677 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f751f7fe700 1 -- 172.21.15.39:0/3919749677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b18480 msgr2=0x558df5b21840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f751f7fe700 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b18480 0x558df5b21840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f751f7fe700 1 -- 172.21.15.39:0/3919749677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 msgr2=0x558df5b2dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f751f7fe700 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b2dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f751f7fe700 1 -- 172.21.15.39:0/3919749677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752010d4f0 con 0x558df5b1d950 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f751f7fe700 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b21d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f75140ca7e0 tx=0x7f751409a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b92cbc646b58adaf server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f74ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:49:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f74ff7fe700 1 -- 172.21.15.39:0/3919749677 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75140a25c0 con 0x558df5b1d950 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f74ff7fe700 1 -- 172.21.15.39:0/3919749677 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f75140b8300 con 0x558df5b1d950 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f75326c2b80 1 -- 172.21.15.39:0/3919749677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558df5ae77b0 con 0x558df5b1d950 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f74ff7fe700 1 -- 172.21.15.39:0/3919749677 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75140b84c0 con 0x558df5b1d950 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f7524aa4700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f75080a1290 0x7f75080a3750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7510004790 tx=0x7f751000b470 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f75326c2b80 10 client.25780 fetch_fsmap learned FSMap version 29 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.577+0000 7f75326c2b80 1 -- 172.21.15.39:0/3919749677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558df5895b00 con 0x558df5b1d950 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.578+0000 7f74ff7fe700 1 -- 172.21.15.39:0/3919749677 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558df5ae77b0 con 0x558df5b1d950 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.578+0000 7f74ff7fe700 1 -- 172.21.15.39:0/3919749677 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75140996a0 con 0x558df5b1d950 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.578+0000 7f75326c2b80 10 client.25780 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.578+0000 7f75326c2b80 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.578+0000 7f75326c2b80 20 client.25780 populate_metadata read hostname 'smithi039' 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.578+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558df5b32210 0x558df5b34630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.578+0000 7f75326c2b80 4 client.25780 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:49:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.578+0000 7f75326c2b80 1 -- 172.21.15.39:0/3919749677 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558df5a60e60 con 0x558df5b32210 2024-05-24T11:49:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.578+0000 7f751ffff700 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558df5b32210 0x558df5b34630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.579+0000 7f751ffff700 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558df5b32210 0x558df5b34630 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ca44af3ba2cb1 server_cookie=240ab0996375ff7e in_seq=0 out_seq=0 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.579+0000 7f74ff7fe700 10 client.25780 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f74ff7fe700 1 -- 172.21.15.39:0/3919749677 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558df5a60e60 con 0x558df5b32210 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f74ff7fe700 10 client.25780 handle_command_reply: tid=0 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 client.25780 shutdown 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 -- 172.21.15.39:0/3919749677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 msgr2=0x558df5b21d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b21d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f75140ca7e0 tx=0x7f751409a8f0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 -- 172.21.15.39:0/3919749677 shutdown_connections 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558df5b32210 0x558df5b34630 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b2dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b18480 0x558df5b21840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 --2- 172.21.15.39:0/3919749677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b21d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 -- 172.21.15.39:0/3919749677 >> 172.21.15.39:0/3919749677 conn(0x558df5a761b0 msgr2=0x558df5b1c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f74ff7fe700 0 client.25780 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 -- 172.21.15.39:0/3919749677 shutdown_connections 2024-05-24T11:49:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.580+0000 7f75326c2b80 1 -- 172.21.15.39:0/3919749677 wait complete. 2024-05-24T11:49:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.581+0000 7f75326c2b80 20 client.25780 trim_cache size 0 max 16384 2024-05-24T11:49:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.603+0000 7f75326c2b80 1 Processor -- start 2024-05-24T11:49:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f75326c2b80 1 -- start start 2024-05-24T11:49:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 0x558df5b2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1d950 0x558df5b12910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b15f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f75326c2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5b22e40 con 0x558df5b222c0 2024-05-24T11:49:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f75326c2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5b227e0 con 0x558df5b1d950 2024-05-24T11:49:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f75326c2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5b16470 con 0x558df5b18480 2024-05-24T11:49:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f751ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 0x558df5b2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f751ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 0x558df5b2e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59412/0 (socket says 172.21.15.39:59412) 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f751ffff700 1 -- 172.21.15.39:0/1221101835 learned_addr learned my addr 172.21.15.39:0/1221101835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f751f7fe700 1 --2- 172.21.15.39:0/1221101835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1d950 0x558df5b12910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f751ffff700 1 -- 172.21.15.39:0/1221101835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1d950 msgr2=0x558df5b12910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f751ffff700 1 --2- 172.21.15.39:0/1221101835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1d950 0x558df5b12910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f751ffff700 1 -- 172.21.15.39:0/1221101835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 msgr2=0x558df5b15f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f751ffff700 1 --2- 172.21.15.39:0/1221101835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b15f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.604+0000 7f751ffff700 1 -- 172.21.15.39:0/1221101835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558df5a2e500 con 0x558df5b18480 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.605+0000 7f751ffff700 1 --2- 172.21.15.39:0/1221101835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 0x558df5b2e2a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7518009790 tx=0x7f751800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc310f1620ea004c server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.605+0000 7f7506ffd700 1 -- 172.21.15.39:0/1221101835 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751800e040 con 0x558df5b18480 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.605+0000 7f7506ffd700 1 -- 172.21.15.39:0/1221101835 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f751800b700 con 0x558df5b18480 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.605+0000 7f7506ffd700 1 -- 172.21.15.39:0/1221101835 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518007ce0 con 0x558df5b18480 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.606+0000 7f75326c2b80 1 -- 172.21.15.39:0/1221101835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 msgr2=0x558df5b2e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.606+0000 7f75326c2b80 1 --2- 172.21.15.39:0/1221101835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 0x558df5b2e2a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7518009790 tx=0x7f751800b330 comp rx=0 tx=0).stop 2024-05-24T11:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.606+0000 7f75326c2b80 1 -- 172.21.15.39:0/1221101835 shutdown_connections 2024-05-24T11:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.606+0000 7f75326c2b80 1 --2- 172.21.15.39:0/1221101835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b18480 0x558df5b2e2a0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7518009790 tx=0x7f751800b330 comp rx=0 tx=0).stop 2024-05-24T11:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.606+0000 7f75326c2b80 1 --2- 172.21.15.39:0/1221101835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b15f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.606+0000 7f75326c2b80 1 --2- 172.21.15.39:0/1221101835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b1d950 0x558df5b12910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.606+0000 7f75326c2b80 1 -- 172.21.15.39:0/1221101835 >> 172.21.15.39:0/1221101835 conn(0x558df5a761b0 msgr2=0x558df5b2d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.606+0000 7f75326c2b80 1 -- 172.21.15.39:0/1221101835 shutdown_connections 2024-05-24T11:49:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.606+0000 7f75326c2b80 1 -- 172.21.15.39:0/1221101835 wait complete. 2024-05-24T11:49:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.607+0000 7f75326c2b80 1 -- start start 2024-05-24T11:49:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.607+0000 7f75326c2b80 1 Processor -- start 2024-05-24T11:49:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.607+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b29040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.607+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b26bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f75326c2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b27130 0x558df5b275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f75326c2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5b877f0 con 0x558df5b222c0 2024-05-24T11:49:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f75326c2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5b87960 con 0x558df5b27130 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f75326c2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5b87b20 con 0x558df5b1d950 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f75326c2b80 10 client.0 _ll_register_callbacks cb 0x558df5a0dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f7524aa4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b27130 0x558df5b275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f7524aa4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b27130 0x558df5b275b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:48896/0 (socket says 172.21.15.39:48896) 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f7524aa4700 1 -- 172.21.15.39:0/2435335520 learned_addr learned my addr 172.21.15.39:0/2435335520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f751f7fe700 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b26bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f751ffff700 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b29040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f7524aa4700 1 -- 172.21.15.39:0/2435335520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 msgr2=0x558df5b29040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f7524aa4700 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b29040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f7524aa4700 1 -- 172.21.15.39:0/2435335520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 msgr2=0x558df5b26bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.608+0000 7f7524aa4700 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b26bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f7524aa4700 1 -- 172.21.15.39:0/2435335520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558df58e9350 con 0x558df5b27130 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f751f7fe700 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b26bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f7524aa4700 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b27130 0x558df5b275b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f751000f080 tx=0x7f751001b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=234b4fdc3da35118 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f74ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f74ff7fe700 1 -- 172.21.15.39:0/2435335520 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7510012040 con 0x558df5b27130 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f74ff7fe700 1 -- 172.21.15.39:0/2435335520 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f751001be30 con 0x558df5b27130 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f75326c2b80 1 -- 172.21.15.39:0/2435335520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558df59c1210 con 0x558df5b27130 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f74ff7fe700 1 -- 172.21.15.39:0/2435335520 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f751000be20 con 0x558df5b27130 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f75326c2b80 10 client.25874 fetch_fsmap learned FSMap version 29 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.609+0000 7f75326c2b80 1 -- 172.21.15.39:0/2435335520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558df58edc70 con 0x558df5b27130 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.610+0000 7f74ff7fe700 1 -- 172.21.15.39:0/2435335520 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7510031080 con 0x558df5b27130 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.610+0000 7f74ff7fe700 1 -- 172.21.15.39:0/2435335520 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f751001c410 con 0x558df5b27130 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.610+0000 7f75326c2b80 10 client.25874 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.610+0000 7f75326c2b80 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.610+0000 7f75326c2b80 20 client.25874 populate_metadata read hostname 'smithi039' 2024-05-24T11:49:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.610+0000 7f75326c2b80 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558df5b892a0 0x558df5b8b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.610+0000 7f75326c2b80 4 client.25874 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:49:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.610+0000 7f75326c2b80 1 -- 172.21.15.39:0/2435335520 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558df58b6c90 con 0x558df5b892a0 2024-05-24T11:49:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.610+0000 7f751ffff700 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558df5b892a0 0x558df5b8b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.611+0000 7f751ffff700 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558df5b892a0 0x558df5b8b6c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a82538de26dec3d6 server_cookie=9508e6ed22c06bec in_seq=0 out_seq=0 2024-05-24T11:49:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.611+0000 7f74ff7fe700 10 client.25874 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.639+0000 7f74ff7fe700 1 -- 172.21.15.39:0/2435335520 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558df58b6c90 con 0x558df5b892a0 2024-05-24T11:49:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.639+0000 7f74ff7fe700 10 client.25874 handle_command_reply: tid=0 2024-05-24T11:49:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.639+0000 7f75326c2b80 1 client.25874 shutdown 2024-05-24T11:49:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.640+0000 7f75326c2b80 1 -- 172.21.15.39:0/2435335520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b27130 msgr2=0x558df5b275b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.640+0000 7f75326c2b80 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b27130 0x558df5b275b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f751000f080 tx=0x7f751001b900 comp rx=0 tx=0).stop 2024-05-24T11:49:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f75326c2b80 1 -- 172.21.15.39:0/2435335520 shutdown_connections 2024-05-24T11:49:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f75326c2b80 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558df5b1d950 0x558df5b29040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f75326c2b80 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558df5b892a0 0x558df5b8b6c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f75326c2b80 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558df5b27130 0x558df5b275b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f75326c2b80 1 --2- 172.21.15.39:0/2435335520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df5b222c0 0x558df5b26bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f75326c2b80 1 -- 172.21.15.39:0/2435335520 >> 172.21.15.39:0/2435335520 conn(0x558df5a761b0 msgr2=0x558df5b178a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f74ff7fe700 0 client.25874 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f75326c2b80 1 -- 172.21.15.39:0/2435335520 shutdown_connections 2024-05-24T11:49:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f75326c2b80 1 -- 172.21.15.39:0/2435335520 wait complete. 2024-05-24T11:49:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.645+0000 7f75326c2b80 20 client.25874 trim_cache size 0 max 16384 2024-05-24T11:49:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:49:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:49:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:49:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:49:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 -- 172.21.15.39:0/4223601858 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f75080a1290 msgr2=0x7f75080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f75080a1290 0x7f75080a3750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7510004790 tx=0x7f751000b470 comp rx=0 tx=0).stop 2024-05-24T11:49:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 -- 172.21.15.39:0/4223601858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520100760 msgr2=0x7f752006bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520100760 0x7f752006bb50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f751400e9c0 tx=0x7f751400ed80 comp rx=0 tx=0).stop 2024-05-24T11:49:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 -- 172.21.15.39:0/4223601858 shutdown_connections 2024-05-24T11:49:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f75080a1290 0x7f75080a3750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f752006c090 0x7f752006c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75200ffda0 0x7f7520072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 --2- 172.21.15.39:0/4223601858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7520100760 0x7f752006bb50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 -- 172.21.15.39:0/4223601858 >> 172.21.15.39:0/4223601858 conn(0x7f7520076f10 msgr2=0x7f75200feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.646+0000 7f7526507700 1 -- 172.21.15.39:0/4223601858 shutdown_connections 2024-05-24T11:49:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:13.647+0000 7f7526507700 1 -- 172.21.15.39:0/4223601858 wait complete. 2024-05-24T11:49:14.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:13 smithi039 ceph-mon[42919]: pgmap v344: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 8.6 MiB/s wr, 700 op/s 2024-05-24T11:49:14.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 590.0 s, 957.0 tps, lat 104.725 ms stddev 34.528 2024-05-24T11:49:14.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:13 smithi053 ceph-mon[47448]: pgmap v344: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 8.6 MiB/s wr, 700 op/s 2024-05-24T11:49:14.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:13 smithi137 ceph-mon[47907]: pgmap v344: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 8.6 MiB/s wr, 700 op/s 2024-05-24T11:49:16.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:15 smithi039 ceph-mon[42919]: pgmap v345: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:49:16.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:15 smithi053 ceph-mon[47448]: pgmap v345: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:49:16.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:15 smithi137 ceph-mon[47907]: pgmap v345: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:49:18.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:17 smithi039 ceph-mon[42919]: pgmap v346: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:49:18.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:17 smithi053 ceph-mon[47448]: pgmap v346: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:49:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:17 smithi137 ceph-mon[47907]: pgmap v346: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-05-24T11:49:19.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 595.0 s, 897.0 tps, lat 111.701 ms stddev 73.335 2024-05-24T11:49:20.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:19 smithi039 ceph-mon[42919]: pgmap v347: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 16 MiB/s wr, 1.40k op/s 2024-05-24T11:49:20.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:19 smithi053 ceph-mon[47448]: pgmap v347: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 16 MiB/s wr, 1.40k op/s 2024-05-24T11:49:20.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:19 smithi137 ceph-mon[47907]: pgmap v347: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 16 MiB/s wr, 1.40k op/s 2024-05-24T11:49:22.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:21 smithi039 ceph-mon[42919]: pgmap v348: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2024-05-24T11:49:22.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:21 smithi053 ceph-mon[47448]: pgmap v348: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2024-05-24T11:49:22.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:21 smithi137 ceph-mon[47907]: pgmap v348: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2024-05-24T11:49:24.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:23 smithi039 ceph-mon[42919]: pgmap v349: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 13 MiB/s wr, 1.17k op/s 2024-05-24T11:49:24.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 600.0 s, 895.4 tps, lat 111.399 ms stddev 130.099 2024-05-24T11:49:24.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:23 smithi053 ceph-mon[47448]: pgmap v349: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 13 MiB/s wr, 1.17k op/s 2024-05-24T11:49:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:23 smithi137 ceph-mon[47907]: pgmap v349: 97 pgs: 97 active+clean; 5.5 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 13 MiB/s wr, 1.17k op/s 2024-05-24T11:49:26.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:25 smithi039 ceph-mon[42919]: pgmap v350: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 20 MiB/s wr, 1.75k op/s 2024-05-24T11:49:26.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:25 smithi053 ceph-mon[47448]: pgmap v350: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 20 MiB/s wr, 1.75k op/s 2024-05-24T11:49:26.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:25 smithi137 ceph-mon[47907]: pgmap v350: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 20 MiB/s wr, 1.75k op/s 2024-05-24T11:49:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.821+0000 7f53677e2700 1 -- 172.21.15.39:0/2249937862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f536010d5a0 msgr2=0x7f536010d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.821+0000 7f53677e2700 1 --2- 172.21.15.39:0/2249937862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f536010d5a0 0x7f536010d980 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f535c0099c0 tx=0x7f535c009cd0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.821+0000 7f53677e2700 1 -- 172.21.15.39:0/2249937862 shutdown_connections 2024-05-24T11:49:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.821+0000 7f53677e2700 1 --2- 172.21.15.39:0/2249937862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5360107ea0 0x7f5360114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.821+0000 7f53677e2700 1 --2- 172.21.15.39:0/2249937862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f536010d5a0 0x7f536010d980 secure :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f535c0099c0 tx=0x7f535c009cd0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.821+0000 7f53677e2700 1 --2- 172.21.15.39:0/2249937862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53601074e0 0x7f5360107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.821+0000 7f53677e2700 1 -- 172.21.15.39:0/2249937862 >> 172.21.15.39:0/2249937862 conn(0x7f5360076dc0 msgr2=0x7f5360105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.821+0000 7f53677e2700 1 -- 172.21.15.39:0/2249937862 shutdown_connections 2024-05-24T11:49:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.821+0000 7f53677e2700 1 -- 172.21.15.39:0/2249937862 wait complete. 2024-05-24T11:49:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.822+0000 7f53677e2700 1 Processor -- start 2024-05-24T11:49:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.822+0000 7f53677e2700 1 -- start start 2024-05-24T11:49:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.822+0000 7f53677e2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53601074e0 0x7f53601a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.822+0000 7f53677e2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5360107ea0 0x7f53601a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.822+0000 7f53677e2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f536019d930 0x7f536019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.822+0000 7f53677e2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536019e3b0 con 0x7f536019d930 2024-05-24T11:49:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.822+0000 7f53677e2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536019e4f0 con 0x7f5360107ea0 2024-05-24T11:49:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.822+0000 7f53677e2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536019e6b0 con 0x7f53601074e0 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f536557e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53601074e0 0x7f53601a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f5365d7f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f536019d930 0x7f536019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f536557e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53601074e0 0x7f53601a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59242/0 (socket says 172.21.15.39:59242) 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f536557e700 1 -- 172.21.15.39:0/3665197680 learned_addr learned my addr 172.21.15.39:0/3665197680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f5364d7d700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5360107ea0 0x7f53601a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f5365d7f700 1 -- 172.21.15.39:0/3665197680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53601074e0 msgr2=0x7f53601a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f5365d7f700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53601074e0 0x7f53601a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f5365d7f700 1 -- 172.21.15.39:0/3665197680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5360107ea0 msgr2=0x7f53601a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f5365d7f700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5360107ea0 0x7f53601a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.823+0000 7f5365d7f700 1 -- 172.21.15.39:0/3665197680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0095f0 con 0x7f536019d930 2024-05-24T11:49:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.824+0000 7f5365d7f700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f536019d930 0x7f536019dde0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f535800be20 tx=0x7f535800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.824+0000 7f53567fc700 1 -- 172.21.15.39:0/3665197680 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535800cd10 con 0x7f536019d930 2024-05-24T11:49:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.824+0000 7f53567fc700 1 -- 172.21.15.39:0/3665197680 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f535800ce70 con 0x7f536019d930 2024-05-24T11:49:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.824+0000 7f53567fc700 1 -- 172.21.15.39:0/3665197680 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358012700 con 0x7f536019d930 2024-05-24T11:49:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.824+0000 7f53677e2700 1 -- 172.21.15.39:0/3665197680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5360079850 con 0x7f536019d930 2024-05-24T11:49:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.824+0000 7f53677e2700 1 -- 172.21.15.39:0/3665197680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5360079d10 con 0x7f536019d930 2024-05-24T11:49:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.829+0000 7f53567fc700 1 -- 172.21.15.39:0/3665197680 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5358014440 con 0x7f536019d930 2024-05-24T11:49:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.830+0000 7f537399db80 1 Processor -- start 2024-05-24T11:49:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.831+0000 7f53567fc700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f534c0a1310 0x7f534c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.831+0000 7f53567fc700 1 -- 172.21.15.39:0/3665197680 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5358006e40 con 0x7f536019d930 2024-05-24T11:49:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.832+0000 7f5364d7d700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f534c0a1310 0x7f534c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.832+0000 7f5364d7d700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f534c0a1310 0x7f534c0a37d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5350005950 tx=0x7f53500058e0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.833+0000 7f537399db80 1 -- start start 2024-05-24T11:49:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.833+0000 7f537399db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.833+0000 7f537399db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 0x564ea25dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.833+0000 7f537399db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea25dc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.833+0000 7f537399db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea25330f0 con 0x564ea25e1950 2024-05-24T11:49:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.833+0000 7f537399db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea2530ab0 con 0x564ea25dc480 2024-05-24T11:49:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.833+0000 7f537399db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea252c3a0 con 0x564ea25e2270 2024-05-24T11:49:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.834+0000 7f536557e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.834+0000 7f536557e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25e1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49410/0 (socket says 172.21.15.39:49410) 2024-05-24T11:49:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.834+0000 7f536557e700 1 -- 172.21.15.39:0/3783057554 learned_addr learned my addr 172.21.15.39:0/3783057554 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.834+0000 7f5364d7d700 1 --2- 172.21.15.39:0/3783057554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 0x564ea25dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.834+0000 7f5365d7f700 1 --2- 172.21.15.39:0/3783057554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea25dc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.835+0000 7f536557e700 1 -- 172.21.15.39:0/3783057554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 msgr2=0x564ea25dbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.835+0000 7f536557e700 1 --2- 172.21.15.39:0/3783057554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 0x564ea25dbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.835+0000 7f536557e700 1 -- 172.21.15.39:0/3783057554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 msgr2=0x564ea25dc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.835+0000 7f536557e700 1 --2- 172.21.15.39:0/3783057554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea25dc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.835+0000 7f536557e700 1 -- 172.21.15.39:0/3783057554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ea23ad350 con 0x564ea25e1950 2024-05-24T11:49:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.835+0000 7f536557e700 1 --2- 172.21.15.39:0/3783057554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25e1d30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f535c015000 tx=0x7f535c003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87053486316416c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.835+0000 7f534b7fe700 1 -- 172.21.15.39:0/3783057554 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c005240 con 0x564ea25e1950 2024-05-24T11:49:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.835+0000 7f534b7fe700 1 -- 172.21.15.39:0/3783057554 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f535c01e040 con 0x564ea25e1950 2024-05-24T11:49:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.835+0000 7f534b7fe700 1 -- 172.21.15.39:0/3783057554 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c004760 con 0x564ea25e1950 2024-05-24T11:49:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.836+0000 7f537399db80 1 -- 172.21.15.39:0/3783057554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 msgr2=0x564ea25e1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.836+0000 7f537399db80 1 --2- 172.21.15.39:0/3783057554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25e1d30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f535c015000 tx=0x7f535c003700 comp rx=0 tx=0).stop 2024-05-24T11:49:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.837+0000 7f537399db80 1 -- 172.21.15.39:0/3783057554 shutdown_connections 2024-05-24T11:49:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.837+0000 7f537399db80 1 --2- 172.21.15.39:0/3783057554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea25dc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.837+0000 7f537399db80 1 --2- 172.21.15.39:0/3783057554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25e1d30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.837+0000 7f537399db80 1 --2- 172.21.15.39:0/3783057554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 0x564ea25dbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.837+0000 7f537399db80 1 -- 172.21.15.39:0/3783057554 >> 172.21.15.39:0/3783057554 conn(0x564ea253a1b0 msgr2=0x564ea25d6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.837+0000 7f537399db80 1 -- 172.21.15.39:0/3783057554 shutdown_connections 2024-05-24T11:49:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.838+0000 7f537399db80 1 -- 172.21.15.39:0/3783057554 wait complete. 2024-05-24T11:49:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.838+0000 7f537399db80 1 -- start start 2024-05-24T11:49:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.838+0000 7f537399db80 1 Processor -- start 2024-05-24T11:49:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f537399db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25e8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f537399db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25e54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f537399db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 0x564ea25e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f537399db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea24c8500 con 0x564ea25e1950 2024-05-24T11:49:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f537399db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea24c81a0 con 0x564ea25e2270 2024-05-24T11:49:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f537399db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea23af640 con 0x564ea25dc480 2024-05-24T11:49:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f537399db80 10 client.0 _ll_register_callbacks cb 0x564ea24d1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:49:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f536557e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f536557e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25e8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59250/0 (socket says 172.21.15.39:59250) 2024-05-24T11:49:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f536557e700 1 -- 172.21.15.39:0/990663567 learned_addr learned my addr 172.21.15.39:0/990663567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.839+0000 7f5365d7f700 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 0x564ea25e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.840+0000 7f536557e700 1 -- 172.21.15.39:0/990663567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 msgr2=0x564ea25e5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.840+0000 7f536557e700 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 0x564ea25e5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.840+0000 7f536557e700 1 -- 172.21.15.39:0/990663567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 msgr2=0x564ea25e54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.840+0000 7f536557e700 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25e54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.840+0000 7f536557e700 1 -- 172.21.15.39:0/990663567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5360114c60 con 0x564ea25dc480 2024-05-24T11:49:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.840+0000 7f536557e700 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25e8160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f535c009990 tx=0x7f535c00bfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d9780ebf4c66500 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.840+0000 7f5348ff9700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.840+0000 7f5348ff9700 1 -- 172.21.15.39:0/990663567 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c005510 con 0x564ea25dc480 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.840+0000 7f537399db80 1 -- 172.21.15.39:0/990663567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ea2485210 con 0x564ea25dc480 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f5348ff9700 1 -- 172.21.15.39:0/990663567 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f535c005670 con 0x564ea25dc480 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f5348ff9700 1 -- 172.21.15.39:0/990663567 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f535c004500 con 0x564ea25dc480 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f537399db80 10 client.25795 fetch_fsmap learned FSMap version 29 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f537399db80 1 -- 172.21.15.39:0/990663567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564ea2359b00 con 0x564ea25dc480 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f5348ff9700 1 -- 172.21.15.39:0/990663567 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564ea2485210 con 0x564ea25dc480 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f5348ff9700 1 -- 172.21.15.39:0/990663567 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f535c029070 con 0x564ea25dc480 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f537399db80 10 client.25795 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f537399db80 10 client.25795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f537399db80 20 client.25795 populate_metadata read hostname 'smithi039' 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f537399db80 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564ea2606750 0x564ea2608b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f537399db80 4 client.25795 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:49:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.841+0000 7f537399db80 1 -- 172.21.15.39:0/990663567 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ea25ab7b0 con 0x564ea2606750 2024-05-24T11:49:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.842+0000 7f5364d7d700 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564ea2606750 0x564ea2608b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.842+0000 7f5364d7d700 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564ea2606750 0x564ea2608b70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13bb27d6f97cdf0a server_cookie=f10a4f5c739b0f9f in_seq=0 out_seq=0 2024-05-24T11:49:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.842+0000 7f5348ff9700 10 client.25795 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.844+0000 7f5348ff9700 1 -- 172.21.15.39:0/990663567 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564ea25ab7b0 con 0x564ea2606750 2024-05-24T11:49:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.844+0000 7f5348ff9700 10 client.25795 handle_command_reply: tid=0 2024-05-24T11:49:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.844+0000 7f537399db80 1 client.25795 shutdown 2024-05-24T11:49:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 -- 172.21.15.39:0/990663567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 msgr2=0x564ea25e8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25e8160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f535c009990 tx=0x7f535c00bfb0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 -- 172.21.15.39:0/990663567 shutdown_connections 2024-05-24T11:49:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25e8160 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564ea2606750 0x564ea2608b70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 0x564ea25e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 --2- 172.21.15.39:0/990663567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25e54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f5348ff9700 0 client.25795 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 -- 172.21.15.39:0/990663567 >> 172.21.15.39:0/990663567 conn(0x564ea253a1b0 msgr2=0x564ea25daf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 -- 172.21.15.39:0/990663567 shutdown_connections 2024-05-24T11:49:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.845+0000 7f537399db80 1 -- 172.21.15.39:0/990663567 wait complete. 2024-05-24T11:49:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.846+0000 7f537399db80 20 client.25795 trim_cache size 0 max 16384 2024-05-24T11:49:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.874+0000 7f537399db80 1 Processor -- start 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.875+0000 7f537399db80 1 -- start start 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.875+0000 7f537399db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea2608b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.875+0000 7f537399db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25d8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.875+0000 7f537399db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 0x564ea25e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.875+0000 7f537399db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea260dd60 con 0x564ea25e1950 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.875+0000 7f537399db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea25e0100 con 0x564ea25dc480 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.875+0000 7f537399db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea25f97d0 con 0x564ea25e2270 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f536557e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea2608b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f536557e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea2608b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:53928/0 (socket says 172.21.15.39:53928) 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f536557e700 1 -- 172.21.15.39:0/3045684464 learned_addr learned my addr 172.21.15.39:0/3045684464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f5364d7d700 1 --2- 172.21.15.39:0/3045684464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f5365d7f700 1 --2- 172.21.15.39:0/3045684464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 0x564ea25e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f536557e700 1 -- 172.21.15.39:0/3045684464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 msgr2=0x564ea25e5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f536557e700 1 --2- 172.21.15.39:0/3045684464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 0x564ea25e5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f536557e700 1 -- 172.21.15.39:0/3045684464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 msgr2=0x564ea25d8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f536557e700 1 --2- 172.21.15.39:0/3045684464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25d8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f536557e700 1 -- 172.21.15.39:0/3045684464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ea24f2500 con 0x564ea25dc480 2024-05-24T11:49:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.876+0000 7f536557e700 1 --2- 172.21.15.39:0/3045684464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea2608b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f535c000c00 tx=0x7f535c009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38c6676c70472866 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.877+0000 7f534bfff700 1 -- 172.21.15.39:0/3045684464 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c005bc0 con 0x564ea25dc480 2024-05-24T11:49:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.877+0000 7f534bfff700 1 -- 172.21.15.39:0/3045684464 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f535c00fa00 con 0x564ea25dc480 2024-05-24T11:49:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.877+0000 7f534bfff700 1 -- 172.21.15.39:0/3045684464 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c00e3e0 con 0x564ea25dc480 2024-05-24T11:49:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.878+0000 7f537399db80 1 -- 172.21.15.39:0/3045684464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 msgr2=0x564ea2608b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.878+0000 7f537399db80 1 --2- 172.21.15.39:0/3045684464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea2608b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f535c000c00 tx=0x7f535c009cd0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.879+0000 7f537399db80 1 -- 172.21.15.39:0/3045684464 shutdown_connections 2024-05-24T11:49:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.879+0000 7f537399db80 1 --2- 172.21.15.39:0/3045684464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25e2270 0x564ea25e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.879+0000 7f537399db80 1 --2- 172.21.15.39:0/3045684464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25dc480 0x564ea2608b40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.879+0000 7f537399db80 1 --2- 172.21.15.39:0/3045684464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25d8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.879+0000 7f537399db80 1 -- 172.21.15.39:0/3045684464 >> 172.21.15.39:0/3045684464 conn(0x564ea253a1b0 msgr2=0x564ea26026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.879+0000 7f537399db80 1 -- 172.21.15.39:0/3045684464 shutdown_connections 2024-05-24T11:49:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.879+0000 7f537399db80 1 -- 172.21.15.39:0/3045684464 wait complete. 2024-05-24T11:49:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.879+0000 7f537399db80 1 -- start start 2024-05-24T11:49:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.879+0000 7f537399db80 1 Processor -- start 2024-05-24T11:49:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f537399db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25d6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f537399db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25d6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f537399db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 0x564ea265ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f537399db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea25d6d60 con 0x564ea25e1950 2024-05-24T11:49:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f537399db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea265b180 con 0x564ea25e2270 2024-05-24T11:49:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f537399db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea265b2f0 con 0x564ea25dc480 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f537399db80 10 client.0 _ll_register_callbacks cb 0x564ea24d1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f536557e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25d6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f536557e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25d6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59282/0 (socket says 172.21.15.39:59282) 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f536557e700 1 -- 172.21.15.39:0/2462417643 learned_addr learned my addr 172.21.15.39:0/2462417643 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.880+0000 7f5365d7f700 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 0x564ea265ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f536557e700 1 -- 172.21.15.39:0/2462417643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 msgr2=0x564ea265ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f536557e700 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 0x564ea265ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f536557e700 1 -- 172.21.15.39:0/2462417643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 msgr2=0x564ea25d6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f536557e700 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25d6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f536557e700 1 -- 172.21.15.39:0/2462417643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ea2359b00 con 0x564ea25dc480 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f536557e700 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25d6110 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f535c00fed0 tx=0x7f535c00f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42fbdd043ee83a76 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f5348ff9700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:49:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f5348ff9700 1 -- 172.21.15.39:0/2462417643 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c005720 con 0x564ea25dc480 2024-05-24T11:49:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f5348ff9700 1 -- 172.21.15.39:0/2462417643 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f535c000e40 con 0x564ea25dc480 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f537399db80 1 -- 172.21.15.39:0/2462417643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f535c01e040 con 0x564ea25dc480 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.881+0000 7f5348ff9700 1 -- 172.21.15.39:0/2462417643 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f535c00f810 con 0x564ea25dc480 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f537399db80 10 client.25801 fetch_fsmap learned FSMap version 29 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f537399db80 1 -- 172.21.15.39:0/2462417643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564ea23b1c70 con 0x564ea25dc480 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f5348ff9700 1 -- 172.21.15.39:0/2462417643 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f535c01e040 con 0x564ea25dc480 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f5348ff9700 1 -- 172.21.15.39:0/2462417643 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f535c029070 con 0x564ea25dc480 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f537399db80 10 client.25801 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f537399db80 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f537399db80 20 client.25801 populate_metadata read hostname 'smithi039' 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f537399db80 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564ea265c5f0 0x564ea265e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f537399db80 4 client.25801 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f537399db80 1 -- 172.21.15.39:0/2462417643 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ea2524e60 con 0x564ea265c5f0 2024-05-24T11:49:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.882+0000 7f5364d7d700 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564ea265c5f0 0x564ea265e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.885+0000 7f5364d7d700 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564ea265c5f0 0x564ea265e9e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e42a8739b78d0b2 server_cookie=b442bfd3437773bb in_seq=0 out_seq=0 2024-05-24T11:49:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.885+0000 7f5348ff9700 10 client.25801 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.918+0000 7f5348ff9700 1 -- 172.21.15.39:0/2462417643 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ea2524e60 con 0x564ea265c5f0 2024-05-24T11:49:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.918+0000 7f5348ff9700 10 client.25801 handle_command_reply: tid=0 2024-05-24T11:49:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.918+0000 7f537399db80 1 client.25801 shutdown 2024-05-24T11:49:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.919+0000 7f537399db80 1 -- 172.21.15.39:0/2462417643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 msgr2=0x564ea25d6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.919+0000 7f537399db80 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25d6110 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f535c00fed0 tx=0x7f535c00f460 comp rx=0 tx=0).stop 2024-05-24T11:49:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.928+0000 7f537399db80 1 -- 172.21.15.39:0/2462417643 shutdown_connections 2024-05-24T11:49:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.928+0000 7f537399db80 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ea25dc480 0x564ea25d6110 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.928+0000 7f537399db80 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x564ea265c5f0 0x564ea265e9e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.929+0000 7f537399db80 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ea25e2270 0x564ea265ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.929+0000 7f537399db80 1 --2- 172.21.15.39:0/2462417643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ea25e1950 0x564ea25d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.929+0000 7f5348ff9700 0 client.25801 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.929+0000 7f537399db80 1 -- 172.21.15.39:0/2462417643 >> 172.21.15.39:0/2462417643 conn(0x564ea253a1b0 msgr2=0x564ea25df380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.931+0000 7f537399db80 1 -- 172.21.15.39:0/2462417643 shutdown_connections 2024-05-24T11:49:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.931+0000 7f537399db80 1 -- 172.21.15.39:0/2462417643 wait complete. 2024-05-24T11:49:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.931+0000 7f537399db80 20 client.25801 trim_cache size 0 max 16384 2024-05-24T11:49:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:49:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:49:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:49:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:49:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.945+0000 7f53677e2700 1 -- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f534c0a1310 msgr2=0x7f534c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.945+0000 7f53677e2700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f534c0a1310 0x7f534c0a37d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5350005950 tx=0x7f53500058e0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.945+0000 7f53677e2700 1 -- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f536019d930 msgr2=0x7f536019dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.945+0000 7f53677e2700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f536019d930 0x7f536019dde0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f535800be20 tx=0x7f535800bf00 comp rx=0 tx=0).stop 2024-05-24T11:49:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.945+0000 7f53677e2700 1 -- 172.21.15.39:0/3665197680 shutdown_connections 2024-05-24T11:49:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.945+0000 7f53677e2700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53601074e0 0x7f53601a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.945+0000 7f53677e2700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f534c0a1310 0x7f534c0a37d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.945+0000 7f53677e2700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f536019d930 0x7f536019dde0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.945+0000 7f53677e2700 1 --2- 172.21.15.39:0/3665197680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5360107ea0 0x7f53601a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.946+0000 7f53677e2700 1 -- 172.21.15.39:0/3665197680 >> 172.21.15.39:0/3665197680 conn(0x7f5360076dc0 msgr2=0x7f5360078130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.946+0000 7f53677e2700 1 -- 172.21.15.39:0/3665197680 shutdown_connections 2024-05-24T11:49:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:26.946+0000 7f53677e2700 1 -- 172.21.15.39:0/3665197680 wait complete. 2024-05-24T11:49:27.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:26 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:49:27.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:26 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:49:27.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:26 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:49:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:27 smithi039 ceph-mon[42919]: pgmap v351: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:49:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:49:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:49:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:49:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:27 smithi053 ceph-mon[47448]: pgmap v351: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:49:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:49:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:49:28.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:49:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:27 smithi137 ceph-mon[47907]: pgmap v351: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:49:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:49:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:49:28.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:49:29.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 605.0 s, 904.0 tps, lat 110.787 ms stddev 46.396 2024-05-24T11:49:30.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:29 smithi039 ceph-mon[42919]: pgmap v352: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 24 MiB/s wr, 1.77k op/s 2024-05-24T11:49:30.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:29 smithi053 ceph-mon[47448]: pgmap v352: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 24 MiB/s wr, 1.77k op/s 2024-05-24T11:49:30.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:29 smithi137 ceph-mon[47907]: pgmap v352: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 24 MiB/s wr, 1.77k op/s 2024-05-24T11:49:32.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:31 smithi053 ceph-mon[47448]: pgmap v353: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-05-24T11:49:32.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:31 smithi137 ceph-mon[47907]: pgmap v353: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-05-24T11:49:32.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:31 smithi039 ceph-mon[42919]: pgmap v353: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-05-24T11:49:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 610.0 s, 1001.4 tps, lat 99.622 ms stddev 38.030 2024-05-24T11:49:34.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:33 smithi053 ceph-mon[47448]: pgmap v354: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-05-24T11:49:34.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:33 smithi137 ceph-mon[47907]: pgmap v354: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-05-24T11:49:34.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:33 smithi039 ceph-mon[42919]: pgmap v354: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-05-24T11:49:36.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:35 smithi053 ceph-mon[47448]: pgmap v355: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 28 MiB/s wr, 1.77k op/s 2024-05-24T11:49:36.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:35 smithi137 ceph-mon[47907]: pgmap v355: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 28 MiB/s wr, 1.77k op/s 2024-05-24T11:49:36.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:35 smithi039 ceph-mon[42919]: pgmap v355: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 28 MiB/s wr, 1.77k op/s 2024-05-24T11:49:38.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:37 smithi053 ceph-mon[47448]: pgmap v356: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 21 MiB/s wr, 1.18k op/s 2024-05-24T11:49:38.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:37 smithi137 ceph-mon[47907]: pgmap v356: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 21 MiB/s wr, 1.18k op/s 2024-05-24T11:49:38.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:37 smithi039 ceph-mon[42919]: pgmap v356: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 21 MiB/s wr, 1.18k op/s 2024-05-24T11:49:39.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 615.0 s, 863.6 tps, lat 115.715 ms stddev 52.940 2024-05-24T11:49:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.134+0000 7f51fb5f8700 1 -- 172.21.15.39:0/3522904114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 msgr2=0x7f51f4108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.134+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/3522904114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 0x7f51f4108450 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b0a0 tx=0x7f51ec00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.135+0000 7f51fb5f8700 1 -- 172.21.15.39:0/3522904114 shutdown_connections 2024-05-24T11:49:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.135+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/3522904114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f410f630 0x7f51f410fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.135+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/3522904114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 0x7f51f4108450 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.135+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/3522904114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f410d440 0x7f51f410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.135+0000 7f51fb5f8700 1 -- 172.21.15.39:0/3522904114 >> 172.21.15.39:0/3522904114 conn(0x7f51f406bf80 msgr2=0x7f51f406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.139+0000 7f51fb5f8700 1 -- 172.21.15.39:0/3522904114 shutdown_connections 2024-05-24T11:49:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.139+0000 7f51fb5f8700 1 -- 172.21.15.39:0/3522904114 wait complete. 2024-05-24T11:49:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.140+0000 7f51fb5f8700 1 Processor -- start 2024-05-24T11:49:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.140+0000 7f51fb5f8700 1 -- start start 2024-05-24T11:49:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.140+0000 7f51fb5f8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 0x7f51f41a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.140+0000 7f51fb5f8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f410d440 0x7f51f41a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.140+0000 7f51fb5f8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f410f630 0x7f51f41aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.140+0000 7f51fb5f8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41ab1e0 con 0x7f51f4107fa0 2024-05-24T11:49:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.140+0000 7f51fb5f8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41ab350 con 0x7f51f410f630 2024-05-24T11:49:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.140+0000 7f51fb5f8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41ab510 con 0x7f51f410d440 2024-05-24T11:49:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.140+0000 7f51f9394700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 0x7f51f41a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f9394700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 0x7f51f41a62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60374/0 (socket says 172.21.15.39:60374) 2024-05-24T11:49:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f9394700 1 -- 172.21.15.39:0/2359562227 learned_addr learned my addr 172.21.15.39:0/2359562227 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f8b93700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f410d440 0x7f51f41a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f9b95700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f410f630 0x7f51f41aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f9394700 1 -- 172.21.15.39:0/2359562227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f410d440 msgr2=0x7f51f41a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f9394700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f410d440 0x7f51f41a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f9394700 1 -- 172.21.15.39:0/2359562227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f410f630 msgr2=0x7f51f41aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f9394700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f410f630 0x7f51f41aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f9394700 1 -- 172.21.15.39:0/2359562227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4009520 con 0x7f51f4107fa0 2024-05-24T11:49:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.141+0000 7f51f9b95700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f410f630 0x7f51f41aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:49:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.142+0000 7f51f9394700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 0x7f51f41a62c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e970 tx=0x7f51f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.142+0000 7f51ea7fc700 1 -- 172.21.15.39:0/2359562227 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0009790 con 0x7f51f4107fa0 2024-05-24T11:49:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.142+0000 7f51ea7fc700 1 -- 172.21.15.39:0/2359562227 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51f000eec0 con 0x7f51f4107fa0 2024-05-24T11:49:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.142+0000 7f51ea7fc700 1 -- 172.21.15.39:0/2359562227 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0003c10 con 0x7f51f4107fa0 2024-05-24T11:49:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.142+0000 7f51fb5f8700 1 -- 172.21.15.39:0/2359562227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec009d00 con 0x7f51f4107fa0 2024-05-24T11:49:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.142+0000 7f51fb5f8700 1 -- 172.21.15.39:0/2359562227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f41b2e50 con 0x7f51f4107fa0 2024-05-24T11:49:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.147+0000 7f51ea7fc700 1 -- 172.21.15.39:0/2359562227 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f51f0003db0 con 0x7f51f4107fa0 2024-05-24T11:49:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.147+0000 7f52077b3b80 1 Processor -- start 2024-05-24T11:49:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.148+0000 7f52077b3b80 1 -- start start 2024-05-24T11:49:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.148+0000 7f51ea7fc700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f51e00a14a0 0x7f51e00a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.148+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a168950 0x55701a168d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.148+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a169270 0x55701a162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.148+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a163480 0x55701a1638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.148+0000 7f52077b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a0ba0f0 con 0x55701a168950 2024-05-24T11:49:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.148+0000 7f52077b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a0b7ab0 con 0x55701a163480 2024-05-24T11:49:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.148+0000 7f52077b3b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a0b33a0 con 0x55701a169270 2024-05-24T11:49:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.148+0000 7f51ea7fc700 1 -- 172.21.15.39:0/2359562227 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f51f0014070 con 0x7f51f4107fa0 2024-05-24T11:49:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f8b93700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a169270 0x55701a162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f8b93700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a169270 0x55701a162f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:50190/0 (socket says 172.21.15.39:50190) 2024-05-24T11:49:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f8b93700 1 -- 172.21.15.39:0/3101776148 learned_addr learned my addr 172.21.15.39:0/3101776148 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f9b95700 1 --2- 172.21.15.39:0/3101776148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a163480 0x55701a1638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f8b93700 1 -- 172.21.15.39:0/3101776148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a163480 msgr2=0x55701a1638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f8b93700 1 --2- 172.21.15.39:0/3101776148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a163480 0x55701a1638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f8b93700 1 -- 172.21.15.39:0/3101776148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a168950 msgr2=0x55701a168d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f8b93700 1 --2- 172.21.15.39:0/3101776148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a168950 0x55701a168d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f8b93700 1 -- 172.21.15.39:0/3101776148 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557019f34350 con 0x55701a169270 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.149+0000 7f51f8b93700 1 --2- 172.21.15.39:0/3101776148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a169270 0x55701a162f40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009eb0 tx=0x7f51e4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cad2d10681fddee server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.150+0000 7f51f9b95700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f51e00a14a0 0x7f51e00a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.150+0000 7f51df7fe700 1 -- 172.21.15.39:0/3101776148 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e400e670 con 0x55701a169270 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.150+0000 7f51df7fe700 1 -- 172.21.15.39:0/3101776148 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51e400ec60 con 0x55701a169270 2024-05-24T11:49:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.150+0000 7f51df7fe700 1 -- 172.21.15.39:0/3101776148 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4019d40 con 0x55701a169270 2024-05-24T11:49:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.151+0000 7f52077b3b80 1 -- 172.21.15.39:0/3101776148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a169270 msgr2=0x55701a162f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.151+0000 7f52077b3b80 1 --2- 172.21.15.39:0/3101776148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a169270 0x55701a162f40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009eb0 tx=0x7f51e4011720 comp rx=0 tx=0).stop 2024-05-24T11:49:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.151+0000 7f51f9b95700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f51e00a14a0 0x7f51e00a3960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009cd0 tx=0x7f51ec00b700 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.151+0000 7f52077b3b80 1 -- 172.21.15.39:0/3101776148 shutdown_connections 2024-05-24T11:49:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.151+0000 7f52077b3b80 1 --2- 172.21.15.39:0/3101776148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a163480 0x55701a1638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.151+0000 7f52077b3b80 1 --2- 172.21.15.39:0/3101776148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a168950 0x55701a168d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.151+0000 7f52077b3b80 1 --2- 172.21.15.39:0/3101776148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a169270 0x55701a162f40 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009eb0 tx=0x7f51e4011720 comp rx=0 tx=0).stop 2024-05-24T11:49:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.151+0000 7f52077b3b80 1 -- 172.21.15.39:0/3101776148 >> 172.21.15.39:0/3101776148 conn(0x55701a0c11b0 msgr2=0x55701a15d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.154+0000 7f52077b3b80 1 -- 172.21.15.39:0/3101776148 shutdown_connections 2024-05-24T11:49:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.154+0000 7f52077b3b80 1 -- 172.21.15.39:0/3101776148 wait complete. 2024-05-24T11:49:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.154+0000 7f52077b3b80 1 -- start start 2024-05-24T11:49:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.154+0000 7f52077b3b80 1 Processor -- start 2024-05-24T11:49:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a16c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a168950 0x55701a16cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a16d220 0x55701a178d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f52077b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a04f500 con 0x55701a16d220 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f52077b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a04f1a0 con 0x55701a168950 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f52077b3b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557019f36640 con 0x55701a163480 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f52077b3b80 10 client.0 _ll_register_callbacks cb 0x55701a058ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f51f9394700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a16c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f51f9394700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a16c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:50198/0 (socket says 172.21.15.39:50198) 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f51f9394700 1 -- 172.21.15.39:0/2475570060 learned_addr learned my addr 172.21.15.39:0/2475570060 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f51f9b95700 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a16d220 0x55701a178d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f51f9394700 1 -- 172.21.15.39:0/2475570060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a168950 msgr2=0x55701a16cce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:49:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f51f9394700 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a168950 0x55701a16cce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f51f9394700 1 -- 172.21.15.39:0/2475570060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a16d220 msgr2=0x55701a178d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f51f9394700 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a16d220 0x55701a178d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.155+0000 7f51f9394700 1 -- 172.21.15.39:0/2475570060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e4009b10 con 0x55701a163480 2024-05-24T11:49:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.157+0000 7f51f9394700 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a16c7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c5b0 tx=0x7f51f00999a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f9d5cce69fc4a56 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.157+0000 7f51dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:49:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.162+0000 7f51dcff9700 1 -- 172.21.15.39:0/2475570060 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f005d340 con 0x55701a163480 2024-05-24T11:49:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.162+0000 7f51dcff9700 1 -- 172.21.15.39:0/2475570060 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f51f0099f10 con 0x55701a163480 2024-05-24T11:49:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.162+0000 7f52077b3b80 1 -- 172.21.15.39:0/2475570060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55701a00c210 con 0x55701a163480 2024-05-24T11:49:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.162+0000 7f51dcff9700 1 -- 172.21.15.39:0/2475570060 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51f009dd40 con 0x55701a163480 2024-05-24T11:49:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.162+0000 7f52077b3b80 10 client.25813 fetch_fsmap learned FSMap version 29 2024-05-24T11:49:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.162+0000 7f52077b3b80 1 -- 172.21.15.39:0/2475570060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557019ee0b00 con 0x55701a163480 2024-05-24T11:49:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.162+0000 7f51dcff9700 1 -- 172.21.15.39:0/2475570060 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55701a00c210 con 0x55701a163480 2024-05-24T11:49:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.163+0000 7f51dcff9700 1 -- 172.21.15.39:0/2475570060 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f51f009a170 con 0x55701a163480 2024-05-24T11:49:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.163+0000 7f52077b3b80 10 client.25813 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:49:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.163+0000 7f52077b3b80 10 client.25813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:49:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.163+0000 7f52077b3b80 20 client.25813 populate_metadata read hostname 'smithi039' 2024-05-24T11:49:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.163+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55701a17d310 0x55701a17f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.163+0000 7f52077b3b80 4 client.25813 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:49:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.163+0000 7f52077b3b80 1 -- 172.21.15.39:0/2475570060 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55701a1327b0 con 0x55701a17d310 2024-05-24T11:49:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.163+0000 7f51f8b93700 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55701a17d310 0x55701a17f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.164+0000 7f51f8b93700 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55701a17d310 0x55701a17f730 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=975401343ae89d72 server_cookie=20baa266f70e09ff in_seq=0 out_seq=0 2024-05-24T11:49:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.164+0000 7f51dcff9700 10 client.25813 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.164+0000 7f51dcff9700 1 -- 172.21.15.39:0/2475570060 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55701a1327b0 con 0x55701a17d310 2024-05-24T11:49:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.164+0000 7f51dcff9700 10 client.25813 handle_command_reply: tid=0 2024-05-24T11:49:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 client.25813 shutdown 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 -- 172.21.15.39:0/2475570060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 msgr2=0x55701a16c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a16c7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c5b0 tx=0x7f51f00999a0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 -- 172.21.15.39:0/2475570060 shutdown_connections 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a16c7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55701a17d310 0x55701a17f730 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a16d220 0x55701a178d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2475570060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a168950 0x55701a16cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 -- 172.21.15.39:0/2475570060 >> 172.21.15.39:0/2475570060 conn(0x55701a0c11b0 msgr2=0x55701a1674a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f51dcff9700 0 client.25813 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 -- 172.21.15.39:0/2475570060 shutdown_connections 2024-05-24T11:49:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.165+0000 7f52077b3b80 1 -- 172.21.15.39:0/2475570060 wait complete. 2024-05-24T11:49:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.166+0000 7f52077b3b80 20 client.25813 trim_cache size 0 max 16384 2024-05-24T11:49:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.186+0000 7f52077b3b80 1 Processor -- start 2024-05-24T11:49:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.186+0000 7f52077b3b80 1 -- start start 2024-05-24T11:49:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.186+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a15d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.186+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a168950 0x55701a167400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.186+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a16d220 0x55701a178740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.186+0000 7f52077b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a16d080 con 0x55701a168950 2024-05-24T11:49:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.186+0000 7f52077b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a16f310 con 0x55701a16d220 2024-05-24T11:49:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.186+0000 7f52077b3b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a16f550 con 0x55701a163480 2024-05-24T11:49:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9b95700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a16d220 0x55701a178740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9394700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a15d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f8b93700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a168950 0x55701a167400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9b95700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a16d220 0x55701a178740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:47234/0 (socket says 172.21.15.39:47234) 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9b95700 1 -- 172.21.15.39:0/481472769 learned_addr learned my addr 172.21.15.39:0/481472769 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9b95700 1 -- 172.21.15.39:0/481472769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 msgr2=0x55701a15d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9b95700 1 --2- 172.21.15.39:0/481472769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a15d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9b95700 1 -- 172.21.15.39:0/481472769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a168950 msgr2=0x55701a167400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9b95700 1 --2- 172.21.15.39:0/481472769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a168950 0x55701a167400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9b95700 1 -- 172.21.15.39:0/481472769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55701a079500 con 0x55701a16d220 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.187+0000 7f51f9b95700 1 --2- 172.21.15.39:0/481472769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a16d220 0x55701a178740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51ec015000 tx=0x7f51ec006020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9194221ee28557a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.188+0000 7f51dffff700 1 -- 172.21.15.39:0/481472769 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00e040 con 0x55701a16d220 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.188+0000 7f51dffff700 1 -- 172.21.15.39:0/481472769 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51ec02c070 con 0x55701a16d220 2024-05-24T11:49:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.188+0000 7f51dffff700 1 -- 172.21.15.39:0/481472769 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0055e0 con 0x55701a16d220 2024-05-24T11:49:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.189+0000 7f52077b3b80 1 -- 172.21.15.39:0/481472769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a16d220 msgr2=0x55701a178740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.189+0000 7f52077b3b80 1 --2- 172.21.15.39:0/481472769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a16d220 0x55701a178740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51ec015000 tx=0x7f51ec006020 comp rx=0 tx=0).stop 2024-05-24T11:49:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.189+0000 7f52077b3b80 1 -- 172.21.15.39:0/481472769 shutdown_connections 2024-05-24T11:49:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.189+0000 7f52077b3b80 1 --2- 172.21.15.39:0/481472769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a15d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.189+0000 7f52077b3b80 1 --2- 172.21.15.39:0/481472769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a16d220 0x55701a178740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.189+0000 7f52077b3b80 1 --2- 172.21.15.39:0/481472769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a168950 0x55701a167400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.189+0000 7f52077b3b80 1 -- 172.21.15.39:0/481472769 >> 172.21.15.39:0/481472769 conn(0x55701a0c11b0 msgr2=0x55701a161730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.190+0000 7f52077b3b80 1 -- 172.21.15.39:0/481472769 shutdown_connections 2024-05-24T11:49:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.190+0000 7f52077b3b80 1 -- 172.21.15.39:0/481472769 wait complete. 2024-05-24T11:49:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.190+0000 7f52077b3b80 1 -- start start 2024-05-24T11:49:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.190+0000 7f52077b3b80 1 Processor -- start 2024-05-24T11:49:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.191+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a171c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.191+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a168950 0x55701a1721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.191+0000 7f52077b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a16d220 0x55701a1726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.191+0000 7f52077b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a1d2810 con 0x55701a16d220 2024-05-24T11:49:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.191+0000 7f52077b3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a1d2950 con 0x55701a168950 2024-05-24T11:49:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.191+0000 7f52077b3b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701a1e3b20 con 0x55701a163480 2024-05-24T11:49:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.191+0000 7f52077b3b80 10 client.0 _ll_register_callbacks cb 0x55701a058ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:49:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f9394700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a171c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f9394700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a171c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:50210/0 (socket says 172.21.15.39:50210) 2024-05-24T11:49:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f9394700 1 -- 172.21.15.39:0/2083817633 learned_addr learned my addr 172.21.15.39:0/2083817633 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f8b93700 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a168950 0x55701a1721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f9394700 1 -- 172.21.15.39:0/2083817633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a168950 msgr2=0x55701a1721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f9394700 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a168950 0x55701a1721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f9394700 1 -- 172.21.15.39:0/2083817633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a16d220 msgr2=0x55701a1726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f9394700 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a16d220 0x55701a1726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f9394700 1 -- 172.21.15.39:0/2083817633 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f410b0e0 con 0x55701a163480 2024-05-24T11:49:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51f9394700 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a171c70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005950 tx=0x7f51f001f570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e071f8c7b343b887 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.194+0000 7f51dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:49:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.195+0000 7f51dcff9700 1 -- 172.21.15.39:0/2083817633 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f009dd40 con 0x55701a163480 2024-05-24T11:49:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.195+0000 7f51dcff9700 1 -- 172.21.15.39:0/2083817633 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f51f00a2390 con 0x55701a163480 2024-05-24T11:49:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.195+0000 7f52077b3b80 1 -- 172.21.15.39:0/2083817633 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51e400ec60 con 0x55701a163480 2024-05-24T11:49:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.195+0000 7f51dcff9700 1 -- 172.21.15.39:0/2083817633 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51f00b8170 con 0x55701a163480 2024-05-24T11:49:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.195+0000 7f52077b3b80 10 client.25819 fetch_fsmap learned FSMap version 29 2024-05-24T11:49:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.195+0000 7f52077b3b80 1 -- 172.21.15.39:0/2083817633 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557019f38c70 con 0x55701a163480 2024-05-24T11:49:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.195+0000 7f51dcff9700 1 -- 172.21.15.39:0/2083817633 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51e400ec60 con 0x55701a163480 2024-05-24T11:49:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.196+0000 7f51dcff9700 1 -- 172.21.15.39:0/2083817633 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f51f0098e70 con 0x55701a163480 2024-05-24T11:49:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.196+0000 7f52077b3b80 10 client.25819 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:49:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.196+0000 7f52077b3b80 10 client.25819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:49:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.196+0000 7f52077b3b80 20 client.25819 populate_metadata read hostname 'smithi039' 2024-05-24T11:49:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.196+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55701a1e4eb0 0x55701a1e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.196+0000 7f52077b3b80 4 client.25819 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:49:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.196+0000 7f52077b3b80 1 -- 172.21.15.39:0/2083817633 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55701a0abe60 con 0x55701a1e4eb0 2024-05-24T11:49:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.196+0000 7f51f8b93700 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55701a1e4eb0 0x55701a1e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.197+0000 7f51dcff9700 10 client.25819 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:40.197+0000 7f51f8b93700 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55701a1e4eb0 0x55701a1e72d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6298bcdca905020 server_cookie=6408a48cd6b524f7 in_seq=0 out_seq=0 2024-05-24T11:49:40.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:39 smithi053 ceph-mon[47448]: pgmap v357: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 30 MiB/s wr, 1.81k op/s 2024-05-24T11:49:40.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:39 smithi137 ceph-mon[47907]: pgmap v357: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 30 MiB/s wr, 1.81k op/s 2024-05-24T11:49:40.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:39 smithi039 ceph-mon[42919]: pgmap v357: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 30 MiB/s wr, 1.81k op/s 2024-05-24T11:49:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f51dcff9700 1 -- 172.21.15.39:0/2083817633 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55701a0abe60 con 0x55701a1e4eb0 2024-05-24T11:49:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f51dcff9700 10 client.25819 handle_command_reply: tid=0 2024-05-24T11:49:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 client.25819 shutdown 2024-05-24T11:49:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 -- 172.21.15.39:0/2083817633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 msgr2=0x55701a171c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a171c70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005950 tx=0x7f51f001f570 comp rx=0 tx=0).stop 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 -- 172.21.15.39:0/2083817633 shutdown_connections 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701a163480 0x55701a171c70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55701a1e4eb0 0x55701a1e72d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55701a16d220 0x55701a1726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f51dcff9700 0 client.25819 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 --2- 172.21.15.39:0/2083817633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55701a168950 0x55701a1721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 -- 172.21.15.39:0/2083817633 >> 172.21.15.39:0/2083817633 conn(0x55701a0c11b0 msgr2=0x55701a178740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 -- 172.21.15.39:0/2083817633 shutdown_connections 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.112+0000 7f52077b3b80 1 -- 172.21.15.39:0/2083817633 wait complete. 2024-05-24T11:49:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.113+0000 7f52077b3b80 20 client.25819 trim_cache size 0 max 16384 2024-05-24T11:49:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:49:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:49:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:49:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:49:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.113+0000 7f51fb5f8700 1 -- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f51e00a14a0 msgr2=0x7f51e00a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.113+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f51e00a14a0 0x7f51e00a3960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009cd0 tx=0x7f51ec00b700 comp rx=0 tx=0).stop 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.113+0000 7f51fb5f8700 1 -- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 msgr2=0x7f51f41a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.113+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 0x7f51f41a62c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e970 tx=0x7f51f000ec80 comp rx=0 tx=0).stop 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.114+0000 7f51fb5f8700 1 -- 172.21.15.39:0/2359562227 shutdown_connections 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.114+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f51e00a14a0 0x7f51e00a3960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.114+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f410f630 0x7f51f41aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.114+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51f4107fa0 0x7f51f41a62c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.114+0000 7f51fb5f8700 1 --2- 172.21.15.39:0/2359562227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f410d440 0x7f51f41a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.114+0000 7f51fb5f8700 1 -- 172.21.15.39:0/2359562227 >> 172.21.15.39:0/2359562227 conn(0x7f51f406bf80 msgr2=0x7f51f40719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.114+0000 7f51fb5f8700 1 -- 172.21.15.39:0/2359562227 shutdown_connections 2024-05-24T11:49:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:42.114+0000 7f51fb5f8700 1 -- 172.21.15.39:0/2359562227 wait complete. 2024-05-24T11:49:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:41 smithi053 ceph-mon[47448]: pgmap v358: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-05-24T11:49:42.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:41 smithi137 ceph-mon[47907]: pgmap v358: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-05-24T11:49:42.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:41 smithi039 ceph-mon[42919]: pgmap v358: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-05-24T11:49:44.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 620.0 s, 812.8 tps, lat 98.730 ms stddev 107.538 2024-05-24T11:49:44.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:43 smithi053 ceph-mon[47448]: pgmap v359: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-05-24T11:49:44.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:43 smithi137 ceph-mon[47907]: pgmap v359: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-05-24T11:49:44.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:43 smithi039 ceph-mon[42919]: pgmap v359: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-05-24T11:49:46.240 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:45 smithi053 ceph-mon[47448]: pgmap v360: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 28 MiB/s wr, 1.83k op/s 2024-05-24T11:49:46.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:45 smithi137 ceph-mon[47907]: pgmap v360: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 28 MiB/s wr, 1.83k op/s 2024-05-24T11:49:46.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:45 smithi039 ceph-mon[42919]: pgmap v360: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 28 MiB/s wr, 1.83k op/s 2024-05-24T11:49:48.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:47 smithi053 ceph-mon[47448]: pgmap v361: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 17 MiB/s wr, 1.21k op/s 2024-05-24T11:49:48.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:47 smithi137 ceph-mon[47907]: pgmap v361: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 17 MiB/s wr, 1.21k op/s 2024-05-24T11:49:48.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:47 smithi039 ceph-mon[42919]: pgmap v361: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 17 MiB/s wr, 1.21k op/s 2024-05-24T11:49:49.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 625.0 s, 750.4 tps, lat 159.601 ms stddev 464.384 2024-05-24T11:49:50.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:49 smithi053 ceph-mon[47448]: pgmap v362: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2024-05-24T11:49:50.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:49 smithi137 ceph-mon[47907]: pgmap v362: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2024-05-24T11:49:50.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:49 smithi039 ceph-mon[42919]: pgmap v362: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2024-05-24T11:49:52.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:51 smithi053 ceph-mon[47448]: pgmap v363: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-05-24T11:49:52.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:51 smithi137 ceph-mon[47907]: pgmap v363: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-05-24T11:49:52.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:51 smithi039 ceph-mon[42919]: pgmap v363: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-05-24T11:49:54.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 630.0 s, 939.0 tps, lat 106.486 ms stddev 37.619 2024-05-24T11:49:54.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:53 smithi053 ceph-mon[47448]: pgmap v364: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-05-24T11:49:54.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:53 smithi137 ceph-mon[47907]: pgmap v364: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-05-24T11:49:54.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:53 smithi039 ceph-mon[42919]: pgmap v364: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-05-24T11:49:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.287+0000 7f9d0566e700 1 -- 172.21.15.39:0/197406583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00106260 msgr2=0x7f9d0010dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.287+0000 7f9d0566e700 1 --2- 172.21.15.39:0/197406583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00106260 0x7f9d0010dde0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800b0a0 tx=0x7f9cf800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.287+0000 7f9d0566e700 1 -- 172.21.15.39:0/197406583 shutdown_connections 2024-05-24T11:49:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.287+0000 7f9d0566e700 1 --2- 172.21.15.39:0/197406583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00106260 0x7f9d0010dde0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.287+0000 7f9d0566e700 1 --2- 172.21.15.39:0/197406583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d00069360 0x7f9d00069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.287+0000 7f9d0566e700 1 --2- 172.21.15.39:0/197406583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d00069d10 0x7f9d00105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.287+0000 7f9d0566e700 1 -- 172.21.15.39:0/197406583 >> 172.21.15.39:0/197406583 conn(0x7f9d00076bc0 msgr2=0x7f9d00076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.287+0000 7f9d0566e700 1 -- 172.21.15.39:0/197406583 shutdown_connections 2024-05-24T11:49:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.287+0000 7f9d0566e700 1 -- 172.21.15.39:0/197406583 wait complete. 2024-05-24T11:49:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9d0566e700 1 Processor -- start 2024-05-24T11:49:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9d0566e700 1 -- start start 2024-05-24T11:49:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9d0566e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d00069360 0x7f9d0019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9d0566e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00106260 0x7f9d0019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9d0566e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0010cfb0 0x7f9d0010d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9d0566e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0010dc10 con 0x7f9d0010cfb0 2024-05-24T11:49:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9d0566e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0010dd80 con 0x7f9d00069360 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9d0566e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00199a70 con 0x7f9d00106260 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9cff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0010cfb0 0x7f9d0010d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9cfeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d00069360 0x7f9d0019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9cfeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d00069360 0x7f9d0019dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:34124/0 (socket says 172.21.15.39:34124) 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.288+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2041586238 learned_addr learned my addr 172.21.15.39:0/2041586238 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cfe7fc700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00106260 0x7f9d0019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2041586238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00106260 msgr2=0x7f9d0019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00106260 0x7f9d0019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0010cfb0 msgr2=0x7f9d0010d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0010cfb0 0x7f9d0010d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2041586238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8009d00 con 0x7f9d00069360 2024-05-24T11:49:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cfe7fc700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00106260 0x7f9d0019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:49:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cff7fe700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0010cfb0 0x7f9d0010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:49:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d00069360 0x7f9d0019dfc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000e9c0 tx=0x7f9cf000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cf7fff700 1 -- 172.21.15.39:0/2041586238 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00097b0 con 0x7f9d00069360 2024-05-24T11:49:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cf7fff700 1 -- 172.21.15.39:0/2041586238 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf0004cc0 con 0x7f9d00069360 2024-05-24T11:49:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9cf7fff700 1 -- 172.21.15.39:0/2041586238 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0005530 con 0x7f9d00069360 2024-05-24T11:49:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9d0566e700 1 -- 172.21.15.39:0/2041586238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d00199d20 con 0x7f9d00069360 2024-05-24T11:49:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.289+0000 7f9d0566e700 1 -- 172.21.15.39:0/2041586238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0019a1e0 con 0x7f9d00069360 2024-05-24T11:49:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.293+0000 7f9cf7fff700 1 -- 172.21.15.39:0/2041586238 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9cf0004750 con 0x7f9d00069360 2024-05-24T11:49:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.293+0000 7f9cf7fff700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9ce40a1310 0x7f9ce40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.294+0000 7f9cf7fff700 1 -- 172.21.15.39:0/2041586238 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9cf0014070 con 0x7f9d00069360 2024-05-24T11:49:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.294+0000 7f9d11829b80 1 Processor -- start 2024-05-24T11:49:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.294+0000 7f9d11829b80 1 -- start start 2024-05-24T11:49:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.294+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 0x55db55cb9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.294+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cba270 0x55db55cb3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.294+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cb4480 0x55db55cb48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.294+0000 7f9d11829b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55c0b0f0 con 0x55db55cb4480 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.294+0000 7f9d11829b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55c08ab0 con 0x55db55cba270 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.294+0000 7f9d11829b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55c043a0 con 0x55db55cb9950 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 0x55db55cb9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 0x55db55cb9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:33420/0 (socket says 172.21.15.39:33420) 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2749832479 learned_addr learned my addr 172.21.15.39:0/2749832479 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cff7fe700 1 --2- 172.21.15.39:0/2749832479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cb4480 0x55db55cb48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2749832479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cba270 msgr2=0x55db55cb3f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:49:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2749832479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cba270 0x55db55cb3f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2749832479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cb4480 msgr2=0x55db55cb48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2749832479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cb4480 0x55db55cb48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2749832479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db55a85350 con 0x55db55cb9950 2024-05-24T11:49:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfe7fc700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9ce40a1310 0x7f9ce40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2749832479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 0x55db55cb9d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00a20f0 tx=0x7f9cf0099b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30becf9fbb9c6d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.295+0000 7f9cf57fa700 1 -- 172.21.15.39:0/2749832479 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf009a1a0 con 0x55db55cb9950 2024-05-24T11:49:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.296+0000 7f9cf57fa700 1 -- 172.21.15.39:0/2749832479 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf009dfd0 con 0x55db55cb9950 2024-05-24T11:49:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.296+0000 7f9cf57fa700 1 -- 172.21.15.39:0/2749832479 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf009a400 con 0x55db55cb9950 2024-05-24T11:49:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.296+0000 7f9d11829b80 1 -- 172.21.15.39:0/2749832479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 msgr2=0x55db55cb9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.296+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2749832479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 0x55db55cb9d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00a20f0 tx=0x7f9cf0099b00 comp rx=0 tx=0).stop 2024-05-24T11:49:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.296+0000 7f9cfe7fc700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9ce40a1310 0x7f9ce40a37d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55db55caf790 tx=0x7f9cec009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.299+0000 7f9d11829b80 1 -- 172.21.15.39:0/2749832479 shutdown_connections 2024-05-24T11:49:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.299+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2749832479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 0x55db55cb9d30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.299+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2749832479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cb4480 0x55db55cb48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.299+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2749832479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cba270 0x55db55cb3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.299+0000 7f9d11829b80 1 -- 172.21.15.39:0/2749832479 >> 172.21.15.39:0/2749832479 conn(0x55db55c121b0 msgr2=0x55db55cae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.299+0000 7f9d11829b80 1 -- 172.21.15.39:0/2749832479 shutdown_connections 2024-05-24T11:49:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.299+0000 7f9d11829b80 1 -- 172.21.15.39:0/2749832479 wait complete. 2024-05-24T11:49:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.300+0000 7f9d11829b80 1 -- start start 2024-05-24T11:49:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.300+0000 7f9d11829b80 1 Processor -- start 2024-05-24T11:49:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.300+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 0x55db55cbd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.300+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cb9950 0x55db55cbdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.300+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cba270 0x55db55cbe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.300+0000 7f9d11829b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55ba0500 con 0x55db55cb9950 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.300+0000 7f9d11829b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55ba01a0 con 0x55db55cb4480 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.300+0000 7f9d11829b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55a87640 con 0x55db55cba270 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.300+0000 7f9d11829b80 10 client.0 _ll_register_callbacks cb 0x55db55ba9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cfeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 0x55db55cbd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cfeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 0x55db55cbd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:34138/0 (socket says 172.21.15.39:34138) 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2646385695 learned_addr learned my addr 172.21.15.39:0/2646385695 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cff7fe700 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cba270 0x55db55cbe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2646385695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cba270 msgr2=0x55db55cbe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cba270 0x55db55cbe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2646385695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cb9950 msgr2=0x55db55cbdaf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cb9950 0x55db55cbdaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cfeffd700 1 -- 172.21.15.39:0/2646385695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0019a1e0 con 0x55db55cb4480 2024-05-24T11:49:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 0x55db55cbd5b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0096650 tx=0x7f9cf000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acf548d9833119f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9ce27fc700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2646385695 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0098540 con 0x55db55cb4480 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.301+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2646385695 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9cf0004a00 con 0x55db55cb4480 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.302+0000 7f9d11829b80 1 -- 172.21.15.39:0/2646385695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db55b5d210 con 0x55db55cb4480 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.302+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2646385695 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf0099690 con 0x55db55cb4480 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.302+0000 7f9d11829b80 10 client.25913 fetch_fsmap learned FSMap version 29 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.302+0000 7f9d11829b80 1 -- 172.21.15.39:0/2646385695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55db55a31b00 con 0x55db55cb4480 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.302+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2646385695 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf00a6080 con 0x55db55cb4480 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.303+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2646385695 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9cf0083020 con 0x55db55cb4480 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.303+0000 7f9d11829b80 10 client.25913 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.303+0000 7f9d11829b80 10 client.25913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:49:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.303+0000 7f9d11829b80 20 client.25913 populate_metadata read hostname 'smithi039' 2024-05-24T11:49:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.303+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55db55ccd790 0x55db55ccfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.303+0000 7f9d11829b80 4 client.25913 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:49:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.303+0000 7f9d11829b80 1 -- 172.21.15.39:0/2646385695 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db55c837b0 con 0x55db55ccd790 2024-05-24T11:49:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.304+0000 7f9cfe7fc700 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55db55ccd790 0x55db55ccfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.304+0000 7f9cfe7fc700 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55db55ccd790 0x55db55ccfb80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dba17f9d2114301 server_cookie=e7b2980c5d71d744 in_seq=0 out_seq=0 2024-05-24T11:49:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.304+0000 7f9ce27fc700 10 client.25913 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2646385695 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db55c837b0 con 0x55db55ccd790 2024-05-24T11:49:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9ce27fc700 10 client.25913 handle_command_reply: tid=0 2024-05-24T11:49:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9d11829b80 1 client.25913 shutdown 2024-05-24T11:49:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9d11829b80 1 -- 172.21.15.39:0/2646385695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 msgr2=0x55db55cbd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 0x55db55cbd5b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0096650 tx=0x7f9cf000ef90 comp rx=0 tx=0).stop 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9d11829b80 1 -- 172.21.15.39:0/2646385695 shutdown_connections 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cba270 0x55db55cbe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55db55ccd790 0x55db55ccfb80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 0x55db55cbd5b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2646385695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cb9950 0x55db55cbdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9d11829b80 1 -- 172.21.15.39:0/2646385695 >> 172.21.15.39:0/2646385695 conn(0x55db55c121b0 msgr2=0x55db55cb2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.305+0000 7f9ce27fc700 0 client.25913 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.306+0000 7f9d11829b80 1 -- 172.21.15.39:0/2646385695 shutdown_connections 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.306+0000 7f9d11829b80 1 -- 172.21.15.39:0/2646385695 wait complete. 2024-05-24T11:49:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.306+0000 7f9d11829b80 20 client.25913 trim_cache size 0 max 16384 2024-05-24T11:49:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9d11829b80 1 Processor -- start 2024-05-24T11:49:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9d11829b80 1 -- start start 2024-05-24T11:49:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 0x55db55cb9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 0x55db55cae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cba270 0x55db55cb4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9d11829b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55cb4da0 con 0x55db55cba270 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9d11829b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55cbb660 con 0x55db55cb4480 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9d11829b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55cafa30 con 0x55db55cb9950 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9cff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cba270 0x55db55cb4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.327+0000 7f9cff7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cba270 0x55db55cb4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54422/0 (socket says 172.21.15.39:54422) 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.328+0000 7f9cff7fe700 1 -- 172.21.15.39:0/1848392621 learned_addr learned my addr 172.21.15.39:0/1848392621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.328+0000 7f9cff7fe700 1 -- 172.21.15.39:0/1848392621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 msgr2=0x55db55cae910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.328+0000 7f9cff7fe700 1 --2- 172.21.15.39:0/1848392621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 0x55db55cae910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.328+0000 7f9cff7fe700 1 -- 172.21.15.39:0/1848392621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 msgr2=0x55db55cb9d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.328+0000 7f9cff7fe700 1 --2- 172.21.15.39:0/1848392621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 0x55db55cb9d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.328+0000 7f9cff7fe700 1 -- 172.21.15.39:0/1848392621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db55bca500 con 0x55db55cba270 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.328+0000 7f9cff7fe700 1 --2- 172.21.15.39:0/1848392621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cba270 0x55db55cb4860 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80093c0 tx=0x7f9cf80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f83fd2a86b64c8a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.328+0000 7f9cf5ffb700 1 -- 172.21.15.39:0/1848392621 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800e040 con 0x55db55cba270 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.329+0000 7f9cf5ffb700 1 -- 172.21.15.39:0/1848392621 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf800bc50 con 0x55db55cba270 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.329+0000 7f9cf5ffb700 1 -- 172.21.15.39:0/1848392621 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80406a0 con 0x55db55cba270 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.329+0000 7f9d11829b80 1 -- 172.21.15.39:0/1848392621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cba270 msgr2=0x55db55cb4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.329+0000 7f9d11829b80 1 --2- 172.21.15.39:0/1848392621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cba270 0x55db55cb4860 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80093c0 tx=0x7f9cf80093f0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.329+0000 7f9d11829b80 1 -- 172.21.15.39:0/1848392621 shutdown_connections 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.330+0000 7f9d11829b80 1 --2- 172.21.15.39:0/1848392621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cba270 0x55db55cb4860 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.330+0000 7f9d11829b80 1 --2- 172.21.15.39:0/1848392621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cb4480 0x55db55cb9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.330+0000 7f9d11829b80 1 --2- 172.21.15.39:0/1848392621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cb9950 0x55db55cae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.330+0000 7f9d11829b80 1 -- 172.21.15.39:0/1848392621 >> 172.21.15.39:0/1848392621 conn(0x55db55c121b0 msgr2=0x55db55cb38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.330+0000 7f9d11829b80 1 -- 172.21.15.39:0/1848392621 shutdown_connections 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.331+0000 7f9d11829b80 1 -- 172.21.15.39:0/1848392621 wait complete. 2024-05-24T11:49:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.331+0000 7f9d11829b80 1 -- start start 2024-05-24T11:49:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.331+0000 7f9d11829b80 1 Processor -- start 2024-05-24T11:49:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cba270 0x55db55cc5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cc2bc0 0x55db55cc3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9d11829b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cc3580 0x55db55d22c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9d11829b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55d231a0 con 0x55db55cc3580 2024-05-24T11:49:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9d11829b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55d23310 con 0x55db55cba270 2024-05-24T11:49:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9d11829b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db55d234d0 con 0x55db55cc2bc0 2024-05-24T11:49:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9d11829b80 10 client.0 _ll_register_callbacks cb 0x55db55ba9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:49:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9cfe7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cc2bc0 0x55db55cc3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9cfe7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cc2bc0 0x55db55cc3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:33456/0 (socket says 172.21.15.39:33456) 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9cfe7fc700 1 -- 172.21.15.39:0/2958317046 learned_addr learned my addr 172.21.15.39:0/2958317046 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cba270 0x55db55cc5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.332+0000 7f9cff7fe700 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cc3580 0x55db55d22c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.333+0000 7f9cfe7fc700 1 -- 172.21.15.39:0/2958317046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cba270 msgr2=0x55db55cc5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.333+0000 7f9cfe7fc700 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cba270 0x55db55cc5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.333+0000 7f9cfe7fc700 1 -- 172.21.15.39:0/2958317046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cc3580 msgr2=0x55db55d22c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.333+0000 7f9cfe7fc700 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cc3580 0x55db55d22c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.333+0000 7f9cfe7fc700 1 -- 172.21.15.39:0/2958317046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0010e650 con 0x55db55cc2bc0 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.333+0000 7f9cfe7fc700 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cc2bc0 0x55db55cc3040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d001058b0 tx=0x7f9cec0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e588665ea5d3b065 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.333+0000 7f9ce27fc700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.334+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2958317046 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec024070 con 0x55db55cc2bc0 2024-05-24T11:49:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.334+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2958317046 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9cec005c80 con 0x55db55cc2bc0 2024-05-24T11:49:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.334+0000 7f9d11829b80 1 -- 172.21.15.39:0/2958317046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cf009dfd0 con 0x55db55cc2bc0 2024-05-24T11:49:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.334+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2958317046 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cec024e90 con 0x55db55cc2bc0 2024-05-24T11:49:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.334+0000 7f9d11829b80 10 client.25831 fetch_fsmap learned FSMap version 29 2024-05-24T11:49:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.334+0000 7f9d11829b80 1 -- 172.21.15.39:0/2958317046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55db55a89c70 con 0x55db55cc2bc0 2024-05-24T11:49:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.335+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2958317046 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf009dfd0 con 0x55db55cc2bc0 2024-05-24T11:49:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.335+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2958317046 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9cec003790 con 0x55db55cc2bc0 2024-05-24T11:49:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.337+0000 7f9d11829b80 10 client.25831 fetch_fsmap finished waiting for FSMap version 29 2024-05-24T11:49:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.337+0000 7f9d11829b80 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:49:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.337+0000 7f9d11829b80 20 client.25831 populate_metadata read hostname 'smithi039' 2024-05-24T11:49:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.337+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55db55d24c50 0x55db55d27070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:49:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.337+0000 7f9d11829b80 4 client.25831 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:49:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.337+0000 7f9d11829b80 1 -- 172.21.15.39:0/2958317046 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db55bfce60 con 0x55db55d24c50 2024-05-24T11:49:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.337+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55db55d24c50 0x55db55d27070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:49:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.338+0000 7f9cfeffd700 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55db55d24c50 0x55db55d27070 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877bde8b0cb1cbb2 server_cookie=d82395132f958055 in_seq=0 out_seq=0 2024-05-24T11:49:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.338+0000 7f9ce27fc700 10 client.25831 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.403+0000 7f9ce27fc700 1 -- 172.21.15.39:0/2958317046 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db55bfce60 con 0x55db55d24c50 2024-05-24T11:49:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.403+0000 7f9ce27fc700 10 client.25831 handle_command_reply: tid=0 2024-05-24T11:49:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.403+0000 7f9d11829b80 1 client.25831 shutdown 2024-05-24T11:49:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.403+0000 7f9d11829b80 1 -- 172.21.15.39:0/2958317046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cc2bc0 msgr2=0x55db55cc3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.403+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cc2bc0 0x55db55cc3040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d001058b0 tx=0x7f9cec0047c0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9d11829b80 1 -- 172.21.15.39:0/2958317046 shutdown_connections 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55db55d24c50 0x55db55d27070 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db55cc3580 0x55db55d22c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55db55cba270 0x55db55cc5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9ce27fc700 0 client.25831 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9d11829b80 1 --2- 172.21.15.39:0/2958317046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db55cc2bc0 0x55db55cc3040 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9d11829b80 1 -- 172.21.15.39:0/2958317046 >> 172.21.15.39:0/2958317046 conn(0x55db55c121b0 msgr2=0x55db55cb7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9d11829b80 1 -- 172.21.15.39:0/2958317046 shutdown_connections 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9d11829b80 1 -- 172.21.15.39:0/2958317046 wait complete. 2024-05-24T11:49:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.404+0000 7f9d11829b80 20 client.25831 trim_cache size 0 max 16384 2024-05-24T11:49:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:49:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:49:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:49:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:49:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 -- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9ce40a1310 msgr2=0x7f9ce40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9ce40a1310 0x7f9ce40a37d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55db55caf790 tx=0x7f9cec009260 comp rx=0 tx=0).stop 2024-05-24T11:49:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 -- 172.21.15.39:0/2041586238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d00069360 msgr2=0x7f9d0019dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:49:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d00069360 0x7f9d0019dfc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000e9c0 tx=0x7f9cf000ecd0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 -- 172.21.15.39:0/2041586238 shutdown_connections 2024-05-24T11:49:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f9ce40a1310 0x7f9ce40a37d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0010cfb0 0x7f9d0010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d00069360 0x7f9d0019dfc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 --2- 172.21.15.39:0/2041586238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d00106260 0x7f9d0019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:49:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 -- 172.21.15.39:0/2041586238 >> 172.21.15.39:0/2041586238 conn(0x7f9d00076bc0 msgr2=0x7f9d00102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:49:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 -- 172.21.15.39:0/2041586238 shutdown_connections 2024-05-24T11:49:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:49:55.405+0000 7f9d0566e700 1 -- 172.21.15.39:0/2041586238 wait complete. 2024-05-24T11:49:56.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:55 smithi053 ceph-mon[47448]: pgmap v365: 97 pgs: 97 active+clean; 6.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2024-05-24T11:49:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:55 smithi137 ceph-mon[47907]: pgmap v365: 97 pgs: 97 active+clean; 6.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2024-05-24T11:49:56.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:55 smithi039 ceph-mon[42919]: pgmap v365: 97 pgs: 97 active+clean; 6.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2024-05-24T11:49:56.749 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:49:56 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setuser ceph since I am not root 2024-05-24T11:49:56.749 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:49:56 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setgroup ceph since I am not root 2024-05-24T11:49:56.749 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:49:56 smithi137 ceph-mds[84908]: -- 172.21.15.137:0/840406174 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558afa5641a0 con 0x558afa53b000 2024-05-24T11:49:56.749 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:49:56 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: starting mds.f at 2024-05-24T11:49:57.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:56 smithi053 ceph-mon[47448]: daemon mds.f restarted 2024-05-24T11:49:57.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:56 smithi053 ceph-mon[47448]: osdmap e92: 12 total, 12 up, 12 in 2024-05-24T11:49:57.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:56 smithi137 ceph-mon[47907]: daemon mds.f restarted 2024-05-24T11:49:57.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:56 smithi137 ceph-mon[47907]: osdmap e92: 12 total, 12 up, 12 in 2024-05-24T11:49:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:56 smithi039 ceph-mon[42919]: daemon mds.f restarted 2024-05-24T11:49:57.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:56 smithi039 ceph-mon[42919]: osdmap e92: 12 total, 12 up, 12 in 2024-05-24T11:49:58.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:57 smithi053 ceph-mon[47448]: pgmap v367: 97 pgs: 97 active+clean; 6.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2024-05-24T11:49:58.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:57 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6834/2891228595,v1:172.21.15.137:6835/2891228595] up:boot 2024-05-24T11:49:58.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:57 smithi053 ceph-mon[47448]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:49:58.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:57 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:49:58.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:57 smithi053 ceph-mon[47448]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:49:58.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:57 smithi137 ceph-mon[47907]: pgmap v367: 97 pgs: 97 active+clean; 6.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2024-05-24T11:49:58.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:57 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6834/2891228595,v1:172.21.15.137:6835/2891228595] up:boot 2024-05-24T11:49:58.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:57 smithi137 ceph-mon[47907]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:49:58.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:57 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:49:58.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:57 smithi137 ceph-mon[47907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:49:58.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:57 smithi039 ceph-mon[42919]: pgmap v367: 97 pgs: 97 active+clean; 6.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2024-05-24T11:49:58.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:57 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6834/2891228595,v1:172.21.15.137:6835/2891228595] up:boot 2024-05-24T11:49:58.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:57 smithi039 ceph-mon[42919]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:49:58.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:57 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:49:58.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:57 smithi039 ceph-mon[42919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:49:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 635.0 s, 890.6 tps, lat 112.494 ms stddev 41.940 2024-05-24T11:50:00.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:49:59 smithi053 ceph-mon[47448]: pgmap v368: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-05-24T11:50:00.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:49:59 smithi137 ceph-mon[47907]: pgmap v368: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-05-24T11:50:00.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:49:59 smithi039 ceph-mon[42919]: pgmap v368: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-05-24T11:50:01.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:00 smithi053 ceph-mon[47448]: overall HEALTH_OK 2024-05-24T11:50:01.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:00 smithi137 ceph-mon[47907]: overall HEALTH_OK 2024-05-24T11:50:01.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:00 smithi039 ceph-mon[42919]: overall HEALTH_OK 2024-05-24T11:50:02.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:01 smithi053 ceph-mon[47448]: pgmap v369: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-05-24T11:50:02.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:01 smithi137 ceph-mon[47907]: pgmap v369: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-05-24T11:50:02.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:01 smithi039 ceph-mon[42919]: pgmap v369: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-05-24T11:50:04.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 640.0 s, 807.8 tps, lat 123.512 ms stddev 69.598 2024-05-24T11:50:04.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:03 smithi053 ceph-mon[47448]: pgmap v370: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-05-24T11:50:04.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:03 smithi053 ceph-mon[47448]: osdmap e93: 12 total, 12 up, 12 in 2024-05-24T11:50:04.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:03 smithi137 ceph-mon[47907]: pgmap v370: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-05-24T11:50:04.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:03 smithi137 ceph-mon[47907]: osdmap e93: 12 total, 12 up, 12 in 2024-05-24T11:50:04.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:03 smithi039 ceph-mon[42919]: pgmap v370: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-05-24T11:50:04.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:03 smithi039 ceph-mon[42919]: osdmap e93: 12 total, 12 up, 12 in 2024-05-24T11:50:06.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:05 smithi053 ceph-mon[47448]: pgmap v372: 97 pgs: 13 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 56 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:50:06.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:05 smithi137 ceph-mon[47907]: pgmap v372: 97 pgs: 13 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 56 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:50:06.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:05 smithi039 ceph-mon[42919]: pgmap v372: 97 pgs: 13 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 56 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:50:08.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:07 smithi053 ceph-mon[47448]: pgmap v373: 97 pgs: 13 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 56 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-05-24T11:50:08.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:07 smithi137 ceph-mon[47907]: pgmap v373: 97 pgs: 13 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 56 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-05-24T11:50:08.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:07 smithi039 ceph-mon[42919]: pgmap v373: 97 pgs: 13 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 56 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-05-24T11:50:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.597+0000 7f4748048700 1 -- 172.21.15.39:0/739716543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47401118b0 msgr2=0x7f4740111c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.597+0000 7f4748048700 1 --2- 172.21.15.39:0/739716543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47401118b0 0x7f4740111c90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f473c0130d0 tx=0x7f473c0133e0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.597+0000 7f4748048700 1 -- 172.21.15.39:0/739716543 shutdown_connections 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.597+0000 7f4748048700 1 --2- 172.21.15.39:0/739716543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47401118b0 0x7f4740111c90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.597+0000 7f4748048700 1 --2- 172.21.15.39:0/739716543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f474010c270 0x7f4740118700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.597+0000 7f4748048700 1 --2- 172.21.15.39:0/739716543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474010b8b0 0x7f474010bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.597+0000 7f4748048700 1 -- 172.21.15.39:0/739716543 >> 172.21.15.39:0/739716543 conn(0x7f474006c020 msgr2=0x7f4740109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.597+0000 7f4748048700 1 -- 172.21.15.39:0/739716543 shutdown_connections 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.598+0000 7f4748048700 1 -- 172.21.15.39:0/739716543 wait complete. 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.598+0000 7f4748048700 1 Processor -- start 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.598+0000 7f4748048700 1 -- start start 2024-05-24T11:50:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4748048700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474010b8b0 0x7f47401a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4748048700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f474010c270 0x7f47401a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4748048700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47401118b0 0x7f474019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4748048700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474019e740 con 0x7f474010c270 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4748048700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474019e8b0 con 0x7f47401118b0 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4748048700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474019ea70 con 0x7f474010b8b0 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4745de4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474010b8b0 0x7f47401a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4745de4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474010b8b0 0x7f47401a2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:34258/0 (socket says 172.21.15.39:34258) 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4745de4700 1 -- 172.21.15.39:0/634153013 learned_addr learned my addr 172.21.15.39:0/634153013 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f47465e5700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47401118b0 0x7f474019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4745de4700 1 -- 172.21.15.39:0/634153013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47401118b0 msgr2=0x7f474019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4745de4700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47401118b0 0x7f474019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4745de4700 1 -- 172.21.15.39:0/634153013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f474010c270 msgr2=0x7f47401a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4745de4700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f474010c270 0x7f47401a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f4745de4700 1 -- 172.21.15.39:0/634153013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47300095f0 con 0x7f474010b8b0 2024-05-24T11:50:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.599+0000 7f47465e5700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47401118b0 0x7f474019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:50:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.600+0000 7f4745de4700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474010b8b0 0x7f47401a2560 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f473c015010 tx=0x7f473c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.600+0000 7f4736ffd700 1 -- 172.21.15.39:0/634153013 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c016040 con 0x7f474010b8b0 2024-05-24T11:50:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.600+0000 7f4748048700 1 -- 172.21.15.39:0/634153013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c011d30 con 0x7f474010b8b0 2024-05-24T11:50:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.600+0000 7f4736ffd700 1 -- 172.21.15.39:0/634153013 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f473c011870 con 0x7f474010b8b0 2024-05-24T11:50:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.600+0000 7f4736ffd700 1 -- 172.21.15.39:0/634153013 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c00da90 con 0x7f474010b8b0 2024-05-24T11:50:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.600+0000 7f4748048700 1 -- 172.21.15.39:0/634153013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47401aa4d0 con 0x7f474010b8b0 2024-05-24T11:50:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.603+0000 7f4736ffd700 1 -- 172.21.15.39:0/634153013 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f473c0048d0 con 0x7f474010b8b0 2024-05-24T11:50:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.604+0000 7f4736ffd700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f472c0a1350 0x7f472c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f4736ffd700 1 -- 172.21.15.39:0/634153013 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f473c0cde00 con 0x7f474010b8b0 2024-05-24T11:50:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f4754203b80 1 Processor -- start 2024-05-24T11:50:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f4754203b80 1 -- start start 2024-05-24T11:50:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f47455e3700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f472c0a1350 0x7f472c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f4754203b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b75cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f4754203b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b756f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f4754203b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b7578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f4754203b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b6ae0f0 con 0x55965b757480 2024-05-24T11:50:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f4754203b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b6abab0 con 0x55965b75c950 2024-05-24T11:50:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.605+0000 7f4754203b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b6a73a0 con 0x55965b75d270 2024-05-24T11:50:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47465e5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b7578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47465e5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b7578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51788/0 (socket says 172.21.15.39:51788) 2024-05-24T11:50:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47465e5700 1 -- 172.21.15.39:0/2851245954 learned_addr learned my addr 172.21.15.39:0/2851245954 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f4745de4700 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b75cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47455e3700 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b756f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47465e5700 1 -- 172.21.15.39:0/2851245954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 msgr2=0x55965b756f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47465e5700 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b756f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47465e5700 1 -- 172.21.15.39:0/2851245954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 msgr2=0x55965b75cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47465e5700 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b75cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47465e5700 1 -- 172.21.15.39:0/2851245954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55965b528350 con 0x55965b757480 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f4745de4700 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b75cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47455e3700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f472c0a1350 0x7f472c0a3810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f47300095c0 tx=0x7f47300126d0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.606+0000 7f47465e5700 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b7578a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4738009fd0 tx=0x7f4738007960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7db0c1e3c90dfd server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.607+0000 7f472bfff700 1 -- 172.21.15.39:0/2851245954 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738004bb0 con 0x55965b757480 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.607+0000 7f472bfff700 1 -- 172.21.15.39:0/2851245954 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f473800f040 con 0x55965b757480 2024-05-24T11:50:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.607+0000 7f472bfff700 1 -- 172.21.15.39:0/2851245954 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738005260 con 0x55965b757480 2024-05-24T11:50:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.607+0000 7f4754203b80 1 -- 172.21.15.39:0/2851245954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 msgr2=0x55965b7578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.607+0000 7f4754203b80 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b7578a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4738009fd0 tx=0x7f4738007960 comp rx=0 tx=0).stop 2024-05-24T11:50:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.607+0000 7f4754203b80 1 -- 172.21.15.39:0/2851245954 shutdown_connections 2024-05-24T11:50:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.607+0000 7f4754203b80 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b7578a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.608+0000 7f4754203b80 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b75cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.608+0000 7f4754203b80 1 --2- 172.21.15.39:0/2851245954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b756f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.608+0000 7f4754203b80 1 -- 172.21.15.39:0/2851245954 >> 172.21.15.39:0/2851245954 conn(0x55965b6b51b0 msgr2=0x55965b751840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.608+0000 7f4754203b80 1 -- 172.21.15.39:0/2851245954 shutdown_connections 2024-05-24T11:50:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.608+0000 7f4754203b80 1 -- 172.21.15.39:0/2851245954 wait complete. 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.608+0000 7f4754203b80 1 -- start start 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.608+0000 7f4754203b80 1 Processor -- start 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4754203b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b757480 0x55965b763140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4754203b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b7604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4754203b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75d270 0x55965b760a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4754203b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b643500 con 0x55965b75d270 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4754203b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b6431a0 con 0x55965b75c950 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4754203b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b52a640 con 0x55965b757480 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4754203b80 10 client.0 _ll_register_callbacks cb 0x55965b64cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4745de4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b757480 0x55965b763140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f47455e3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b7604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f47455e3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b7604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:40110/0 (socket says 172.21.15.39:40110) 2024-05-24T11:50:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4745de4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b757480 0x55965b763140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:34276/0 (socket says 172.21.15.39:34276) 2024-05-24T11:50:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f47455e3700 1 -- 172.21.15.39:0/1366795181 learned_addr learned my addr 172.21.15.39:0/1366795181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f47465e5700 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75d270 0x55965b760a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4745de4700 1 -- 172.21.15.39:0/1366795181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 msgr2=0x55965b7604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4745de4700 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b7604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4745de4700 1 -- 172.21.15.39:0/1366795181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75d270 msgr2=0x55965b760a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.609+0000 7f4745de4700 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75d270 0x55965b760a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.610+0000 7f4745de4700 1 -- 172.21.15.39:0/1366795181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47401aa4d0 con 0x55965b757480 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.610+0000 7f4745de4700 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b757480 0x55965b763140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f473c0cb4e0 tx=0x7f473c004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=174759dfbd9df448 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.610+0000 7f47297fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f47297fa700 1 -- 172.21.15.39:0/1366795181 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c004df0 con 0x55965b757480 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f47297fa700 1 -- 172.21.15.39:0/1366795181 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f473c09af30 con 0x55965b757480 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f4754203b80 1 -- 172.21.15.39:0/1366795181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55965b600210 con 0x55965b757480 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f47297fa700 1 -- 172.21.15.39:0/1366795181 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f473c09ed60 con 0x55965b757480 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f4754203b80 10 client.25849 fetch_fsmap learned FSMap version 31 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f4754203b80 1 -- 172.21.15.39:0/1366795181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f4738009db0 con 0x55965b757480 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f47297fa700 1 -- 172.21.15.39:0/1366795181 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55965b600210 con 0x55965b757480 2024-05-24T11:50:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f47297fa700 1 -- 172.21.15.39:0/1366795181 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f473c09b220 con 0x55965b757480 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f4754203b80 10 client.25849 fetch_fsmap finished waiting for FSMap version 31 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f4754203b80 10 client.25849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f4754203b80 20 client.25849 populate_metadata read hostname 'smithi039' 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f4754203b80 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55965b770a30 0x55965b772e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f4754203b80 4 client.25849 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.611+0000 7f4754203b80 1 -- 172.21.15.39:0/1366795181 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55965b7267b0 con 0x55965b770a30 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.612+0000 7f47455e3700 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55965b770a30 0x55965b772e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.612+0000 7f47297fa700 10 client.25849 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.612+0000 7f47455e3700 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55965b770a30 0x55965b772e20 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6f9cf595980f97 server_cookie=462a2e535de72b8f in_seq=0 out_seq=0 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.613+0000 7f47297fa700 1 -- 172.21.15.39:0/1366795181 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55965b7267b0 con 0x55965b770a30 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.613+0000 7f47297fa700 10 client.25849 handle_command_reply: tid=0 2024-05-24T11:50:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.613+0000 7f4754203b80 1 client.25849 shutdown 2024-05-24T11:50:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.615+0000 7f4754203b80 1 -- 172.21.15.39:0/1366795181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b757480 msgr2=0x55965b763140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.615+0000 7f4754203b80 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b757480 0x55965b763140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f473c0cb4e0 tx=0x7f473c004b80 comp rx=0 tx=0).stop 2024-05-24T11:50:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f4754203b80 1 -- 172.21.15.39:0/1366795181 shutdown_connections 2024-05-24T11:50:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f4754203b80 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b757480 0x55965b763140 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f4754203b80 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55965b770a30 0x55965b772e20 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f4754203b80 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75d270 0x55965b760a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f4754203b80 1 --2- 172.21.15.39:0/1366795181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b7604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f4754203b80 1 -- 172.21.15.39:0/1366795181 >> 172.21.15.39:0/1366795181 conn(0x55965b6b51b0 msgr2=0x55965b75b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f47297fa700 0 client.25849 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f4754203b80 1 -- 172.21.15.39:0/1366795181 shutdown_connections 2024-05-24T11:50:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f4754203b80 1 -- 172.21.15.39:0/1366795181 wait complete. 2024-05-24T11:50:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.616+0000 7f4754203b80 20 client.25849 trim_cache size 0 max 16384 2024-05-24T11:50:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.640+0000 7f4754203b80 1 Processor -- start 2024-05-24T11:50:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.640+0000 7f4754203b80 1 -- start start 2024-05-24T11:50:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.640+0000 7f4754203b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b757860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.640+0000 7f4754203b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b751910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.640+0000 7f4754203b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b762c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.640+0000 7f4754203b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b761070 con 0x55965b757480 2024-05-24T11:50:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.640+0000 7f4754203b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b76d710 con 0x55965b75c950 2024-05-24T11:50:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.640+0000 7f4754203b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b761a30 con 0x55965b75d270 2024-05-24T11:50:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.640+0000 7f47455e3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b751910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.641+0000 7f4745de4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b757860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.641+0000 7f47455e3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b751910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:40114/0 (socket says 172.21.15.39:40114) 2024-05-24T11:50:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.641+0000 7f47455e3700 1 -- 172.21.15.39:0/60741267 learned_addr learned my addr 172.21.15.39:0/60741267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.641+0000 7f47455e3700 1 -- 172.21.15.39:0/60741267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 msgr2=0x55965b762c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:50:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.641+0000 7f47455e3700 1 --2- 172.21.15.39:0/60741267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b762c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.641+0000 7f47455e3700 1 -- 172.21.15.39:0/60741267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 msgr2=0x55965b757860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.641+0000 7f47455e3700 1 --2- 172.21.15.39:0/60741267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b757860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.641+0000 7f47455e3700 1 -- 172.21.15.39:0/60741267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55965b4d4b00 con 0x55965b75c950 2024-05-24T11:50:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.641+0000 7f47455e3700 1 --2- 172.21.15.39:0/60741267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b751910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55965b749a80 tx=0x7f4730015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71fe08f37204ee0f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.642+0000 7f4734ff9700 1 -- 172.21.15.39:0/60741267 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730021070 con 0x55965b75c950 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.642+0000 7f4734ff9700 1 -- 172.21.15.39:0/60741267 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4730022040 con 0x55965b75c950 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.642+0000 7f4734ff9700 1 -- 172.21.15.39:0/60741267 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730025a90 con 0x55965b75c950 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.642+0000 7f4754203b80 1 -- 172.21.15.39:0/60741267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 msgr2=0x55965b751910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.642+0000 7f4754203b80 1 --2- 172.21.15.39:0/60741267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b751910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55965b749a80 tx=0x7f4730015000 comp rx=0 tx=0).stop 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.643+0000 7f4754203b80 1 -- 172.21.15.39:0/60741267 shutdown_connections 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.643+0000 7f4754203b80 1 --2- 172.21.15.39:0/60741267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b762c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.643+0000 7f4754203b80 1 --2- 172.21.15.39:0/60741267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b757480 0x55965b757860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.643+0000 7f4754203b80 1 --2- 172.21.15.39:0/60741267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b75c950 0x55965b751910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.643+0000 7f4754203b80 1 -- 172.21.15.39:0/60741267 >> 172.21.15.39:0/60741267 conn(0x55965b6b51b0 msgr2=0x55965b7555d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.643+0000 7f4754203b80 1 -- 172.21.15.39:0/60741267 shutdown_connections 2024-05-24T11:50:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.643+0000 7f4754203b80 1 -- 172.21.15.39:0/60741267 wait complete. 2024-05-24T11:50:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.643+0000 7f4754203b80 1 -- start start 2024-05-24T11:50:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.643+0000 7f4754203b80 1 Processor -- start 2024-05-24T11:50:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f4754203b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b757480 0x55965b765cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f4754203b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75c950 0x55965b766230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f4754203b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b7c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f4754203b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b766b40 con 0x55965b75c950 2024-05-24T11:50:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f4754203b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b7c6440 con 0x55965b757480 2024-05-24T11:50:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f4754203b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55965b7c6580 con 0x55965b75d270 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f4754203b80 10 client.0 _ll_register_callbacks cb 0x55965b64cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f47465e5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b7c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f47465e5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b7c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:34290/0 (socket says 172.21.15.39:34290) 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f47465e5700 1 -- 172.21.15.39:0/1860717743 learned_addr learned my addr 172.21.15.39:0/1860717743 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f47455e3700 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75c950 0x55965b766230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.644+0000 7f4745de4700 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b757480 0x55965b765cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47465e5700 1 -- 172.21.15.39:0/1860717743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b757480 msgr2=0x55965b765cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47465e5700 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b757480 0x55965b765cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47465e5700 1 -- 172.21.15.39:0/1860717743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75c950 msgr2=0x55965b766230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47465e5700 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75c950 0x55965b766230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47465e5700 1 -- 172.21.15.39:0/1860717743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4730018050 con 0x55965b75d270 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f4745de4700 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b757480 0x55965b765cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47465e5700 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b7c5f00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4738009230 tx=0x7f4738015d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4cff936d0a8479f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47297fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47297fa700 1 -- 172.21.15.39:0/1860717743 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738003c10 con 0x55965b75d270 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47297fa700 1 -- 172.21.15.39:0/1860717743 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4738003d70 con 0x55965b75d270 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f4754203b80 1 -- 172.21.15.39:0/1860717743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f473800f040 con 0x55965b75d270 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f47297fa700 1 -- 172.21.15.39:0/1860717743 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4738005070 con 0x55965b75d270 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f4754203b80 10 client.25852 fetch_fsmap learned FSMap version 31 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.645+0000 7f4754203b80 1 -- 172.21.15.39:0/1860717743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55965b66d500 con 0x55965b75d270 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.646+0000 7f47297fa700 1 -- 172.21.15.39:0/1860717743 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f473800f040 con 0x55965b75d270 2024-05-24T11:50:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.646+0000 7f47297fa700 1 -- 172.21.15.39:0/1860717743 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f473801e6d0 con 0x55965b75d270 2024-05-24T11:50:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.646+0000 7f4754203b80 10 client.25852 fetch_fsmap finished waiting for FSMap version 31 2024-05-24T11:50:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.646+0000 7f4754203b80 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:50:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.646+0000 7f4754203b80 20 client.25852 populate_metadata read hostname 'smithi039' 2024-05-24T11:50:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.646+0000 7f4754203b80 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55965b7c7c40 0x55965b7ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.646+0000 7f4754203b80 4 client.25852 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:50:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.646+0000 7f4754203b80 1 -- 172.21.15.39:0/1860717743 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55965b69fe60 con 0x55965b7c7c40 2024-05-24T11:50:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.647+0000 7f4745de4700 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55965b7c7c40 0x55965b7ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.647+0000 7f4745de4700 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55965b7c7c40 0x55965b7ca060 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e996d7f3361f86fb server_cookie=c4a9b3d4b0171324 in_seq=0 out_seq=0 2024-05-24T11:50:08.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.647+0000 7f47297fa700 10 client.25852 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.676+0000 7f47297fa700 1 -- 172.21.15.39:0/1860717743 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55965b69fe60 con 0x55965b7c7c40 2024-05-24T11:50:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.676+0000 7f47297fa700 10 client.25852 handle_command_reply: tid=0 2024-05-24T11:50:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.678+0000 7f4754203b80 1 client.25852 shutdown 2024-05-24T11:50:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f4754203b80 1 -- 172.21.15.39:0/1860717743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 msgr2=0x55965b7c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f4754203b80 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b7c5f00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4738009230 tx=0x7f4738015d60 comp rx=0 tx=0).stop 2024-05-24T11:50:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f4754203b80 1 -- 172.21.15.39:0/1860717743 shutdown_connections 2024-05-24T11:50:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f4754203b80 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965b75d270 0x55965b7c5f00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f4754203b80 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55965b7c7c40 0x55965b7ca060 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f4754203b80 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55965b757480 0x55965b765cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f4754203b80 1 --2- 172.21.15.39:0/1860717743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55965b75c950 0x55965b766230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f47297fa700 0 client.25852 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f4754203b80 1 -- 172.21.15.39:0/1860717743 >> 172.21.15.39:0/1860717743 conn(0x55965b6b51b0 msgr2=0x55965b75ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.679+0000 7f4754203b80 1 -- 172.21.15.39:0/1860717743 shutdown_connections 2024-05-24T11:50:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.682+0000 7f4754203b80 1 -- 172.21.15.39:0/1860717743 wait complete. 2024-05-24T11:50:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.682+0000 7f4754203b80 20 client.25852 trim_cache size 0 max 16384 2024-05-24T11:50:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:50:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:50:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:50:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:50:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 -- 172.21.15.39:0/634153013 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f472c0a1350 msgr2=0x7f472c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f472c0a1350 0x7f472c0a3810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f47300095c0 tx=0x7f47300126d0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 -- 172.21.15.39:0/634153013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474010b8b0 msgr2=0x7f47401a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474010b8b0 0x7f47401a2560 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f473c015010 tx=0x7f473c005270 comp rx=0 tx=0).stop 2024-05-24T11:50:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 -- 172.21.15.39:0/634153013 shutdown_connections 2024-05-24T11:50:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474010b8b0 0x7f47401a2560 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f472c0a1350 0x7f472c0a3810 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47401118b0 0x7f474019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 --2- 172.21.15.39:0/634153013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f474010c270 0x7f47401a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 -- 172.21.15.39:0/634153013 >> 172.21.15.39:0/634153013 conn(0x7f474006c020 msgr2=0x7f474010a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 -- 172.21.15.39:0/634153013 shutdown_connections 2024-05-24T11:50:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:08.683+0000 7f4748048700 1 -- 172.21.15.39:0/634153013 wait complete. 2024-05-24T11:50:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 645.0 s, 869.0 tps, lat 114.790 ms stddev 48.344 2024-05-24T11:50:09.495 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:50:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setuser ceph since I am not root 2024-05-24T11:50:09.495 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:50:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setgroup ceph since I am not root 2024-05-24T11:50:09.495 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:50:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: starting mds.f at 2024-05-24T11:50:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:09 smithi053 ceph-mon[47448]: pgmap v374: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2024-05-24T11:50:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:09 smithi053 ceph-mon[47448]: daemon mds.f restarted 2024-05-24T11:50:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:09 smithi053 ceph-mon[47448]: osdmap e94: 12 total, 12 up, 12 in 2024-05-24T11:50:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:09 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6834/904038971,v1:172.21.15.137:6835/904038971] up:boot 2024-05-24T11:50:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:09 smithi053 ceph-mon[47448]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:50:10.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:09 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:50:10.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:09 smithi137 ceph-mon[47907]: pgmap v374: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2024-05-24T11:50:10.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:09 smithi137 ceph-mon[47907]: daemon mds.f restarted 2024-05-24T11:50:10.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:09 smithi137 ceph-mon[47907]: osdmap e94: 12 total, 12 up, 12 in 2024-05-24T11:50:10.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:09 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6834/904038971,v1:172.21.15.137:6835/904038971] up:boot 2024-05-24T11:50:10.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:09 smithi137 ceph-mon[47907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:50:10.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:09 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:50:10.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:09 smithi039 ceph-mon[42919]: pgmap v374: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2024-05-24T11:50:10.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:09 smithi039 ceph-mon[42919]: daemon mds.f restarted 2024-05-24T11:50:10.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:09 smithi039 ceph-mon[42919]: osdmap e94: 12 total, 12 up, 12 in 2024-05-24T11:50:10.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:09 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6834/904038971,v1:172.21.15.137:6835/904038971] up:boot 2024-05-24T11:50:10.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:09 smithi039 ceph-mon[42919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:50:10.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:09 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:50:12.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:11 smithi053 ceph-mon[47448]: pgmap v376: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 20 MiB/s wr, 1.69k op/s 2024-05-24T11:50:12.247 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:11 smithi137 ceph-mon[47907]: pgmap v376: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 20 MiB/s wr, 1.69k op/s 2024-05-24T11:50:12.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:11 smithi039 ceph-mon[42919]: pgmap v376: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 20 MiB/s wr, 1.69k op/s 2024-05-24T11:50:14.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 650.0 s, 853.7 tps, lat 117.302 ms stddev 82.621 2024-05-24T11:50:14.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:13 smithi137 ceph-mon[47907]: pgmap v377: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-05-24T11:50:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:13 smithi039 ceph-mon[42919]: pgmap v377: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-05-24T11:50:14.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:13 smithi053 ceph-mon[47448]: pgmap v377: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-05-24T11:50:16.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:15 smithi039 ceph-mon[42919]: pgmap v378: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-05-24T11:50:16.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:15 smithi039 ceph-mon[42919]: osdmap e95: 12 total, 12 up, 12 in 2024-05-24T11:50:16.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:15 smithi053 ceph-mon[47448]: pgmap v378: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-05-24T11:50:16.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:15 smithi053 ceph-mon[47448]: osdmap e95: 12 total, 12 up, 12 in 2024-05-24T11:50:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:15 smithi137 ceph-mon[47907]: pgmap v378: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-05-24T11:50:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:15 smithi137 ceph-mon[47907]: osdmap e95: 12 total, 12 up, 12 in 2024-05-24T11:50:18.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:17 smithi039 ceph-mon[42919]: pgmap v380: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-05-24T11:50:18.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:17 smithi053 ceph-mon[47448]: pgmap v380: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-05-24T11:50:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:17 smithi137 ceph-mon[47907]: pgmap v380: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-05-24T11:50:19.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 655.0 s, 889.9 tps, lat 112.825 ms stddev 63.394 2024-05-24T11:50:20.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:19 smithi039 ceph-mon[42919]: pgmap v381: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-05-24T11:50:20.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:19 smithi053 ceph-mon[47448]: pgmap v381: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-05-24T11:50:20.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:19 smithi137 ceph-mon[47907]: pgmap v381: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-05-24T11:50:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.865+0000 7efeeca82700 1 -- 172.21.15.39:0/3201495153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee81055e0 msgr2=0x7efee81059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.865+0000 7efeeca82700 1 --2- 172.21.15.39:0/3201495153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee81055e0 0x7efee81059c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efedc0099c0 tx=0x7efedc009cd0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.865+0000 7efeeca82700 1 -- 172.21.15.39:0/3201495153 shutdown_connections 2024-05-24T11:50:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.865+0000 7efeeca82700 1 --2- 172.21.15.39:0/3201495153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee80ffee0 0x7efee810c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.865+0000 7efeeca82700 1 --2- 172.21.15.39:0/3201495153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee81055e0 0x7efee81059c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.865+0000 7efeeca82700 1 --2- 172.21.15.39:0/3201495153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee80ff520 0x7efee80ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.865+0000 7efeeca82700 1 -- 172.21.15.39:0/3201495153 >> 172.21.15.39:0/3201495153 conn(0x7efee80763a0 msgr2=0x7efee80fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.865+0000 7efeeca82700 1 -- 172.21.15.39:0/3201495153 shutdown_connections 2024-05-24T11:50:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.865+0000 7efeeca82700 1 -- 172.21.15.39:0/3201495153 wait complete. 2024-05-24T11:50:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.866+0000 7efeeca82700 1 Processor -- start 2024-05-24T11:50:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.866+0000 7efeeca82700 1 -- start start 2024-05-24T11:50:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.866+0000 7efeeca82700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee80ff520 0x7efee81063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.866+0000 7efeeca82700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee80ffee0 0x7efee81068f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.866+0000 7efeeca82700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee81055e0 0x7efee8106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.866+0000 7efeeca82700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee81a9ac0 con 0x7efee81055e0 2024-05-24T11:50:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.866+0000 7efeeca82700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee81991d0 con 0x7efee80ffee0 2024-05-24T11:50:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.866+0000 7efeeca82700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee8199340 con 0x7efee80ff520 2024-05-24T11:50:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee5d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee80ffee0 0x7efee81068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee5d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee80ffee0 0x7efee81068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49262/0 (socket says 172.21.15.39:49262) 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee5d9b700 1 -- 172.21.15.39:0/642451823 learned_addr learned my addr 172.21.15.39:0/642451823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee659c700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee80ff520 0x7efee81063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee5d9b700 1 -- 172.21.15.39:0/642451823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee80ff520 msgr2=0x7efee81063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee5d9b700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee80ff520 0x7efee81063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee5d9b700 1 -- 172.21.15.39:0/642451823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee81055e0 msgr2=0x7efee8106e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee5d9b700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee81055e0 0x7efee8106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee5d9b700 1 -- 172.21.15.39:0/642451823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc0095f0 con 0x7efee80ffee0 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee659c700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee80ff520 0x7efee81063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efee5d9b700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee80ffee0 0x7efee81068f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efed800e9a0 tx=0x7efed800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.867+0000 7efed77fe700 1 -- 172.21.15.39:0/642451823 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8009790 con 0x7efee80ffee0 2024-05-24T11:50:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.868+0000 7efed77fe700 1 -- 172.21.15.39:0/642451823 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efed800cbd0 con 0x7efee80ffee0 2024-05-24T11:50:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.868+0000 7efeeca82700 1 -- 172.21.15.39:0/642451823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee8199620 con 0x7efee80ffee0 2024-05-24T11:50:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.868+0000 7efed77fe700 1 -- 172.21.15.39:0/642451823 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8018680 con 0x7efee80ffee0 2024-05-24T11:50:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.868+0000 7efeeca82700 1 -- 172.21.15.39:0/642451823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee8199b30 con 0x7efee80ffee0 2024-05-24T11:50:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.871+0000 7efed77fe700 1 -- 172.21.15.39:0/642451823 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efed80187e0 con 0x7efee80ffee0 2024-05-24T11:50:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.871+0000 7efed77fe700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efed00a12d0 0x7efed00a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.872+0000 7efed77fe700 1 -- 172.21.15.39:0/642451823 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efed8014070 con 0x7efee80ffee0 2024-05-24T11:50:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.872+0000 7efee659c700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efed00a12d0 0x7efed00a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.872+0000 7efef8c3db80 1 Processor -- start 2024-05-24T11:50:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.873+0000 7efef8c3db80 1 -- start start 2024-05-24T11:50:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.873+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c262d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.873+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c25cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.873+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c25d480 0x562e8c25d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.873+0000 7efef8c3db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c1b40f0 con 0x562e8c25d480 2024-05-24T11:50:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.873+0000 7efef8c3db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c1b1ab0 con 0x562e8c263270 2024-05-24T11:50:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.873+0000 7efef8c3db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c1ad3a0 con 0x562e8c262950 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee5d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c25cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee5d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c25cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49272/0 (socket says 172.21.15.39:49272) 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee5d9b700 1 -- 172.21.15.39:0/870436315 learned_addr learned my addr 172.21.15.39:0/870436315 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee659c700 1 --2- 172.21.15.39:0/870436315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c262d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee659c700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efed00a12d0 0x7efed00a3790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efedc009990 tx=0x7efedc019040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee5d9b700 1 -- 172.21.15.39:0/870436315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 msgr2=0x562e8c262d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee5d9b700 1 --2- 172.21.15.39:0/870436315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c262d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee5d9b700 1 -- 172.21.15.39:0/870436315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c25d480 msgr2=0x562e8c25d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee5d9b700 1 --2- 172.21.15.39:0/870436315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c25d480 0x562e8c25d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee5d9b700 1 -- 172.21.15.39:0/870436315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e8c02e350 con 0x562e8c263270 2024-05-24T11:50:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.874+0000 7efee5d9b700 1 --2- 172.21.15.39:0/870436315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c25cf40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efed80a1fe0 tx=0x7efed80999f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cbde74c3c42c9d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.875+0000 7efed4ff9700 1 -- 172.21.15.39:0/870436315 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed809a090 con 0x562e8c263270 2024-05-24T11:50:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.875+0000 7efed4ff9700 1 -- 172.21.15.39:0/870436315 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efed809dec0 con 0x562e8c263270 2024-05-24T11:50:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.875+0000 7efed4ff9700 1 -- 172.21.15.39:0/870436315 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8099580 con 0x562e8c263270 2024-05-24T11:50:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.876+0000 7efef8c3db80 1 -- 172.21.15.39:0/870436315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 msgr2=0x562e8c25cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.876+0000 7efef8c3db80 1 --2- 172.21.15.39:0/870436315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c25cf40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efed80a1fe0 tx=0x7efed80999f0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.876+0000 7efef8c3db80 1 -- 172.21.15.39:0/870436315 shutdown_connections 2024-05-24T11:50:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.876+0000 7efef8c3db80 1 --2- 172.21.15.39:0/870436315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c262d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.876+0000 7efef8c3db80 1 --2- 172.21.15.39:0/870436315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c25d480 0x562e8c25d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.876+0000 7efef8c3db80 1 --2- 172.21.15.39:0/870436315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c25cf40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.876+0000 7efef8c3db80 1 -- 172.21.15.39:0/870436315 >> 172.21.15.39:0/870436315 conn(0x562e8c1bb1b0 msgr2=0x562e8c257840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.876+0000 7efef8c3db80 1 -- 172.21.15.39:0/870436315 shutdown_connections 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.877+0000 7efef8c3db80 1 -- 172.21.15.39:0/870436315 wait complete. 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.877+0000 7efef8c3db80 1 -- start start 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.877+0000 7efef8c3db80 1 Processor -- start 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.877+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c25d480 0x562e8c266660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.877+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c266ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.877+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c2670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.877+0000 7efef8c3db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c149500 con 0x562e8c25d480 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.877+0000 7efef8c3db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c1491a0 con 0x562e8c263270 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.877+0000 7efef8c3db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c030640 con 0x562e8c262950 2024-05-24T11:50:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efef8c3db80 10 client.0 _ll_register_callbacks cb 0x562e8c152ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:50:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c266ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee5d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c266ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47882/0 (socket says 172.21.15.39:47882) 2024-05-24T11:50:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee5d9b700 1 -- 172.21.15.39:0/1481289591 learned_addr learned my addr 172.21.15.39:0/1481289591 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee659c700 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c25d480 0x562e8c266660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee6d9d700 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c2670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee5d9b700 1 -- 172.21.15.39:0/1481289591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 msgr2=0x562e8c2670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee5d9b700 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c2670e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee5d9b700 1 -- 172.21.15.39:0/1481289591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c25d480 msgr2=0x562e8c266660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee5d9b700 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c25d480 0x562e8c266660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee5d9b700 1 -- 172.21.15.39:0/1481289591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee8199b30 con 0x562e8c262950 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efee5d9b700 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c266ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efed80ca780 tx=0x7efed8095d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9680346a1be3ca8f server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.878+0000 7efecdffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.879+0000 7efecdffb700 1 -- 172.21.15.39:0/1481289591 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed809a090 con 0x562e8c262950 2024-05-24T11:50:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.879+0000 7efecdffb700 1 -- 172.21.15.39:0/1481289591 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efed8099790 con 0x562e8c262950 2024-05-24T11:50:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.879+0000 7efef8c3db80 1 -- 172.21.15.39:0/1481289591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e8c106210 con 0x562e8c262950 2024-05-24T11:50:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.879+0000 7efecdffb700 1 -- 172.21.15.39:0/1481289591 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed80b82c0 con 0x562e8c262950 2024-05-24T11:50:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.879+0000 7efef8c3db80 10 client.25870 fetch_fsmap learned FSMap version 32 2024-05-24T11:50:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.879+0000 7efef8c3db80 1 -- 172.21.15.39:0/1481289591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562e8bfdab00 con 0x562e8c262950 2024-05-24T11:50:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.879+0000 7efecdffb700 1 -- 172.21.15.39:0/1481289591 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562e8c106210 con 0x562e8c262950 2024-05-24T11:50:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.880+0000 7efecdffb700 1 -- 172.21.15.39:0/1481289591 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efed8095fa0 con 0x562e8c262950 2024-05-24T11:50:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.880+0000 7efef8c3db80 10 client.25870 fetch_fsmap finished waiting for FSMap version 32 2024-05-24T11:50:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.880+0000 7efef8c3db80 10 client.25870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:50:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.880+0000 7efef8c3db80 20 client.25870 populate_metadata read hostname 'smithi039' 2024-05-24T11:50:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.881+0000 7efef8c3db80 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e8c276930 0x562e8c278d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.881+0000 7efef8c3db80 4 client.25870 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:50:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.881+0000 7efef8c3db80 1 -- 172.21.15.39:0/1481289591 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e8c22c7b0 con 0x562e8c276930 2024-05-24T11:50:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.881+0000 7efee659c700 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e8c276930 0x562e8c278d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.881+0000 7efecdffb700 10 client.25870 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.881+0000 7efee659c700 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e8c276930 0x562e8c278d20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5452812997f2ef96 server_cookie=adc3906ccd7245f0 in_seq=0 out_seq=0 2024-05-24T11:50:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.882+0000 7efecdffb700 1 -- 172.21.15.39:0/1481289591 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e8c22c7b0 con 0x562e8c276930 2024-05-24T11:50:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.882+0000 7efecdffb700 10 client.25870 handle_command_reply: tid=0 2024-05-24T11:50:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.882+0000 7efef8c3db80 1 client.25870 shutdown 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.882+0000 7efef8c3db80 1 -- 172.21.15.39:0/1481289591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 msgr2=0x562e8c266ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.882+0000 7efef8c3db80 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c266ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efed80ca780 tx=0x7efed8095d60 comp rx=0 tx=0).stop 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efef8c3db80 1 -- 172.21.15.39:0/1481289591 shutdown_connections 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efef8c3db80 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e8c276930 0x562e8c278d20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efef8c3db80 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c263270 0x562e8c2670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efecdffb700 0 client.25870 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efef8c3db80 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c25d480 0x562e8c266660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efef8c3db80 1 --2- 172.21.15.39:0/1481289591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c262950 0x562e8c266ba0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efef8c3db80 1 -- 172.21.15.39:0/1481289591 >> 172.21.15.39:0/1481289591 conn(0x562e8c1bb1b0 msgr2=0x562e8c2614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efef8c3db80 1 -- 172.21.15.39:0/1481289591 shutdown_connections 2024-05-24T11:50:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efef8c3db80 1 -- 172.21.15.39:0/1481289591 wait complete. 2024-05-24T11:50:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.883+0000 7efef8c3db80 20 client.25870 trim_cache size 0 max 16384 2024-05-24T11:50:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.908+0000 7efef8c3db80 1 Processor -- start 2024-05-24T11:50:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.908+0000 7efef8c3db80 1 -- start start 2024-05-24T11:50:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.908+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c25d480 0x562e8c257910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.908+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c262950 0x562e8c265f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.908+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c263270 0x562e8c26dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.908+0000 7efef8c3db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c258330 con 0x562e8c262950 2024-05-24T11:50:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.908+0000 7efef8c3db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c2691f0 con 0x562e8c25d480 2024-05-24T11:50:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.908+0000 7efef8c3db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c273870 con 0x562e8c263270 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c262950 0x562e8c265f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c25d480 0x562e8c257910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c25d480 0x562e8c257910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49290/0 (socket says 172.21.15.39:49290) 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c262950 0x562e8c265f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56806/0 (socket says 172.21.15.39:56806) 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee659c700 1 -- 172.21.15.39:0/200208795 learned_addr learned my addr 172.21.15.39:0/200208795 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee6d9d700 1 --2- 172.21.15.39:0/200208795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c263270 0x562e8c26dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee6d9d700 1 -- 172.21.15.39:0/200208795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c25d480 msgr2=0x562e8c257910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee6d9d700 1 --2- 172.21.15.39:0/200208795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c25d480 0x562e8c257910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee6d9d700 1 -- 172.21.15.39:0/200208795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c262950 msgr2=0x562e8c265f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee6d9d700 1 --2- 172.21.15.39:0/200208795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c262950 0x562e8c265f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.909+0000 7efee6d9d700 1 -- 172.21.15.39:0/200208795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e8c173500 con 0x562e8c263270 2024-05-24T11:50:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.910+0000 7efee6d9d700 1 --2- 172.21.15.39:0/200208795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c263270 0x562e8c26dbd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efee0007ae0 tx=0x7efee0007df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=911c19f25368f7fd server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.910+0000 7efed57fa700 1 -- 172.21.15.39:0/200208795 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee000ca50 con 0x562e8c263270 2024-05-24T11:50:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.910+0000 7efed57fa700 1 -- 172.21.15.39:0/200208795 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efee000cbb0 con 0x562e8c263270 2024-05-24T11:50:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.910+0000 7efed57fa700 1 -- 172.21.15.39:0/200208795 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0010350 con 0x562e8c263270 2024-05-24T11:50:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.911+0000 7efef8c3db80 1 -- 172.21.15.39:0/200208795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c263270 msgr2=0x562e8c26dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.911+0000 7efef8c3db80 1 --2- 172.21.15.39:0/200208795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c263270 0x562e8c26dbd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efee0007ae0 tx=0x7efee0007df0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.911+0000 7efef8c3db80 1 -- 172.21.15.39:0/200208795 shutdown_connections 2024-05-24T11:50:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.911+0000 7efef8c3db80 1 --2- 172.21.15.39:0/200208795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c263270 0x562e8c26dbd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.911+0000 7efef8c3db80 1 --2- 172.21.15.39:0/200208795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c25d480 0x562e8c257910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.911+0000 7efef8c3db80 1 --2- 172.21.15.39:0/200208795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c262950 0x562e8c265f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.911+0000 7efef8c3db80 1 -- 172.21.15.39:0/200208795 >> 172.21.15.39:0/200208795 conn(0x562e8c1bb1b0 msgr2=0x562e8c253e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.911+0000 7efef8c3db80 1 -- 172.21.15.39:0/200208795 shutdown_connections 2024-05-24T11:50:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.912+0000 7efef8c3db80 1 -- 172.21.15.39:0/200208795 wait complete. 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.912+0000 7efef8c3db80 1 -- start start 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.912+0000 7efef8c3db80 1 Processor -- start 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c25d480 0x562e8c26bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c262950 0x562e8c26c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efef8c3db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c263270 0x562e8c2cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efef8c3db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c26ca20 con 0x562e8c263270 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efef8c3db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c2cc2d0 con 0x562e8c262950 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efef8c3db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e8c2cc460 con 0x562e8c25d480 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efef8c3db80 10 client.0 _ll_register_callbacks cb 0x562e8c152ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c25d480 0x562e8c26bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c25d480 0x562e8c26bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:47908/0 (socket says 172.21.15.39:47908) 2024-05-24T11:50:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee659c700 1 -- 172.21.15.39:0/3213067487 learned_addr learned my addr 172.21.15.39:0/3213067487 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee6d9d700 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c263270 0x562e8c2cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee659c700 1 -- 172.21.15.39:0/3213067487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c262950 msgr2=0x562e8c26c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee5d9b700 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c262950 0x562e8c26c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee659c700 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c262950 0x562e8c26c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee659c700 1 -- 172.21.15.39:0/3213067487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c263270 msgr2=0x562e8c2cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee659c700 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c263270 0x562e8c2cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee659c700 1 -- 172.21.15.39:0/3213067487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee0007760 con 0x562e8c25d480 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee6d9d700 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c263270 0x562e8c2cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee5d9b700 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c262950 0x562e8c26c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efee659c700 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c25d480 0x562e8c26bdb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efedc006010 tx=0x7efedc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e0ed68b484dbbe server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.913+0000 7efecdffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:50:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.914+0000 7efecdffb700 1 -- 172.21.15.39:0/3213067487 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc00fee0 con 0x562e8c25d480 2024-05-24T11:50:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.914+0000 7efecdffb700 1 -- 172.21.15.39:0/3213067487 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efedc034620 con 0x562e8c25d480 2024-05-24T11:50:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.914+0000 7efef8c3db80 1 -- 172.21.15.39:0/3213067487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efed809dec0 con 0x562e8c25d480 2024-05-24T11:50:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.914+0000 7efecdffb700 1 -- 172.21.15.39:0/3213067487 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7efedc0347e0 con 0x562e8c25d480 2024-05-24T11:50:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.914+0000 7efef8c3db80 10 client.25876 fetch_fsmap learned FSMap version 32 2024-05-24T11:50:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.914+0000 7efef8c3db80 1 -- 172.21.15.39:0/3213067487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562e8c032c70 con 0x562e8c25d480 2024-05-24T11:50:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.914+0000 7efecdffb700 1 -- 172.21.15.39:0/3213067487 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efed809dec0 con 0x562e8c25d480 2024-05-24T11:50:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.914+0000 7efecdffb700 1 -- 172.21.15.39:0/3213067487 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efedc005350 con 0x562e8c25d480 2024-05-24T11:50:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.914+0000 7efef8c3db80 10 client.25876 fetch_fsmap finished waiting for FSMap version 32 2024-05-24T11:50:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.915+0000 7efef8c3db80 10 client.25876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:50:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.915+0000 7efef8c3db80 20 client.25876 populate_metadata read hostname 'smithi039' 2024-05-24T11:50:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.915+0000 7efef8c3db80 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e8c2cdc70 0x562e8c2d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.915+0000 7efef8c3db80 4 client.25876 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:50:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.915+0000 7efef8c3db80 1 -- 172.21.15.39:0/3213067487 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562e8c1a5e60 con 0x562e8c2cdc70 2024-05-24T11:50:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.915+0000 7efee5d9b700 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e8c2cdc70 0x562e8c2d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.915+0000 7efecdffb700 10 client.25876 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.915+0000 7efee5d9b700 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e8c2cdc70 0x562e8c2d0090 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb384fb9cf3811e server_cookie=13e4605c1d5344a6 in_seq=0 out_seq=0 2024-05-24T11:50:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.965+0000 7efecdffb700 1 -- 172.21.15.39:0/3213067487 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562e8c1a5e60 con 0x562e8c2cdc70 2024-05-24T11:50:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.965+0000 7efecdffb700 10 client.25876 handle_command_reply: tid=0 2024-05-24T11:50:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.965+0000 7efef8c3db80 1 client.25876 shutdown 2024-05-24T11:50:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 -- 172.21.15.39:0/3213067487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c25d480 msgr2=0x562e8c26bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c25d480 0x562e8c26bdb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efedc006010 tx=0x7efedc009310 comp rx=0 tx=0).stop 2024-05-24T11:50:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 -- 172.21.15.39:0/3213067487 shutdown_connections 2024-05-24T11:50:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562e8c25d480 0x562e8c26bdb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562e8c2cdc70 0x562e8c2d0090 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e8c263270 0x562e8c2cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 --2- 172.21.15.39:0/3213067487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562e8c262950 0x562e8c26c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 -- 172.21.15.39:0/3213067487 >> 172.21.15.39:0/3213067487 conn(0x562e8c1bb1b0 msgr2=0x562e8c25c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efecdffb700 0 client.25876 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 -- 172.21.15.39:0/3213067487 shutdown_connections 2024-05-24T11:50:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.966+0000 7efef8c3db80 1 -- 172.21.15.39:0/3213067487 wait complete. 2024-05-24T11:50:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:50:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:50:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:50:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:50:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.967+0000 7efef8c3db80 20 client.25876 trim_cache size 0 max 16384 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.967+0000 7efeeca82700 1 -- 172.21.15.39:0/642451823 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efed00a12d0 msgr2=0x7efed00a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.967+0000 7efeeca82700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efed00a12d0 0x7efed00a3790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efedc009990 tx=0x7efedc019040 comp rx=0 tx=0).stop 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.967+0000 7efeeca82700 1 -- 172.21.15.39:0/642451823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee80ffee0 msgr2=0x7efee81068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.967+0000 7efeeca82700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee80ffee0 0x7efee81068f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efed800e9a0 tx=0x7efed800ecb0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.968+0000 7efeeca82700 1 -- 172.21.15.39:0/642451823 shutdown_connections 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.968+0000 7efeeca82700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee80ff520 0x7efee81063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.968+0000 7efeeca82700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efed00a12d0 0x7efed00a3790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.968+0000 7efeeca82700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee81055e0 0x7efee8106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.968+0000 7efeeca82700 1 --2- 172.21.15.39:0/642451823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efee80ffee0 0x7efee81068f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.968+0000 7efeeca82700 1 -- 172.21.15.39:0/642451823 >> 172.21.15.39:0/642451823 conn(0x7efee80763a0 msgr2=0x7efee810c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.968+0000 7efeeca82700 1 -- 172.21.15.39:0/642451823 shutdown_connections 2024-05-24T11:50:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:21.968+0000 7efeeca82700 1 -- 172.21.15.39:0/642451823 wait complete. 2024-05-24T11:50:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:21 smithi039 ceph-mon[42919]: pgmap v382: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 1.46k op/s 2024-05-24T11:50:22.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:21 smithi053 ceph-mon[47448]: pgmap v382: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 1.46k op/s 2024-05-24T11:50:22.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:21 smithi137 ceph-mon[47907]: pgmap v382: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 1.46k op/s 2024-05-24T11:50:24.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:23 smithi039 ceph-mon[42919]: pgmap v383: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 1.46k op/s 2024-05-24T11:50:24.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 660.0 s, 762.2 tps, lat 130.875 ms stddev 534.821 2024-05-24T11:50:24.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:23 smithi053 ceph-mon[47448]: pgmap v383: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 1.46k op/s 2024-05-24T11:50:24.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:23 smithi137 ceph-mon[47907]: pgmap v383: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 17 MiB/s wr, 1.46k op/s 2024-05-24T11:50:26.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:25 smithi039 ceph-mon[42919]: pgmap v384: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 17 MiB/s wr, 1.46k op/s 2024-05-24T11:50:26.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:25 smithi053 ceph-mon[47448]: pgmap v384: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 17 MiB/s wr, 1.46k op/s 2024-05-24T11:50:26.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:25 smithi137 ceph-mon[47907]: pgmap v384: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 17 MiB/s wr, 1.46k op/s 2024-05-24T11:50:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:27 smithi039 ceph-mon[42919]: pgmap v385: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 14 MiB/s wr, 1.25k op/s 2024-05-24T11:50:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:27 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:50:28.231 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:27 smithi137 ceph-mon[47907]: pgmap v385: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 14 MiB/s wr, 1.25k op/s 2024-05-24T11:50:28.231 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:27 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:50:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:27 smithi053 ceph-mon[47448]: pgmap v385: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 14 MiB/s wr, 1.25k op/s 2024-05-24T11:50:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:27 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:50:29.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:28 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:50:29.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:28 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:50:29.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:28 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:50:29.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 665.0 s, 808.0 tps, lat 123.517 ms stddev 64.689 2024-05-24T11:50:29.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:28 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:50:29.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:28 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:50:29.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:28 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:50:29.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:28 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:50:29.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:28 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:50:29.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:28 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:50:30.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:29 smithi039 ceph-mon[42919]: pgmap v386: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 19 MiB/s wr, 1.69k op/s 2024-05-24T11:50:30.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:29 smithi053 ceph-mon[47448]: pgmap v386: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 19 MiB/s wr, 1.69k op/s 2024-05-24T11:50:30.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:29 smithi137 ceph-mon[47907]: pgmap v386: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 19 MiB/s wr, 1.69k op/s 2024-05-24T11:50:32.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:31 smithi039 ceph-mon[42919]: pgmap v387: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:50:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:31 smithi053 ceph-mon[47448]: pgmap v387: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:50:32.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:31 smithi137 ceph-mon[47907]: pgmap v387: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:50:34.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:33 smithi039 ceph-mon[42919]: pgmap v388: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-05-24T11:50:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 670.0 s, 807.6 tps, lat 122.205 ms stddev 73.976 2024-05-24T11:50:34.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:33 smithi053 ceph-mon[47448]: pgmap v388: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-05-24T11:50:34.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:33 smithi137 ceph-mon[47907]: pgmap v388: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-05-24T11:50:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.151+0000 7f7ba528c700 1 -- 172.21.15.39:0/380242252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104910 msgr2=0x7f7ba010c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.151+0000 7f7ba528c700 1 --2- 172.21.15.39:0/380242252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104910 0x7f7ba010c490 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800aad0 tx=0x7f7b9800ade0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.157+0000 7f7ba528c700 1 -- 172.21.15.39:0/380242252 shutdown_connections 2024-05-24T11:50:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.157+0000 7f7ba528c700 1 --2- 172.21.15.39:0/380242252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba0104910 0x7f7ba010c490 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.157+0000 7f7ba528c700 1 --2- 172.21.15.39:0/380242252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba00ffa50 0x7f7ba00ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.157+0000 7f7ba528c700 1 --2- 172.21.15.39:0/380242252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba0100400 0x7f7ba01041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.157+0000 7f7ba528c700 1 -- 172.21.15.39:0/380242252 >> 172.21.15.39:0/380242252 conn(0x7f7ba0075130 msgr2=0x7f7ba0075540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.157+0000 7f7ba528c700 1 -- 172.21.15.39:0/380242252 shutdown_connections 2024-05-24T11:50:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.157+0000 7f7ba528c700 1 -- 172.21.15.39:0/380242252 wait complete. 2024-05-24T11:50:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.158+0000 7f7ba528c700 1 Processor -- start 2024-05-24T11:50:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.159+0000 7f7ba528c700 1 -- start start 2024-05-24T11:50:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.159+0000 7f7ba528c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba00ffa50 0x7f7ba0195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.159+0000 7f7ba528c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba0104910 0x7f7ba0195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.159+0000 7f7ba528c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01998b0 0x7f7ba0199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.159+0000 7f7ba528c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba019a330 con 0x7f7ba01998b0 2024-05-24T11:50:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.159+0000 7f7ba528c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba019a470 con 0x7f7ba0104910 2024-05-24T11:50:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.159+0000 7f7ba528c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba019a5e0 con 0x7f7ba00ffa50 2024-05-24T11:50:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01998b0 0x7f7ba0199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01998b0 0x7f7ba0199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35260/0 (socket says 172.21.15.39:35260) 2024-05-24T11:50:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/3867603212 learned_addr learned my addr 172.21.15.39:0/3867603212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b967fc700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba0104910 0x7f7ba0195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba00ffa50 0x7f7ba0195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/3867603212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba00ffa50 msgr2=0x7f7ba0195020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba00ffa50 0x7f7ba0195020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/3867603212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba0104910 msgr2=0x7f7ba0195560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba0104910 0x7f7ba0195560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/3867603212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9800a700 con 0x7f7ba01998b0 2024-05-24T11:50:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.160+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01998b0 0x7f7ba0199d30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800afd0 tx=0x7f7b9800f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.161+0000 7f7b9cff9700 1 -- 172.21.15.39:0/3867603212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98003e80 con 0x7f7ba01998b0 2024-05-24T11:50:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.161+0000 7f7b9cff9700 1 -- 172.21.15.39:0/3867603212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b980050d0 con 0x7f7ba01998b0 2024-05-24T11:50:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.161+0000 7f7b9cff9700 1 -- 172.21.15.39:0/3867603212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9801d790 con 0x7f7ba01998b0 2024-05-24T11:50:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.161+0000 7f7ba528c700 1 -- 172.21.15.39:0/3867603212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba006a9c0 con 0x7f7ba01998b0 2024-05-24T11:50:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.161+0000 7f7ba528c700 1 -- 172.21.15.39:0/3867603212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba006adc0 con 0x7f7ba01998b0 2024-05-24T11:50:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.165+0000 7f7bb1447b80 1 Processor -- start 2024-05-24T11:50:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.166+0000 7f7bb1447b80 1 -- start start 2024-05-24T11:50:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.166+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 0x559de51ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.166+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.166+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51d88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.166+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de512f0f0 con 0x559de51d8480 2024-05-24T11:50:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.166+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de512cab0 con 0x559de51dd950 2024-05-24T11:50:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.166+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de51283a0 con 0x559de51de270 2024-05-24T11:50:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.167+0000 7f7b9effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.167+0000 7f7b9effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51d7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:55730/0 (socket says 172.21.15.39:55730) 2024-05-24T11:50:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.167+0000 7f7b9effd700 1 -- 172.21.15.39:0/1454547927 learned_addr learned my addr 172.21.15.39:0/1454547927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.167+0000 7f7b967fc700 1 --2- 172.21.15.39:0/1454547927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 0x559de51ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.167+0000 7f7b967fc700 1 --2- 172.21.15.39:0/1454547927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51d88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.168+0000 7f7b967fc700 1 -- 172.21.15.39:0/1454547927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 msgr2=0x559de51d7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.168+0000 7f7b967fc700 1 --2- 172.21.15.39:0/1454547927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51d7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.168+0000 7f7b967fc700 1 -- 172.21.15.39:0/1454547927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 msgr2=0x559de51d88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.168+0000 7f7b967fc700 1 --2- 172.21.15.39:0/1454547927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51d88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.168+0000 7f7b967fc700 1 -- 172.21.15.39:0/1454547927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559de4fa9350 con 0x559de51dd950 2024-05-24T11:50:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.168+0000 7f7b967fc700 1 --2- 172.21.15.39:0/1454547927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 0x559de51ddd30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0196520 tx=0x7f7b90009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71c641d957031252 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.168+0000 7f7b957fa700 1 -- 172.21.15.39:0/1454547927 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90019430 con 0x559de51dd950 2024-05-24T11:50:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.168+0000 7f7b957fa700 1 -- 172.21.15.39:0/1454547927 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b90018870 con 0x559de51dd950 2024-05-24T11:50:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.168+0000 7f7b957fa700 1 -- 172.21.15.39:0/1454547927 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90013670 con 0x559de51dd950 2024-05-24T11:50:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.169+0000 7f7bb1447b80 1 -- 172.21.15.39:0/1454547927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 msgr2=0x559de51ddd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.169+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/1454547927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 0x559de51ddd30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0196520 tx=0x7f7b90009290 comp rx=0 tx=0).stop 2024-05-24T11:50:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.170+0000 7f7b9cff9700 1 -- 172.21.15.39:0/3867603212 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7b98005500 con 0x7f7ba01998b0 2024-05-24T11:50:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.170+0000 7f7b9cff9700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c0a58b0 0x7f7b8c0a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.171+0000 7f7b9cff9700 1 -- 172.21.15.39:0/3867603212 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7b98012070 con 0x7f7ba01998b0 2024-05-24T11:50:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.171+0000 7f7bb1447b80 1 -- 172.21.15.39:0/1454547927 shutdown_connections 2024-05-24T11:50:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.171+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/1454547927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51d88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.171+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/1454547927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 0x559de51ddd30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.171+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/1454547927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.171+0000 7f7bb1447b80 1 -- 172.21.15.39:0/1454547927 >> 172.21.15.39:0/1454547927 conn(0x559de51361b0 msgr2=0x559de51d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.171+0000 7f7bb1447b80 1 -- 172.21.15.39:0/1454547927 shutdown_connections 2024-05-24T11:50:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.171+0000 7f7bb1447b80 1 -- 172.21.15.39:0/1454547927 wait complete. 2024-05-24T11:50:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.172+0000 7f7bb1447b80 1 -- start start 2024-05-24T11:50:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.172+0000 7f7bb1447b80 1 Processor -- start 2024-05-24T11:50:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.172+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.172+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51dd950 0x559de51e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.172+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51de270 0x559de51e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.172+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de50c4500 con 0x559de51d8480 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de50c41a0 con 0x559de51de270 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de4fab640 con 0x559de51dd950 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7bb1447b80 10 client.0 _ll_register_callbacks cb 0x559de50cdad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7b967fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51dd950 0x559de51e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7b967fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51dd950 0x559de51e1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:55742/0 (socket says 172.21.15.39:55742) 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7b967fc700 1 -- 172.21.15.39:0/2734492614 learned_addr learned my addr 172.21.15.39:0/2734492614 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7b9effd700 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7b967fc700 1 -- 172.21.15.39:0/2734492614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51de270 msgr2=0x559de51e1fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:50:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7b967fc700 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51de270 0x559de51e1fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7b967fc700 1 -- 172.21.15.39:0/2734492614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 msgr2=0x559de51e1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7b967fc700 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.173+0000 7f7b967fc700 1 -- 172.21.15.39:0/2734492614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b90011040 con 0x559de51dd950 2024-05-24T11:50:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.174+0000 7f7b967fc700 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51dd950 0x559de51e1aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7b90018610 tx=0x7f7b900090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b0a4209b0c2a2b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.174+0000 7f7b867fc700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:50:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.174+0000 7f7b867fc700 1 -- 172.21.15.39:0/2734492614 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90020070 con 0x559de51dd950 2024-05-24T11:50:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.174+0000 7f7b867fc700 1 -- 172.21.15.39:0/2734492614 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7b90018c70 con 0x559de51dd950 2024-05-24T11:50:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.174+0000 7f7bb1447b80 1 -- 172.21.15.39:0/2734492614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559de5081210 con 0x559de51dd950 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.175+0000 7f7b867fc700 1 -- 172.21.15.39:0/2734492614 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b90019430 con 0x559de51dd950 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.175+0000 7f7bb1447b80 10 client.25885 fetch_fsmap learned FSMap version 32 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.175+0000 7f7bb1447b80 1 -- 172.21.15.39:0/2734492614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559de4f55b00 con 0x559de51dd950 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.175+0000 7f7b867fc700 1 -- 172.21.15.39:0/2734492614 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559de5081210 con 0x559de51dd950 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.175+0000 7f7b867fc700 1 -- 172.21.15.39:0/2734492614 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b9002c070 con 0x559de51dd950 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7bb1447b80 10 client.25885 fetch_fsmap finished waiting for FSMap version 32 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7bb1447b80 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7bb1447b80 20 client.25885 populate_metadata read hostname 'smithi039' 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559de51f1b30 0x559de51f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7bb1447b80 4 client.25885 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:50:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7bb1447b80 1 -- 172.21.15.39:0/2734492614 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559de51a77b0 con 0x559de51f1b30 2024-05-24T11:50:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7b967fc700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c0a58b0 0x7f7b8c0a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7b9effd700 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559de51f1b30 0x559de51f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7b867fc700 10 client.25885 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7b9effd700 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559de51f1b30 0x559de51f3f50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38760ff1bc4a2f2b server_cookie=6e7b2808f517e72c in_seq=0 out_seq=0 2024-05-24T11:50:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.176+0000 7f7b967fc700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c0a58b0 0x7f7b8c0a7d70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b90017800 tx=0x7f7b90017fd0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.177+0000 7f7b867fc700 1 -- 172.21.15.39:0/2734492614 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559de51a77b0 con 0x559de51f1b30 2024-05-24T11:50:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.177+0000 7f7b867fc700 10 client.25885 handle_command_reply: tid=0 2024-05-24T11:50:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.177+0000 7f7bb1447b80 1 client.25885 shutdown 2024-05-24T11:50:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.177+0000 7f7bb1447b80 1 -- 172.21.15.39:0/2734492614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51dd950 msgr2=0x559de51e1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.177+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51dd950 0x559de51e1aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7b90018610 tx=0x7f7b900090e0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7bb1447b80 1 -- 172.21.15.39:0/2734492614 shutdown_connections 2024-05-24T11:50:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559de51f1b30 0x559de51f3f50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51de270 0x559de51e1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/2734492614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51dd950 0x559de51e1aa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7bb1447b80 1 -- 172.21.15.39:0/2734492614 >> 172.21.15.39:0/2734492614 conn(0x559de51361b0 msgr2=0x559de51d6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7b867fc700 0 client.25885 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7bb1447b80 1 -- 172.21.15.39:0/2734492614 shutdown_connections 2024-05-24T11:50:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7bb1447b80 1 -- 172.21.15.39:0/2734492614 wait complete. 2024-05-24T11:50:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.178+0000 7f7bb1447b80 20 client.25885 trim_cache size 0 max 16384 2024-05-24T11:50:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7bb1447b80 1 Processor -- start 2024-05-24T11:50:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7bb1447b80 1 -- start start 2024-05-24T11:50:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51d8480 0x559de51d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51dd950 0x559de51e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51e9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de5081210 con 0x559de51dd950 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de51cd330 con 0x559de51d8480 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de51dd5e0 con 0x559de51de270 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7b967fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51dd950 0x559de51e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.204+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51e9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:55746/0 (socket says 172.21.15.39:55746) 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.205+0000 7f7b9f7fe700 1 -- 172.21.15.39:0/4199169223 learned_addr learned my addr 172.21.15.39:0/4199169223 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.205+0000 7f7b9effd700 1 --2- 172.21.15.39:0/4199169223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51d8480 0x559de51d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.205+0000 7f7b9effd700 1 -- 172.21.15.39:0/4199169223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 msgr2=0x559de51e9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.205+0000 7f7b9effd700 1 --2- 172.21.15.39:0/4199169223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51e9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.205+0000 7f7b9effd700 1 -- 172.21.15.39:0/4199169223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51dd950 msgr2=0x559de51e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.205+0000 7f7b9effd700 1 --2- 172.21.15.39:0/4199169223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51dd950 0x559de51e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.205+0000 7f7b9effd700 1 -- 172.21.15.39:0/4199169223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559de50ee500 con 0x559de51d8480 2024-05-24T11:50:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.205+0000 7f7b9effd700 1 --2- 172.21.15.39:0/4199169223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51d8480 0x559de51d2910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x559de51f4800 tx=0x7f7b88014750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b534b40fd96303b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.205+0000 7f7b95ffb700 1 -- 172.21.15.39:0/4199169223 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88022070 con 0x559de51d8480 2024-05-24T11:50:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.206+0000 7f7b95ffb700 1 -- 172.21.15.39:0/4199169223 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b8801d8b0 con 0x559de51d8480 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.206+0000 7f7b95ffb700 1 -- 172.21.15.39:0/4199169223 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88018040 con 0x559de51d8480 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.206+0000 7f7bb1447b80 1 -- 172.21.15.39:0/4199169223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51d8480 msgr2=0x559de51d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.206+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/4199169223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51d8480 0x559de51d2910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x559de51f4800 tx=0x7f7b88014750 comp rx=0 tx=0).stop 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.207+0000 7f7bb1447b80 1 -- 172.21.15.39:0/4199169223 shutdown_connections 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.207+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/4199169223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de51e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.207+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/4199169223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51d8480 0x559de51d2910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.207+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/4199169223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51dd950 0x559de51e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.207+0000 7f7bb1447b80 1 -- 172.21.15.39:0/4199169223 >> 172.21.15.39:0/4199169223 conn(0x559de51361b0 msgr2=0x559de51d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.207+0000 7f7bb1447b80 1 -- 172.21.15.39:0/4199169223 shutdown_connections 2024-05-24T11:50:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.207+0000 7f7bb1447b80 1 -- 172.21.15.39:0/4199169223 wait complete. 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.207+0000 7f7bb1447b80 1 -- start start 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.207+0000 7f7bb1447b80 1 Processor -- start 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 0x559de524c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7bb1447b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de524ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de524f3e0 con 0x559de51d8480 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de524f550 con 0x559de51dd950 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7bb1447b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de524f710 con 0x559de51de270 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7bb1447b80 10 client.0 _ll_register_callbacks cb 0x559de50cdad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b9effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b9effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35288/0 (socket says 172.21.15.39:35288) 2024-05-24T11:50:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b9effd700 1 -- 172.21.15.39:0/3069970400 learned_addr learned my addr 172.21.15.39:0/3069970400 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b9f7fe700 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de524ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b967fc700 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 0x559de524c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b9effd700 1 -- 172.21.15.39:0/3069970400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 msgr2=0x559de524ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de524ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b9effd700 1 -- 172.21.15.39:0/3069970400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 msgr2=0x559de524c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 0x559de524c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.208+0000 7f7b9effd700 1 -- 172.21.15.39:0/3069970400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b88009d10 con 0x559de51d8480 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.209+0000 7f7b9effd700 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e8f80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7b8801da60 tx=0x7f7b880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82aaf331bfdef262 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.209+0000 7f7b867fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.209+0000 7f7b867fc700 1 -- 172.21.15.39:0/3069970400 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88018070 con 0x559de51d8480 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.209+0000 7f7b867fc700 1 -- 172.21.15.39:0/3069970400 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7b8801d420 con 0x559de51d8480 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.209+0000 7f7bb1447b80 1 -- 172.21.15.39:0/3069970400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b90018870 con 0x559de51d8480 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.209+0000 7f7b867fc700 1 -- 172.21.15.39:0/3069970400 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b88022040 con 0x559de51d8480 2024-05-24T11:50:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.209+0000 7f7bb1447b80 10 client.16236 fetch_fsmap learned FSMap version 32 2024-05-24T11:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.209+0000 7f7bb1447b80 1 -- 172.21.15.39:0/3069970400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559de4fadc70 con 0x559de51d8480 2024-05-24T11:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.209+0000 7f7b867fc700 1 -- 172.21.15.39:0/3069970400 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b90018870 con 0x559de51d8480 2024-05-24T11:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.210+0000 7f7b867fc700 1 -- 172.21.15.39:0/3069970400 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7b88026400 con 0x559de51d8480 2024-05-24T11:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.210+0000 7f7bb1447b80 10 client.16236 fetch_fsmap finished waiting for FSMap version 32 2024-05-24T11:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.210+0000 7f7bb1447b80 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.210+0000 7f7bb1447b80 20 client.16236 populate_metadata read hostname 'smithi039' 2024-05-24T11:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.210+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559de5250e90 0x559de52532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.210+0000 7f7bb1447b80 4 client.16236 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:50:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.210+0000 7f7bb1447b80 1 -- 172.21.15.39:0/3069970400 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559de5120e60 con 0x559de5250e90 2024-05-24T11:50:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.210+0000 7f7b967fc700 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559de5250e90 0x559de52532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.211+0000 7f7b967fc700 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559de5250e90 0x559de52532b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4efb6d2d7d8d2f44 server_cookie=e900130563c5113a in_seq=0 out_seq=0 2024-05-24T11:50:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.211+0000 7f7b867fc700 10 client.16236 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.748+0000 7f7b867fc700 1 -- 172.21.15.39:0/3069970400 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559de5120e60 con 0x559de5250e90 2024-05-24T11:50:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.748+0000 7f7b867fc700 10 client.16236 handle_command_reply: tid=0 2024-05-24T11:50:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.749+0000 7f7bb1447b80 1 client.16236 shutdown 2024-05-24T11:50:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 -- 172.21.15.39:0/3069970400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 msgr2=0x559de51e8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e8f80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7b8801da60 tx=0x7f7b880057f0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 -- 172.21.15.39:0/3069970400 shutdown_connections 2024-05-24T11:50:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559de51de270 0x559de524ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x559de5250e90 0x559de52532b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559de51d8480 0x559de51e8f80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 --2- 172.21.15.39:0/3069970400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559de51dd950 0x559de524c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 -- 172.21.15.39:0/3069970400 >> 172.21.15.39:0/3069970400 conn(0x559de51361b0 msgr2=0x559de51dfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7b867fc700 0 client.16236 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 -- 172.21.15.39:0/3069970400 shutdown_connections 2024-05-24T11:50:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 1 -- 172.21.15.39:0/3069970400 wait complete. 2024-05-24T11:50:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.750+0000 7f7bb1447b80 20 client.16236 trim_cache size 0 max 16384 2024-05-24T11:50:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:50:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:50:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:50:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:50:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 -- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c0a58b0 msgr2=0x7f7b8c0a7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c0a58b0 0x7f7b8c0a7d70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b90017800 tx=0x7f7b90017fd0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 -- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01998b0 msgr2=0x7f7ba0199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01998b0 0x7f7ba0199d30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800afd0 tx=0x7f7b9800f7b0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 -- 172.21.15.39:0/3867603212 shutdown_connections 2024-05-24T11:50:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ba00ffa50 0x7f7ba0195020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7b8c0a58b0 0x7f7b8c0a7d70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ba01998b0 0x7f7ba0199d30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 --2- 172.21.15.39:0/3867603212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ba0104910 0x7f7ba0195560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.751+0000 7f7ba528c700 1 -- 172.21.15.39:0/3867603212 >> 172.21.15.39:0/3867603212 conn(0x7f7ba0075130 msgr2=0x7f7ba00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.752+0000 7f7ba528c700 1 -- 172.21.15.39:0/3867603212 shutdown_connections 2024-05-24T11:50:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:35.752+0000 7f7ba528c700 1 -- 172.21.15.39:0/3867603212 wait complete. 2024-05-24T11:50:36.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:35 smithi039 ceph-mon[42919]: pgmap v389: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 20 MiB/s wr, 1.72k op/s 2024-05-24T11:50:36.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:35 smithi053 ceph-mon[47448]: pgmap v389: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 20 MiB/s wr, 1.72k op/s 2024-05-24T11:50:36.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:35 smithi137 ceph-mon[47907]: pgmap v389: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 20 MiB/s wr, 1.72k op/s 2024-05-24T11:50:38.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:37 smithi039 ceph-mon[42919]: pgmap v390: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2024-05-24T11:50:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:37 smithi053 ceph-mon[47448]: pgmap v390: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2024-05-24T11:50:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:37 smithi137 ceph-mon[47907]: pgmap v390: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2024-05-24T11:50:39.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 675.0 s, 815.0 tps, lat 123.422 ms stddev 70.718 2024-05-24T11:50:40.241 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:39 smithi053 ceph-mon[47448]: pgmap v391: 97 pgs: 97 active+clean; 5.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 23 MiB/s wr, 1.71k op/s 2024-05-24T11:50:40.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:39 smithi137 ceph-mon[47907]: pgmap v391: 97 pgs: 97 active+clean; 5.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 23 MiB/s wr, 1.71k op/s 2024-05-24T11:50:40.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:39 smithi039 ceph-mon[42919]: pgmap v391: 97 pgs: 97 active+clean; 5.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 23 MiB/s wr, 1.71k op/s 2024-05-24T11:50:42.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:41 smithi053 ceph-mon[47448]: pgmap v392: 97 pgs: 97 active+clean; 5.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-05-24T11:50:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:41 smithi137 ceph-mon[47907]: pgmap v392: 97 pgs: 97 active+clean; 5.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-05-24T11:50:42.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:41 smithi039 ceph-mon[42919]: pgmap v392: 97 pgs: 97 active+clean; 5.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-05-24T11:50:44.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 680.0 s, 831.6 tps, lat 121.591 ms stddev 59.970 2024-05-24T11:50:44.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:43 smithi053 ceph-mon[47448]: pgmap v393: 97 pgs: 97 active+clean; 5.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2024-05-24T11:50:44.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:43 smithi137 ceph-mon[47907]: pgmap v393: 97 pgs: 97 active+clean; 5.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2024-05-24T11:50:44.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:43 smithi039 ceph-mon[42919]: pgmap v393: 97 pgs: 97 active+clean; 5.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2024-05-24T11:50:46.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:45 smithi053 ceph-mon[47448]: pgmap v394: 97 pgs: 97 active+clean; 5.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2024-05-24T11:50:46.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:45 smithi137 ceph-mon[47907]: pgmap v394: 97 pgs: 97 active+clean; 5.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2024-05-24T11:50:46.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:45 smithi039 ceph-mon[42919]: pgmap v394: 97 pgs: 97 active+clean; 5.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2024-05-24T11:50:48.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:47 smithi053 ceph-mon[47448]: pgmap v395: 97 pgs: 97 active+clean; 5.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-05-24T11:50:48.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:47 smithi137 ceph-mon[47907]: pgmap v395: 97 pgs: 97 active+clean; 5.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-05-24T11:50:48.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:47 smithi039 ceph-mon[42919]: pgmap v395: 97 pgs: 97 active+clean; 5.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-05-24T11:50:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.070+0000 7fea5265d700 1 -- 172.21.15.39:0/796191864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea4c10d5c0 msgr2=0x7fea4c10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.070+0000 7fea5265d700 1 --2- 172.21.15.39:0/796191864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea4c10d5c0 0x7fea4c10d9a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009920 tx=0x7fea3c009c30 comp rx=0 tx=0).stop 2024-05-24T11:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.070+0000 7fea5265d700 1 -- 172.21.15.39:0/796191864 shutdown_connections 2024-05-24T11:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.070+0000 7fea5265d700 1 --2- 172.21.15.39:0/796191864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea4c107ec0 0x7fea4c114380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.070+0000 7fea5265d700 1 --2- 172.21.15.39:0/796191864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea4c10d5c0 0x7fea4c10d9a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.070+0000 7fea5265d700 1 --2- 172.21.15.39:0/796191864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea4c107500 0x7fea4c107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.070+0000 7fea5265d700 1 -- 172.21.15.39:0/796191864 >> 172.21.15.39:0/796191864 conn(0x7fea4c076dc0 msgr2=0x7fea4c105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.070+0000 7fea5265d700 1 -- 172.21.15.39:0/796191864 shutdown_connections 2024-05-24T11:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.070+0000 7fea5265d700 1 -- 172.21.15.39:0/796191864 wait complete. 2024-05-24T11:50:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.071+0000 7fea5265d700 1 Processor -- start 2024-05-24T11:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.071+0000 7fea5265d700 1 -- start start 2024-05-24T11:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea5265d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea4c107500 0x7fea4c081440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea5265d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea4c107ec0 0x7fea4c0819c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea5265d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea4c10d5c0 0x7fea4c07a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea5265d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c07ae10 con 0x7fea4c107500 2024-05-24T11:50:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea5265d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c07af80 con 0x7fea4c107ec0 2024-05-24T11:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea5265d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c07b140 con 0x7fea4c10d5c0 2024-05-24T11:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea50bfa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea4c10d5c0 0x7fea4c07a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea4b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea4c107ec0 0x7fea4c0819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea4b7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea4c107ec0 0x7fea4c0819c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:52594/0 (socket says 172.21.15.39:52594) 2024-05-24T11:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.072+0000 7fea4b7fe700 1 -- 172.21.15.39:0/3514833999 learned_addr learned my addr 172.21.15.39:0/3514833999 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:49.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.073+0000 7fea4bfff700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea4c107500 0x7fea4c081440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.073+0000 7fea4b7fe700 1 -- 172.21.15.39:0/3514833999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea4c10d5c0 msgr2=0x7fea4c07a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.073+0000 7fea4b7fe700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea4c10d5c0 0x7fea4c07a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.073+0000 7fea4b7fe700 1 -- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea4c107500 msgr2=0x7fea4c081440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.073+0000 7fea4b7fe700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea4c107500 0x7fea4c081440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.073+0000 7fea4b7fe700 1 -- 172.21.15.39:0/3514833999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea400095f0 con 0x7fea4c107ec0 2024-05-24T11:50:49.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.073+0000 7fea4b7fe700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea4c107ec0 0x7fea4c0819c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fea4000ec10 tx=0x7fea4000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:49.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.076+0000 7fea497fa700 1 -- 172.21.15.39:0/3514833999 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000cd40 con 0x7fea4c107ec0 2024-05-24T11:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.076+0000 7fea497fa700 1 -- 172.21.15.39:0/3514833999 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea400044b0 con 0x7fea4c107ec0 2024-05-24T11:50:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.076+0000 7fea5265d700 1 -- 172.21.15.39:0/3514833999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c009520 con 0x7fea4c107ec0 2024-05-24T11:50:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.076+0000 7fea497fa700 1 -- 172.21.15.39:0/3514833999 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40010430 con 0x7fea4c107ec0 2024-05-24T11:50:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.077+0000 7fea5265d700 1 -- 172.21.15.39:0/3514833999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c07f260 con 0x7fea4c107ec0 2024-05-24T11:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.083+0000 7fea497fa700 1 -- 172.21.15.39:0/3514833999 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fea40004020 con 0x7fea4c107ec0 2024-05-24T11:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.083+0000 7fea5e818b80 1 Processor -- start 2024-05-24T11:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.084+0000 7fea497fa700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea340a1460 0x7fea340a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.084+0000 7fea497fa700 1 -- 172.21.15.39:0/3514833999 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fea40014070 con 0x7fea4c107ec0 2024-05-24T11:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea5e818b80 1 -- start start 2024-05-24T11:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af2ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2b270 0x55fe0af24f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af25480 0x55fe0af258a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea5e818b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0ae7c0f0 con 0x55fe0af25480 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea5e818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0ae79ab0 con 0x55fe0af2b270 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea5e818b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0ae753a0 con 0x55fe0af2a950 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea50bfa700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea340a1460 0x7fea340a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af2ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af2ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:43842/0 (socket says 172.21.15.39:43842) 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.085+0000 7fea4bfff700 1 -- 172.21.15.39:0/1380460310 learned_addr learned my addr 172.21.15.39:0/1380460310 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.086+0000 7fea4bfff700 1 -- 172.21.15.39:0/1380460310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2b270 msgr2=0x55fe0af24f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.086+0000 7fea4bfff700 1 --2- 172.21.15.39:0/1380460310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2b270 0x55fe0af24f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.086+0000 7fea4bfff700 1 -- 172.21.15.39:0/1380460310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af25480 msgr2=0x55fe0af258a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.086+0000 7fea4bfff700 1 --2- 172.21.15.39:0/1380460310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af25480 0x55fe0af258a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.086+0000 7fea4bfff700 1 -- 172.21.15.39:0/1380460310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe0acf6350 con 0x55fe0af2a950 2024-05-24T11:50:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.086+0000 7fea50bfa700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea340a1460 0x7fea340a3920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea4400a450 tx=0x7fea440095a0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.086+0000 7fea4bfff700 1 --2- 172.21.15.39:0/1380460310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af2ad30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00bec0 tx=0x7fea3c00bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221fb892189eb681 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.086+0000 7fea327fc700 1 -- 172.21.15.39:0/1380460310 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c01c040 con 0x55fe0af2a950 2024-05-24T11:50:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.086+0000 7fea327fc700 1 -- 172.21.15.39:0/1380460310 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea3c00fe00 con 0x55fe0af2a950 2024-05-24T11:50:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.087+0000 7fea327fc700 1 -- 172.21.15.39:0/1380460310 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0052a0 con 0x55fe0af2a950 2024-05-24T11:50:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.088+0000 7fea5e818b80 1 -- 172.21.15.39:0/1380460310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 msgr2=0x55fe0af2ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.088+0000 7fea5e818b80 1 --2- 172.21.15.39:0/1380460310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af2ad30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00bec0 tx=0x7fea3c00bef0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.089+0000 7fea5e818b80 1 -- 172.21.15.39:0/1380460310 shutdown_connections 2024-05-24T11:50:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.089+0000 7fea5e818b80 1 --2- 172.21.15.39:0/1380460310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af2ad30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.089+0000 7fea5e818b80 1 --2- 172.21.15.39:0/1380460310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af25480 0x55fe0af258a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.089+0000 7fea5e818b80 1 --2- 172.21.15.39:0/1380460310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2b270 0x55fe0af24f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.089+0000 7fea5e818b80 1 -- 172.21.15.39:0/1380460310 >> 172.21.15.39:0/1380460310 conn(0x55fe0ae831b0 msgr2=0x55fe0af1f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.091+0000 7fea5e818b80 1 -- 172.21.15.39:0/1380460310 shutdown_connections 2024-05-24T11:50:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.091+0000 7fea5e818b80 1 -- 172.21.15.39:0/1380460310 wait complete. 2024-05-24T11:50:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.091+0000 7fea5e818b80 1 -- start start 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea5e818b80 1 Processor -- start 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af25480 0x55fe0af31290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2a950 0x55fe0af2e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af2b270 0x55fe0af2ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea5e818b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0ae11500 con 0x55fe0af2b270 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea5e818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0ae111a0 con 0x55fe0af2a950 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea5e818b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0acf8640 con 0x55fe0af25480 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea5e818b80 10 client.0 _ll_register_callbacks cb 0x55fe0ae1aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af25480 0x55fe0af31290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.092+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af25480 0x55fe0af31290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:43852/0 (socket says 172.21.15.39:43852) 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea4bfff700 1 -- 172.21.15.39:0/3873619858 learned_addr learned my addr 172.21.15.39:0/3873619858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea50bfa700 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af2b270 0x55fe0af2ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea4b7fe700 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2a950 0x55fe0af2e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea4bfff700 1 -- 172.21.15.39:0/3873619858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2a950 msgr2=0x55fe0af2e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea4bfff700 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2a950 0x55fe0af2e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea4bfff700 1 -- 172.21.15.39:0/3873619858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af2b270 msgr2=0x55fe0af2ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea4bfff700 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af2b270 0x55fe0af2ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea4bfff700 1 -- 172.21.15.39:0/3873619858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c114c80 con 0x55fe0af25480 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea4bfff700 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af25480 0x55fe0af31290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009840 tx=0x7fea3c00b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd9fac40a5502f8d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3873619858 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c01c070 con 0x55fe0af25480 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.093+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3873619858 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fea3c005040 con 0x55fe0af25480 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.094+0000 7fea5e818b80 1 -- 172.21.15.39:0/3873619858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe0adce210 con 0x55fe0af25480 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.095+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3873619858 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea3c00f870 con 0x55fe0af25480 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.095+0000 7fea5e818b80 10 client.25900 fetch_fsmap learned FSMap version 32 2024-05-24T11:50:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.095+0000 7fea5e818b80 1 -- 172.21.15.39:0/3873619858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fe0aca2b00 con 0x55fe0af25480 2024-05-24T11:50:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.095+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3873619858 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe0adce210 con 0x55fe0af25480 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3873619858 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fea3c009ca0 con 0x55fe0af25480 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea5e818b80 10 client.25900 fetch_fsmap finished waiting for FSMap version 32 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea5e818b80 10 client.25900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea5e818b80 20 client.25900 populate_metadata read hostname 'smithi039' 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0af606d0 0x55fe0af62af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea5e818b80 4 client.25900 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea5e818b80 1 -- 172.21.15.39:0/3873619858 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe0aef47b0 con 0x55fe0af606d0 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea4b7fe700 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0af606d0 0x55fe0af62af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea4b7fe700 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0af606d0 0x55fe0af62af0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d5cec2949b74955 server_cookie=7ac6ef5749fcbbfa in_seq=0 out_seq=0 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.096+0000 7fea2b7fe700 10 client.25900 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.097+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3873619858 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe0aef47b0 con 0x55fe0af606d0 2024-05-24T11:50:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.097+0000 7fea2b7fe700 10 client.25900 handle_command_reply: tid=0 2024-05-24T11:50:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.097+0000 7fea5e818b80 1 client.25900 shutdown 2024-05-24T11:50:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.098+0000 7fea5e818b80 1 -- 172.21.15.39:0/3873619858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af25480 msgr2=0x55fe0af31290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.098+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af25480 0x55fe0af31290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009840 tx=0x7fea3c00b7c0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea5e818b80 1 -- 172.21.15.39:0/3873619858 shutdown_connections 2024-05-24T11:50:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af25480 0x55fe0af31290 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0af606d0 0x55fe0af62af0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af2b270 0x55fe0af2ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3873619858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2a950 0x55fe0af2e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea5e818b80 1 -- 172.21.15.39:0/3873619858 >> 172.21.15.39:0/3873619858 conn(0x55fe0ae831b0 msgr2=0x55fe0af23f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea2b7fe700 0 client.25900 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea5e818b80 1 -- 172.21.15.39:0/3873619858 shutdown_connections 2024-05-24T11:50:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea5e818b80 1 -- 172.21.15.39:0/3873619858 wait complete. 2024-05-24T11:50:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.104+0000 7fea5e818b80 20 client.25900 trim_cache size 0 max 16384 2024-05-24T11:50:49.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 685.0 s, 829.8 tps, lat 117.833 ms stddev 83.977 2024-05-24T11:50:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea5e818b80 1 Processor -- start 2024-05-24T11:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea5e818b80 1 -- start start 2024-05-24T11:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af25480 0x55fe0af1f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af30dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2b270 0x55fe0af57c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea5e818b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0af1a7c0 con 0x55fe0af25480 2024-05-24T11:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea5e818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0af1c9b0 con 0x55fe0af2b270 2024-05-24T11:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea5e818b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0af2fb80 con 0x55fe0af2a950 2024-05-24T11:50:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea4b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af30dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea4b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af30dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:43862/0 (socket says 172.21.15.39:43862) 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea4b7fe700 1 -- 172.21.15.39:0/3860971582 learned_addr learned my addr 172.21.15.39:0/3860971582 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.123+0000 7fea4bfff700 1 --2- 172.21.15.39:0/3860971582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af25480 0x55fe0af1f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea50bfa700 1 --2- 172.21.15.39:0/3860971582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2b270 0x55fe0af57c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea4b7fe700 1 -- 172.21.15.39:0/3860971582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2b270 msgr2=0x55fe0af57c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea4b7fe700 1 --2- 172.21.15.39:0/3860971582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2b270 0x55fe0af57c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea4b7fe700 1 -- 172.21.15.39:0/3860971582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af25480 msgr2=0x55fe0af1f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea4b7fe700 1 --2- 172.21.15.39:0/3860971582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af25480 0x55fe0af1f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea4b7fe700 1 -- 172.21.15.39:0/3860971582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe0ae3b500 con 0x55fe0af2a950 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea4b7fe700 1 --2- 172.21.15.39:0/3860971582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af30dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea40096cf0 tx=0x7fea4000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de00c00b8e811bae server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea32ffd700 1 -- 172.21.15.39:0/3860971582 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400042d0 con 0x55fe0af2a950 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea32ffd700 1 -- 172.21.15.39:0/3860971582 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea40094e90 con 0x55fe0af2a950 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.124+0000 7fea32ffd700 1 -- 172.21.15.39:0/3860971582 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400a2790 con 0x55fe0af2a950 2024-05-24T11:50:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.125+0000 7fea5e818b80 1 -- 172.21.15.39:0/3860971582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 msgr2=0x55fe0af30dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.125+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3860971582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af30dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea40096cf0 tx=0x7fea4000ef90 comp rx=0 tx=0).stop 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.126+0000 7fea5e818b80 1 -- 172.21.15.39:0/3860971582 shutdown_connections 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.126+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3860971582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af2b270 0x55fe0af57c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.126+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3860971582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af25480 0x55fe0af1f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.126+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3860971582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2a950 0x55fe0af30dc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.126+0000 7fea5e818b80 1 -- 172.21.15.39:0/3860971582 >> 172.21.15.39:0/3860971582 conn(0x55fe0ae831b0 msgr2=0x55fe0af5c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.126+0000 7fea5e818b80 1 -- 172.21.15.39:0/3860971582 shutdown_connections 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.126+0000 7fea5e818b80 1 -- 172.21.15.39:0/3860971582 wait complete. 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.126+0000 7fea5e818b80 1 -- start start 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea5e818b80 1 Processor -- start 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af25480 0x55fe0af57f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af2a950 0x55fe0af55c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea5e818b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2b270 0x55fe0af56140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea5e818b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0af56810 con 0x55fe0af2a950 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea5e818b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0af56980 con 0x55fe0af25480 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea5e818b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0afb5bd0 con 0x55fe0af2b270 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea5e818b80 10 client.0 _ll_register_callbacks cb 0x55fe0ae1aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea50bfa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2b270 0x55fe0af56140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af25480 0x55fe0af57f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea50bfa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2b270 0x55fe0af56140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:43868/0 (socket says 172.21.15.39:43868) 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af25480 0x55fe0af57f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:52626/0 (socket says 172.21.15.39:52626) 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea50bfa700 1 -- 172.21.15.39:0/3488592864 learned_addr learned my addr 172.21.15.39:0/3488592864 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea50bfa700 1 -- 172.21.15.39:0/3488592864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af25480 msgr2=0x55fe0af57f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea50bfa700 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af25480 0x55fe0af57f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea50bfa700 1 -- 172.21.15.39:0/3488592864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af2a950 msgr2=0x55fe0af55c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea50bfa700 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af2a950 0x55fe0af55c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.127+0000 7fea50bfa700 1 -- 172.21.15.39:0/3488592864 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c07f260 con 0x55fe0af2b270 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.128+0000 7fea50bfa700 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2b270 0x55fe0af56140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea44018c40 tx=0x7fea44004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd75d788b15e8a32 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.128+0000 7fea2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.128+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3488592864 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44015bc0 con 0x55fe0af2b270 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.128+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3488592864 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fea44015d20 con 0x55fe0af2b270 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.128+0000 7fea5e818b80 1 -- 172.21.15.39:0/3488592864 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea3c00fe00 con 0x55fe0af2b270 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.128+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3488592864 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea44015ee0 con 0x55fe0af2b270 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.128+0000 7fea5e818b80 10 client.25906 fetch_fsmap learned FSMap version 32 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.128+0000 7fea5e818b80 1 -- 172.21.15.39:0/3488592864 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fe0acfac70 con 0x55fe0af2b270 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.129+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3488592864 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea3c00fe00 con 0x55fe0af2b270 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.129+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3488592864 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fea44022070 con 0x55fe0af2b270 2024-05-24T11:50:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.129+0000 7fea5e818b80 10 client.25906 fetch_fsmap finished waiting for FSMap version 32 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.129+0000 7fea5e818b80 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.129+0000 7fea5e818b80 20 client.25906 populate_metadata read hostname 'smithi039' 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.129+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0afb73a0 0x55fe0afb97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.129+0000 7fea5e818b80 4 client.25906 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.129+0000 7fea5e818b80 1 -- 172.21.15.39:0/3488592864 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe0ae6de60 con 0x55fe0afb73a0 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.129+0000 7fea4bfff700 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0afb73a0 0x55fe0afb97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.130+0000 7fea2b7fe700 10 client.25906 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.130+0000 7fea4bfff700 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0afb73a0 0x55fe0afb97c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=603e15e9172a3b51 server_cookie=f18cbcfc5767a065 in_seq=0 out_seq=0 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.174+0000 7fea2b7fe700 1 -- 172.21.15.39:0/3488592864 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe0ae6de60 con 0x55fe0afb73a0 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.174+0000 7fea2b7fe700 10 client.25906 handle_command_reply: tid=0 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.174+0000 7fea5e818b80 1 client.25906 shutdown 2024-05-24T11:50:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 -- 172.21.15.39:0/3488592864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2b270 msgr2=0x55fe0af56140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2b270 0x55fe0af56140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea44018c40 tx=0x7fea44004af0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 -- 172.21.15.39:0/3488592864 shutdown_connections 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0af2b270 0x55fe0af56140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55fe0afb73a0 0x55fe0afb97c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe0af25480 0x55fe0af57f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 --2- 172.21.15.39:0/3488592864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fe0af2a950 0x55fe0af55c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 -- 172.21.15.39:0/3488592864 >> 172.21.15.39:0/3488592864 conn(0x55fe0ae831b0 msgr2=0x55fe0af28380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea2b7fe700 0 client.25906 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 -- 172.21.15.39:0/3488592864 shutdown_connections 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 1 -- 172.21.15.39:0/3488592864 wait complete. 2024-05-24T11:50:49.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.176+0000 7fea5e818b80 20 client.25906 trim_cache size 0 max 16384 2024-05-24T11:50:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:50:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:50:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:50:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:50:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 -- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea340a1460 msgr2=0x7fea340a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea340a1460 0x7fea340a3920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea4400a450 tx=0x7fea440095a0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 -- 172.21.15.39:0/3514833999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea4c107ec0 msgr2=0x7fea4c0819c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea4c107ec0 0x7fea4c0819c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fea4000ec10 tx=0x7fea4000ef20 comp rx=0 tx=0).stop 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 -- 172.21.15.39:0/3514833999 shutdown_connections 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea4c10d5c0 0x7fea4c07a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fea340a1460 0x7fea340a3920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea4c107500 0x7fea4c081440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 --2- 172.21.15.39:0/3514833999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea4c107ec0 0x7fea4c0819c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 -- 172.21.15.39:0/3514833999 >> 172.21.15.39:0/3514833999 conn(0x7fea4c076dc0 msgr2=0x7fea4c0780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.177+0000 7fea5265d700 1 -- 172.21.15.39:0/3514833999 shutdown_connections 2024-05-24T11:50:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:50:49.178+0000 7fea5265d700 1 -- 172.21.15.39:0/3514833999 wait complete. 2024-05-24T11:50:50.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:49 smithi053 ceph-mon[47448]: pgmap v396: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 28 MiB/s wr, 1.82k op/s 2024-05-24T11:50:50.244 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:50:50 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setuser ceph since I am not root 2024-05-24T11:50:50.244 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:50:50 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setgroup ceph since I am not root 2024-05-24T11:50:50.245 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:50:50 smithi137 ceph-mds[84908]: -- 172.21.15.137:0/2580812443 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b034641a0 con 0x561b0343ac00 2024-05-24T11:50:50.245 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:50:50 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: starting mds.f at 2024-05-24T11:50:50.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:49 smithi137 ceph-mon[47907]: pgmap v396: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 28 MiB/s wr, 1.82k op/s 2024-05-24T11:50:50.366 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:49 smithi039 ceph-mon[42919]: pgmap v396: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 28 MiB/s wr, 1.82k op/s 2024-05-24T11:50:51.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:50 smithi053 ceph-mon[47448]: daemon mds.f restarted 2024-05-24T11:50:51.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:50 smithi053 ceph-mon[47448]: osdmap e96: 12 total, 12 up, 12 in 2024-05-24T11:50:51.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:50 smithi137 ceph-mon[47907]: daemon mds.f restarted 2024-05-24T11:50:51.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:50 smithi137 ceph-mon[47907]: osdmap e96: 12 total, 12 up, 12 in 2024-05-24T11:50:51.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:50 smithi039 ceph-mon[42919]: daemon mds.f restarted 2024-05-24T11:50:51.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:50 smithi039 ceph-mon[42919]: osdmap e96: 12 total, 12 up, 12 in 2024-05-24T11:50:52.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:51 smithi053 ceph-mon[47448]: pgmap v398: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 22 MiB/s wr, 1.47k op/s 2024-05-24T11:50:52.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:51 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6834/2017542238,v1:172.21.15.137:6835/2017542238] up:boot 2024-05-24T11:50:52.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:51 smithi053 ceph-mon[47448]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:50:52.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:51 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:50:52.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:51 smithi053 ceph-mon[47448]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:50:52.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:51 smithi137 ceph-mon[47907]: pgmap v398: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 22 MiB/s wr, 1.47k op/s 2024-05-24T11:50:52.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:51 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6834/2017542238,v1:172.21.15.137:6835/2017542238] up:boot 2024-05-24T11:50:52.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:51 smithi137 ceph-mon[47907]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:50:52.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:51 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:50:52.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:51 smithi137 ceph-mon[47907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:50:52.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:51 smithi039 ceph-mon[42919]: pgmap v398: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 22 MiB/s wr, 1.47k op/s 2024-05-24T11:50:52.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:51 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6834/2017542238,v1:172.21.15.137:6835/2017542238] up:boot 2024-05-24T11:50:52.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:51 smithi039 ceph-mon[42919]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:50:52.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:51 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:50:52.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:51 smithi039 ceph-mon[42919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:50:54.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 690.0 s, 811.2 tps, lat 125.678 ms stddev 87.635 2024-05-24T11:50:54.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:53 smithi053 ceph-mon[47448]: pgmap v399: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 22 MiB/s wr, 1.47k op/s 2024-05-24T11:50:54.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:53 smithi137 ceph-mon[47907]: pgmap v399: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 22 MiB/s wr, 1.47k op/s 2024-05-24T11:50:54.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:53 smithi039 ceph-mon[42919]: pgmap v399: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 22 MiB/s wr, 1.47k op/s 2024-05-24T11:50:56.242 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:55 smithi053 ceph-mon[47448]: pgmap v400: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-05-24T11:50:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:55 smithi137 ceph-mon[47907]: pgmap v400: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-05-24T11:50:56.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:55 smithi039 ceph-mon[42919]: pgmap v400: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-05-24T11:50:58.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:57 smithi053 ceph-mon[47448]: pgmap v401: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-05-24T11:50:58.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:57 smithi137 ceph-mon[47907]: pgmap v401: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-05-24T11:50:58.366 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:57 smithi039 ceph-mon[42919]: pgmap v401: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-05-24T11:50:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 695.0 s, 762.4 tps, lat 131.510 ms stddev 155.866 2024-05-24T11:51:00.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:50:59 smithi053 ceph-mon[47448]: pgmap v402: 97 pgs: 97 active+clean; 6.1 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 20 MiB/s wr, 1.44k op/s 2024-05-24T11:51:00.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:50:59 smithi137 ceph-mon[47907]: pgmap v402: 97 pgs: 97 active+clean; 6.1 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 20 MiB/s wr, 1.44k op/s 2024-05-24T11:51:00.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:50:59 smithi039 ceph-mon[42919]: pgmap v402: 97 pgs: 97 active+clean; 6.1 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 20 MiB/s wr, 1.44k op/s 2024-05-24T11:51:02.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:01 smithi053 ceph-mon[47448]: pgmap v403: 97 pgs: 97 active+clean; 6.1 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-05-24T11:51:02.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:01 smithi137 ceph-mon[47907]: pgmap v403: 97 pgs: 97 active+clean; 6.1 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-05-24T11:51:02.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:01 smithi039 ceph-mon[42919]: pgmap v403: 97 pgs: 97 active+clean; 6.1 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 19 MiB/s wr, 1.40k op/s 2024-05-24T11:51:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.716+0000 7fa6ccd41700 1 -- 172.21.15.39:0/4185925225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6c81002a0 msgr2=0x7fa6c8100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.716+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/4185925225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6c81002a0 0x7fa6c8100720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009990 tx=0x7fa6b8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.716+0000 7fa6ccd41700 1 -- 172.21.15.39:0/4185925225 shutdown_connections 2024-05-24T11:51:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.716+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/4185925225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6c8100c60 0x7fa6c8069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.716+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/4185925225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6c8105f40 0x7fa6c8106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.716+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/4185925225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6c81002a0 0x7fa6c8100720 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009990 tx=0x7fa6b8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.716+0000 7fa6ccd41700 1 -- 172.21.15.39:0/4185925225 >> 172.21.15.39:0/4185925225 conn(0x7fa6c8076c10 msgr2=0x7fa6c8077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.722+0000 7fa6ccd41700 1 -- 172.21.15.39:0/4185925225 shutdown_connections 2024-05-24T11:51:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.722+0000 7fa6ccd41700 1 -- 172.21.15.39:0/4185925225 wait complete. 2024-05-24T11:51:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.723+0000 7fa6ccd41700 1 Processor -- start 2024-05-24T11:51:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.723+0000 7fa6ccd41700 1 -- start start 2024-05-24T11:51:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6ccd41700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6c8100c60 0x7fa6c8102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6ccd41700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6c8105f40 0x7fa6c8103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6ccd41700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6c811a240 0x7fa6c811a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6ccd41700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c811ac00 con 0x7fa6c8100c60 2024-05-24T11:51:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6ccd41700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c811ad70 con 0x7fa6c8105f40 2024-05-24T11:51:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6ccd41700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c811af30 con 0x7fa6c811a240 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c6d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6c811a240 0x7fa6c811a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c6d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6c811a240 0x7fa6c811a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:43358/0 (socket says 172.21.15.39:43358) 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/639994514 learned_addr learned my addr 172.21.15.39:0/639994514 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6c8105f40 0x7fa6c8103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c659c700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6c8100c60 0x7fa6c8102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/639994514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6c8105f40 msgr2=0x7fa6c8103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6c8105f40 0x7fa6c8103360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6c8100c60 msgr2=0x7fa6c8102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6c8100c60 0x7fa6c8102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.724+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/639994514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b80095f0 con 0x7fa6c811a240 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.725+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6c811a240 0x7fa6c811a6c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00b4a0 tx=0x7fa6bc00b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.725+0000 7fa6b77fe700 1 -- 172.21.15.39:0/639994514 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc011830 con 0x7fa6c811a240 2024-05-24T11:51:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.725+0000 7fa6ccd41700 1 -- 172.21.15.39:0/639994514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c811b210 con 0x7fa6c811a240 2024-05-24T11:51:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.725+0000 7fa6ccd41700 1 -- 172.21.15.39:0/639994514 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c811b6f0 con 0x7fa6c811a240 2024-05-24T11:51:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.726+0000 7fa6b77fe700 1 -- 172.21.15.39:0/639994514 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6bc011e20 con 0x7fa6c811a240 2024-05-24T11:51:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.726+0000 7fa6b77fe700 1 -- 172.21.15.39:0/639994514 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc010de0 con 0x7fa6c811a240 2024-05-24T11:51:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.729+0000 7fa6b77fe700 1 -- 172.21.15.39:0/639994514 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa6bc01a400 con 0x7fa6c811a240 2024-05-24T11:51:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.730+0000 7fa6b77fe700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fa6b00a1310 0x7fa6b00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.730+0000 7fa6b77fe700 1 -- 172.21.15.39:0/639994514 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa6bc0cd460 con 0x7fa6c811a240 2024-05-24T11:51:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.730+0000 7fa6c659c700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fa6b00a1310 0x7fa6b00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.793+0000 7fa6d8efcb80 1 Processor -- start 2024-05-24T11:51:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.796+0000 7fa6c659c700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fa6b00a1310 0x7fa6b00a37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000e270 tx=0x7fa6c0009040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6d8efcb80 1 -- start start 2024-05-24T11:51:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec520950 0x55a7ec520d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec521270 0x55a7ec51af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec51b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec4720f0 con 0x55a7ec51b480 2024-05-24T11:51:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec46fab0 con 0x55a7ec520950 2024-05-24T11:51:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec46b3a0 con 0x55a7ec521270 2024-05-24T11:51:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6c6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec51b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6c6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec51b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44566/0 (socket says 172.21.15.39:44566) 2024-05-24T11:51:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/1533693651 learned_addr learned my addr 172.21.15.39:0/1533693651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/1533693651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec521270 0x55a7ec51af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/1533693651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec521270 msgr2=0x55a7ec51af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/1533693651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec521270 0x55a7ec51af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/1533693651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec520950 msgr2=0x55a7ec520d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:51:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/1533693651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec520950 0x55a7ec520d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.797+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/1533693651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7ec2ec350 con 0x55a7ec51b480 2024-05-24T11:51:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.798+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/1533693651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec521270 0x55a7ec51af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:51:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.798+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/1533693651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec51b8a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00bfd0 tx=0x7fa6bc09a020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e8fde5f149e1c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.798+0000 7fa6b4ff9700 1 -- 172.21.15.39:0/1533693651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc09e4f0 con 0x55a7ec51b480 2024-05-24T11:51:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.798+0000 7fa6b4ff9700 1 -- 172.21.15.39:0/1533693651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6bc0a2b40 con 0x55a7ec51b480 2024-05-24T11:51:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.798+0000 7fa6b4ff9700 1 -- 172.21.15.39:0/1533693651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc018570 con 0x55a7ec51b480 2024-05-24T11:51:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.799+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1533693651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 msgr2=0x55a7ec51b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.799+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1533693651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec51b8a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00bfd0 tx=0x7fa6bc09a020 comp rx=0 tx=0).stop 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.799+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1533693651 shutdown_connections 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.799+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1533693651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec51b8a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.799+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1533693651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec520950 0x55a7ec520d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.799+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1533693651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec521270 0x55a7ec51af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.799+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1533693651 >> 172.21.15.39:0/1533693651 conn(0x55a7ec4791b0 msgr2=0x55a7ec515840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.800+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1533693651 shutdown_connections 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.800+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1533693651 wait complete. 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.800+0000 7fa6d8efcb80 1 -- start start 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.800+0000 7fa6d8efcb80 1 Processor -- start 2024-05-24T11:51:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec51b480 0x55a7ec524770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec524cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec530cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec407500 con 0x55a7ec521270 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec4071a0 con 0x55a7ec51b480 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec2ee640 con 0x55a7ec5251f0 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6d8efcb80 10 client.0 _ll_register_callbacks cb 0x55a7ec410ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c6d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec530cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c6d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec530cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:43374/0 (socket says 172.21.15.39:43374) 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/97744616 learned_addr learned my addr 172.21.15.39:0/97744616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c659c700 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec51b480 0x55a7ec524770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec524cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/97744616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec51b480 msgr2=0x55a7ec524770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec51b480 0x55a7ec524770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/97744616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 msgr2=0x55a7ec524cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec524cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/97744616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c8069bc0 con 0x55a7ec5251f0 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.801+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec524cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.802+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec530cc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0cadb0 tx=0x7fa6bc010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f5e4a18b141d08 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.802+0000 7fa6adffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:51:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.802+0000 7fa6adffb700 1 -- 172.21.15.39:0/97744616 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc09e330 con 0x55a7ec5251f0 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.802+0000 7fa6adffb700 1 -- 172.21.15.39:0/97744616 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa6bc018510 con 0x55a7ec5251f0 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.802+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/97744616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7ec3c4210 con 0x55a7ec5251f0 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.802+0000 7fa6adffb700 1 -- 172.21.15.39:0/97744616 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6bc09e4f0 con 0x55a7ec5251f0 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.802+0000 7fa6d8efcb80 10 client.25921 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.802+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/97744616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fa6c811b6f0 con 0x55a7ec5251f0 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.802+0000 7fa6adffb700 1 -- 172.21.15.39:0/97744616 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a7ec3c4210 con 0x55a7ec5251f0 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.803+0000 7fa6adffb700 1 -- 172.21.15.39:0/97744616 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa6bc0850f0 con 0x55a7ec5251f0 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.803+0000 7fa6d8efcb80 10 client.25921 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.803+0000 7fa6d8efcb80 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.803+0000 7fa6d8efcb80 20 client.25921 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.803+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a7ec545ce0 0x55a7ec5480d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.803+0000 7fa6d8efcb80 4 client.25921 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:51:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.803+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/97744616 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7ec4ea7b0 con 0x55a7ec545ce0 2024-05-24T11:51:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.804+0000 7fa6c659c700 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a7ec545ce0 0x55a7ec5480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.804+0000 7fa6c659c700 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a7ec545ce0 0x55a7ec5480d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbd47406a2f68ef server_cookie=223bb9d8f55e287e in_seq=0 out_seq=0 2024-05-24T11:51:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.804+0000 7fa6adffb700 10 client.25921 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.814+0000 7fa6adffb700 1 -- 172.21.15.39:0/97744616 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a7ec4ea7b0 con 0x55a7ec545ce0 2024-05-24T11:51:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.814+0000 7fa6adffb700 10 client.25921 handle_command_reply: tid=0 2024-05-24T11:51:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.814+0000 7fa6d8efcb80 1 client.25921 shutdown 2024-05-24T11:51:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.822+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/97744616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 msgr2=0x55a7ec530cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.822+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec530cc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0cadb0 tx=0x7fa6bc010f60 comp rx=0 tx=0).stop 2024-05-24T11:51:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.822+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/97744616 shutdown_connections 2024-05-24T11:51:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.822+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec530cc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.822+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a7ec545ce0 0x55a7ec5480d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.822+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec51b480 0x55a7ec524770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.822+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/97744616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec524cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.822+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/97744616 >> 172.21.15.39:0/97744616 conn(0x55a7ec4791b0 msgr2=0x55a7ec51f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.826+0000 7fa6adffb700 0 client.25921 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.829+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/97744616 shutdown_connections 2024-05-24T11:51:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.830+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/97744616 wait complete. 2024-05-24T11:51:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.830+0000 7fa6d8efcb80 20 client.25921 trim_cache size 0 max 16384 2024-05-24T11:51:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.854+0000 7fa6d8efcb80 1 Processor -- start 2024-05-24T11:51:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.854+0000 7fa6d8efcb80 1 -- start start 2024-05-24T11:51:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec51b480 0x55a7ec518f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec515910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec5251f0 0x55a7ec524050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec524590 con 0x55a7ec521270 2024-05-24T11:51:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec527920 con 0x55a7ec5251f0 2024-05-24T11:51:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec524fc0 con 0x55a7ec51b480 2024-05-24T11:51:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6c5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec515910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6c5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec515910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44588/0 (socket says 172.21.15.39:44588) 2024-05-24T11:51:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6c5d9b700 1 -- 172.21.15.39:0/2668550404 learned_addr learned my addr 172.21.15.39:0/2668550404 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.855+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/2668550404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec5251f0 0x55a7ec524050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.856+0000 7fa6c659c700 1 --2- 172.21.15.39:0/2668550404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec51b480 0x55a7ec518f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.856+0000 7fa6c5d9b700 1 -- 172.21.15.39:0/2668550404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec51b480 msgr2=0x55a7ec518f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.856+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/2668550404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec51b480 0x55a7ec518f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.856+0000 7fa6c5d9b700 1 -- 172.21.15.39:0/2668550404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec5251f0 msgr2=0x55a7ec524050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.856+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/2668550404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec5251f0 0x55a7ec524050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.856+0000 7fa6c5d9b700 1 -- 172.21.15.39:0/2668550404 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7ec298b00 con 0x55a7ec521270 2024-05-24T11:51:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.856+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/2668550404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec515910 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80098e0 tx=0x7fa6b80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b0134fd2caa0cc server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.857+0000 7fa6b57fa700 1 -- 172.21.15.39:0/2668550404 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b801d040 con 0x55a7ec521270 2024-05-24T11:51:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.857+0000 7fa6b57fa700 1 -- 172.21.15.39:0/2668550404 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6b801e040 con 0x55a7ec521270 2024-05-24T11:51:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.857+0000 7fa6b57fa700 1 -- 172.21.15.39:0/2668550404 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8004b10 con 0x55a7ec521270 2024-05-24T11:51:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/2668550404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 msgr2=0x55a7ec515910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/2668550404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec515910 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80098e0 tx=0x7fa6b80057b0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/2668550404 shutdown_connections 2024-05-24T11:51:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/2668550404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec51b480 0x55a7ec518f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/2668550404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec5251f0 0x55a7ec524050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/2668550404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec521270 0x55a7ec515910 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/2668550404 >> 172.21.15.39:0/2668550404 conn(0x55a7ec4791b0 msgr2=0x55a7ec519770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/2668550404 shutdown_connections 2024-05-24T11:51:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/2668550404 wait complete. 2024-05-24T11:51:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 -- start start 2024-05-24T11:51:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.858+0000 7fa6d8efcb80 1 Processor -- start 2024-05-24T11:51:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec52c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec521270 0x55a7ec527be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6d8efcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec528320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec528a30 con 0x55a7ec51b480 2024-05-24T11:51:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec59b130 con 0x55a7ec521270 2024-05-24T11:51:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6d8efcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ec59b270 con 0x55a7ec5251f0 2024-05-24T11:51:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6d8efcb80 10 client.0 _ll_register_callbacks cb 0x55a7ec410ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6c659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec52c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6c659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec52c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44594/0 (socket says 172.21.15.39:44594) 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.859+0000 7fa6c659c700 1 -- 172.21.15.39:0/1590946556 learned_addr learned my addr 172.21.15.39:0/1590946556 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec528320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec521270 0x55a7ec527be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/1590946556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec521270 msgr2=0x55a7ec527be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec521270 0x55a7ec527be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/1590946556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 msgr2=0x55a7ec52c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec52c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6c6d9d700 1 -- 172.21.15.39:0/1590946556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b8005570 con 0x55a7ec5251f0 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6c5d9b700 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec521270 0x55a7ec527be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6c6d9d700 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec528320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc09b960 tx=0x7fa6bc099de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df5948ed4038550 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.864+0000 7fa6adffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6adffb700 1 -- 172.21.15.39:0/1590946556 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc09e330 con 0x55a7ec5251f0 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6adffb700 1 -- 172.21.15.39:0/1590946556 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fa6bc098b20 con 0x55a7ec5251f0 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1590946556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6bc0a2b40 con 0x55a7ec5251f0 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6adffb700 1 -- 172.21.15.39:0/1590946556 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6bc09e4f0 con 0x55a7ec5251f0 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6d8efcb80 10 client.25924 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1590946556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a7ec431500 con 0x55a7ec5251f0 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6adffb700 1 -- 172.21.15.39:0/1590946556 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6bc0a2b40 con 0x55a7ec5251f0 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6adffb700 1 -- 172.21.15.39:0/1590946556 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa6bc09a920 con 0x55a7ec5251f0 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6d8efcb80 10 client.25924 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6d8efcb80 10 client.25924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6d8efcb80 20 client.25924 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a7ec59c5e0 0x55a7ec59ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6d8efcb80 4 client.25924 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:51:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.865+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1590946556 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a7ec463e60 con 0x55a7ec59c5e0 2024-05-24T11:51:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.866+0000 7fa6c659c700 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a7ec59c5e0 0x55a7ec59ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.866+0000 7fa6adffb700 10 client.25924 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.866+0000 7fa6c659c700 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a7ec59c5e0 0x55a7ec59ea00 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57e858e34ed4e0d server_cookie=ff0d619812d31757 in_seq=0 out_seq=0 2024-05-24T11:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.977+0000 7fa6adffb700 1 -- 172.21.15.39:0/1590946556 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a7ec463e60 con 0x55a7ec59c5e0 2024-05-24T11:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.977+0000 7fa6adffb700 10 client.25924 handle_command_reply: tid=0 2024-05-24T11:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.977+0000 7fa6d8efcb80 1 client.25924 shutdown 2024-05-24T11:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.978+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1590946556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 msgr2=0x55a7ec528320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.978+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec528320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc09b960 tx=0x7fa6bc099de0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.986+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1590946556 shutdown_connections 2024-05-24T11:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.986+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7ec5251f0 0x55a7ec528320 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.986+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55a7ec59c5e0 0x55a7ec59ea00 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.986+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a7ec51b480 0x55a7ec52c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.986+0000 7fa6d8efcb80 1 --2- 172.21.15.39:0/1590946556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7ec521270 0x55a7ec527be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.986+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1590946556 >> 172.21.15.39:0/1590946556 conn(0x55a7ec4791b0 msgr2=0x55a7ec5306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.986+0000 7fa6adffb700 0 client.25924 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.991+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1590946556 shutdown_connections 2024-05-24T11:51:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.992+0000 7fa6d8efcb80 1 -- 172.21.15.39:0/1590946556 wait complete. 2024-05-24T11:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.992+0000 7fa6d8efcb80 20 client.25924 trim_cache size 0 max 16384 2024-05-24T11:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:51:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:51:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:51:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.992+0000 7fa6ccd41700 1 -- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fa6b00a1310 msgr2=0x7fa6b00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.992+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fa6b00a1310 0x7fa6b00a37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000e270 tx=0x7fa6c0009040 comp rx=0 tx=0).stop 2024-05-24T11:51:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.992+0000 7fa6ccd41700 1 -- 172.21.15.39:0/639994514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6c811a240 msgr2=0x7fa6c811a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.992+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6c811a240 0x7fa6c811a6c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00b4a0 tx=0x7fa6bc00b7b0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.993+0000 7fa6ccd41700 1 -- 172.21.15.39:0/639994514 shutdown_connections 2024-05-24T11:51:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.993+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6c811a240 0x7fa6c811a6c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.993+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fa6b00a1310 0x7fa6b00a37d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.993+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6c8100c60 0x7fa6c8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.993+0000 7fa6ccd41700 1 --2- 172.21.15.39:0/639994514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6c8105f40 0x7fa6c8103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.993+0000 7fa6ccd41700 1 -- 172.21.15.39:0/639994514 >> 172.21.15.39:0/639994514 conn(0x7fa6c8076c10 msgr2=0x7fa6c80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.993+0000 7fa6ccd41700 1 -- 172.21.15.39:0/639994514 shutdown_connections 2024-05-24T11:51:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:02.993+0000 7fa6ccd41700 1 -- 172.21.15.39:0/639994514 wait complete. 2024-05-24T11:51:04.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 700.0 s, 692.6 tps, lat 144.055 ms stddev 109.843 2024-05-24T11:51:04.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:03 smithi053 ceph-mon[47448]: pgmap v404: 97 pgs: 97 active+clean; 6.1 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-05-24T11:51:04.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:03 smithi053 ceph-mon[47448]: osdmap e97: 12 total, 12 up, 12 in 2024-05-24T11:51:04.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:03 smithi137 ceph-mon[47907]: pgmap v404: 97 pgs: 97 active+clean; 6.1 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-05-24T11:51:04.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:03 smithi137 ceph-mon[47907]: osdmap e97: 12 total, 12 up, 12 in 2024-05-24T11:51:04.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:03 smithi039 ceph-mon[42919]: pgmap v404: 97 pgs: 97 active+clean; 6.1 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-05-24T11:51:04.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:03 smithi039 ceph-mon[42919]: osdmap e97: 12 total, 12 up, 12 in 2024-05-24T11:51:06.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:05 smithi053 ceph-mon[47448]: pgmap v406: 97 pgs: 15 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 55 active+clean; 5.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2024-05-24T11:51:06.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:05 smithi137 ceph-mon[47907]: pgmap v406: 97 pgs: 15 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 55 active+clean; 5.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2024-05-24T11:51:06.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:05 smithi039 ceph-mon[42919]: pgmap v406: 97 pgs: 15 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 55 active+clean; 5.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2024-05-24T11:51:08.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:07 smithi053 ceph-mon[47448]: pgmap v407: 97 pgs: 15 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 55 active+clean; 5.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2024-05-24T11:51:08.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:07 smithi137 ceph-mon[47907]: pgmap v407: 97 pgs: 15 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 55 active+clean; 5.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2024-05-24T11:51:08.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:07 smithi039 ceph-mon[42919]: pgmap v407: 97 pgs: 15 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 55 active+clean; 5.5 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2024-05-24T11:51:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 705.0 s, 793.6 tps, lat 125.119 ms stddev 59.723 2024-05-24T11:51:10.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:09 smithi053 ceph-mon[47448]: pgmap v408: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-05-24T11:51:10.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:09 smithi137 ceph-mon[47907]: pgmap v408: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-05-24T11:51:10.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:09 smithi039 ceph-mon[42919]: pgmap v408: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-05-24T11:51:12.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:11 smithi053 ceph-mon[47448]: pgmap v409: 97 pgs: 97 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 17 MiB/s wr, 1.30k op/s 2024-05-24T11:51:12.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:11 smithi137 ceph-mon[47907]: pgmap v409: 97 pgs: 97 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 17 MiB/s wr, 1.30k op/s 2024-05-24T11:51:12.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:11 smithi039 ceph-mon[42919]: pgmap v409: 97 pgs: 97 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 17 MiB/s wr, 1.30k op/s 2024-05-24T11:51:14.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 710.0 s, 842.0 tps, lat 119.598 ms stddev 59.967 2024-05-24T11:51:14.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:13 smithi039 ceph-mon[42919]: pgmap v410: 97 pgs: 97 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 17 MiB/s wr, 1.30k op/s 2024-05-24T11:51:14.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:13 smithi053 ceph-mon[47448]: pgmap v410: 97 pgs: 97 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 17 MiB/s wr, 1.30k op/s 2024-05-24T11:51:14.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:13 smithi137 ceph-mon[47907]: pgmap v410: 97 pgs: 97 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 17 MiB/s wr, 1.30k op/s 2024-05-24T11:51:16.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:15 smithi039 ceph-mon[42919]: pgmap v411: 97 pgs: 97 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 22 MiB/s wr, 1.78k op/s 2024-05-24T11:51:16.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:15 smithi039 ceph-mon[42919]: osdmap e98: 12 total, 12 up, 12 in 2024-05-24T11:51:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.202+0000 7f76a7a90700 1 -- 172.21.15.39:0/4076007452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0109940 msgr2=0x7f76a0109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.202+0000 7f76a7a90700 1 --2- 172.21.15.39:0/4076007452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0109940 0x7f76a0109d20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f769c009990 tx=0x7f769c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.202+0000 7f76a7a90700 1 -- 172.21.15.39:0/4076007452 shutdown_connections 2024-05-24T11:51:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.202+0000 7f76a7a90700 1 --2- 172.21.15.39:0/4076007452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76a0108830 0x7f76a0108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.202+0000 7f76a7a90700 1 --2- 172.21.15.39:0/4076007452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0109940 0x7f76a0109d20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.202+0000 7f76a7a90700 1 --2- 172.21.15.39:0/4076007452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76a010a260 0x7f76a01129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.202+0000 7f76a7a90700 1 -- 172.21.15.39:0/4076007452 >> 172.21.15.39:0/4076007452 conn(0x7f76a006bf80 msgr2=0x7f76a006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.202+0000 7f76a7a90700 1 -- 172.21.15.39:0/4076007452 shutdown_connections 2024-05-24T11:51:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.202+0000 7f76a7a90700 1 -- 172.21.15.39:0/4076007452 wait complete. 2024-05-24T11:51:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.203+0000 7f76a7a90700 1 Processor -- start 2024-05-24T11:51:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.203+0000 7f76a7a90700 1 -- start start 2024-05-24T11:51:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.203+0000 7f76a7a90700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0108830 0x7f76a01106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a7a90700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76a0109940 0x7f76a0110c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a7a90700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76a010a260 0x7f76a0111e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a7a90700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01124e0 con 0x7f76a0109940 2024-05-24T11:51:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a7a90700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0112650 con 0x7f76a0108830 2024-05-24T11:51:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a7a90700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0112810 con 0x7f76a010a260 2024-05-24T11:51:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a582c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0108830 0x7f76a01106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a582c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0108830 0x7f76a01106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:47228/0 (socket says 172.21.15.39:47228) 2024-05-24T11:51:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a582c700 1 -- 172.21.15.39:0/1312793351 learned_addr learned my addr 172.21.15.39:0/1312793351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a502b700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76a0109940 0x7f76a0110c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a602d700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76a010a260 0x7f76a0111e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a582c700 1 -- 172.21.15.39:0/1312793351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76a010a260 msgr2=0x7f76a0111e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a582c700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76a010a260 0x7f76a0111e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a582c700 1 -- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76a0109940 msgr2=0x7f76a0110c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a582c700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76a0109940 0x7f76a0110c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a582c700 1 -- 172.21.15.39:0/1312793351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7698009c40 con 0x7f76a0108830 2024-05-24T11:51:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.204+0000 7f76a582c700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0108830 0x7f76a01106e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f769c004760 tx=0x7f769c004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.205+0000 7f7696ffd700 1 -- 172.21.15.39:0/1312793351 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c01d070 con 0x7f76a0108830 2024-05-24T11:51:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.205+0000 7f7696ffd700 1 -- 172.21.15.39:0/1312793351 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f769c00bec0 con 0x7f76a0108830 2024-05-24T11:51:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.205+0000 7f7696ffd700 1 -- 172.21.15.39:0/1312793351 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c005280 con 0x7f76a0108830 2024-05-24T11:51:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.205+0000 7f76a7a90700 1 -- 172.21.15.39:0/1312793351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7690009520 con 0x7f76a0108830 2024-05-24T11:51:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.205+0000 7f76a7a90700 1 -- 172.21.15.39:0/1312793351 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c0095f0 con 0x7f76a0108830 2024-05-24T11:51:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.209+0000 7f7696ffd700 1 -- 172.21.15.39:0/1312793351 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f769c0054a0 con 0x7f76a0108830 2024-05-24T11:51:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.209+0000 7f76b3c4bb80 1 Processor -- start 2024-05-24T11:51:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76b3c4bb80 1 -- start start 2024-05-24T11:51:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 0x55d92a2b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b3480 0x55d92a2b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a20a0f0 con 0x55d92a2b9270 2024-05-24T11:51:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a207ab0 con 0x55d92a2b8950 2024-05-24T11:51:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a2033a0 con 0x55d92a2b3480 2024-05-24T11:51:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f7696ffd700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f768c0a1390 0x7f768c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76a602d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76a602d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:47234/0 (socket says 172.21.15.39:47234) 2024-05-24T11:51:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76a602d700 1 -- 172.21.15.39:0/2021036777 learned_addr learned my addr 172.21.15.39:0/2021036777 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76a602d700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f768c0a1390 0x7f768c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f7696ffd700 1 -- 172.21.15.39:0/1312793351 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f769c0cdcc0 con 0x7f76a0108830 2024-05-24T11:51:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.210+0000 7f76a502b700 1 --2- 172.21.15.39:0/2021036777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b3480 0x55d92a2b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f76a582c700 1 --2- 172.21.15.39:0/2021036777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 0x55d92a2b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f76a502b700 1 -- 172.21.15.39:0/2021036777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 msgr2=0x55d92a2b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f76a502b700 1 --2- 172.21.15.39:0/2021036777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f76a502b700 1 -- 172.21.15.39:0/2021036777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 msgr2=0x55d92a2b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f76a502b700 1 --2- 172.21.15.39:0/2021036777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 0x55d92a2b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f76a502b700 1 -- 172.21.15.39:0/2021036777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d92a084350 con 0x55d92a2b3480 2024-05-24T11:51:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f76a502b700 1 --2- 172.21.15.39:0/2021036777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b3480 0x55d92a2b38a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f76a0111c30 tx=0x7f76900106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7afa7b7639971a25 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f76a602d700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f768c0a1390 0x7f768c0a3850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7698009f70 tx=0x7f769800b040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f768bfff700 1 -- 172.21.15.39:0/2021036777 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690010e10 con 0x55d92a2b3480 2024-05-24T11:51:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f768bfff700 1 -- 172.21.15.39:0/2021036777 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7690015070 con 0x55d92a2b3480 2024-05-24T11:51:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.211+0000 7f768bfff700 1 -- 172.21.15.39:0/2021036777 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900055e0 con 0x55d92a2b3480 2024-05-24T11:51:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.212+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/2021036777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b3480 msgr2=0x55d92a2b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.212+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/2021036777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b3480 0x55d92a2b38a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f76a0111c30 tx=0x7f76900106b0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.213+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/2021036777 shutdown_connections 2024-05-24T11:51:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.213+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/2021036777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b3480 0x55d92a2b38a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.213+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/2021036777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.213+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/2021036777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 0x55d92a2b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.213+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/2021036777 >> 172.21.15.39:0/2021036777 conn(0x55d92a2111b0 msgr2=0x55d92a2ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.216+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/2021036777 shutdown_connections 2024-05-24T11:51:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.220+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/2021036777 wait complete. 2024-05-24T11:51:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.221+0000 7f76b3c4bb80 1 -- start start 2024-05-24T11:51:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.221+0000 7f76b3c4bb80 1 Processor -- start 2024-05-24T11:51:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.221+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.221+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b8950 0x55d92a2bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.221+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 0x55d92a2bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.221+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a19f500 con 0x55d92a2b8950 2024-05-24T11:51:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.221+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a19f1a0 con 0x55d92a2b3480 2024-05-24T11:51:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.221+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a086640 con 0x55d92a2b9270 2024-05-24T11:51:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.221+0000 7f76b3c4bb80 10 client.0 _ll_register_callbacks cb 0x55d92a1a8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a582c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a582c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:47242/0 (socket says 172.21.15.39:47242) 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a582c700 1 -- 172.21.15.39:0/1544076582 learned_addr learned my addr 172.21.15.39:0/1544076582 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a602d700 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 0x55d92a2bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a502b700 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b8950 0x55d92a2bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a582c700 1 -- 172.21.15.39:0/1544076582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 msgr2=0x55d92a2bcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a582c700 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 0x55d92a2bcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a582c700 1 -- 172.21.15.39:0/1544076582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b8950 msgr2=0x55d92a2bc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a582c700 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b8950 0x55d92a2bc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.222+0000 7f76a582c700 1 -- 172.21.15.39:0/1544076582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76900099a0 con 0x55d92a2b3480 2024-05-24T11:51:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.223+0000 7f76a502b700 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b8950 0x55d92a2bc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:51:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.223+0000 7f76a582c700 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bf260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f769c09ab10 tx=0x7f769c099660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c633c5a5c93169d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.223+0000 7f76897fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:51:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.223+0000 7f76897fa700 1 -- 172.21.15.39:0/1544076582 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c099470 con 0x55d92a2b3480 2024-05-24T11:51:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.223+0000 7f76897fa700 1 -- 172.21.15.39:0/1544076582 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f769c09b510 con 0x55d92a2b3480 2024-05-24T11:51:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.223+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1544076582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d92a15c210 con 0x55d92a2b3480 2024-05-24T11:51:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76897fa700 1 -- 172.21.15.39:0/1544076582 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f769c0a2b60 con 0x55d92a2b3480 2024-05-24T11:51:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76b3c4bb80 10 client.26006 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1544076582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d92a030b00 con 0x55d92a2b3480 2024-05-24T11:51:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76897fa700 1 -- 172.21.15.39:0/1544076582 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d92a15c210 con 0x55d92a2b3480 2024-05-24T11:51:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76897fa700 1 -- 172.21.15.39:0/1544076582 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f769c098af0 con 0x55d92a2b3480 2024-05-24T11:51:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76b3c4bb80 10 client.26006 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76b3c4bb80 10 client.26006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76b3c4bb80 20 client.26006 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d92a2cc770 0x55d92a2ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76b3c4bb80 4 client.26006 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:51:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.224+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1544076582 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d92a2827b0 con 0x55d92a2cc770 2024-05-24T11:51:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.225+0000 7f76a502b700 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d92a2cc770 0x55d92a2ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.225+0000 7f76a502b700 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d92a2cc770 0x55d92a2ceb90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b34a558cdf9601 server_cookie=c785b504184abc4e in_seq=0 out_seq=0 2024-05-24T11:51:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.225+0000 7f76897fa700 10 client.26006 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.226+0000 7f76897fa700 1 -- 172.21.15.39:0/1544076582 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d92a2827b0 con 0x55d92a2cc770 2024-05-24T11:51:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.226+0000 7f76897fa700 10 client.26006 handle_command_reply: tid=0 2024-05-24T11:51:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.226+0000 7f76b3c4bb80 1 client.26006 shutdown 2024-05-24T11:51:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.226+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1544076582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 msgr2=0x55d92a2bf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.226+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bf260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f769c09ab10 tx=0x7f769c099660 comp rx=0 tx=0).stop 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1544076582 shutdown_connections 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 0x55d92a2bcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d92a2cc770 0x55d92a2ceb90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bf260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1544076582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b8950 0x55d92a2bc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76897fa700 0 client.26006 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1544076582 >> 172.21.15.39:0/1544076582 conn(0x55d92a2111b0 msgr2=0x55d92a2b74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1544076582 shutdown_connections 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1544076582 wait complete. 2024-05-24T11:51:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.227+0000 7f76b3c4bb80 20 client.26006 trim_cache size 0 max 16384 2024-05-24T11:51:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:15 smithi053 ceph-mon[47448]: pgmap v411: 97 pgs: 97 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 22 MiB/s wr, 1.78k op/s 2024-05-24T11:51:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:15 smithi053 ceph-mon[47448]: osdmap e98: 12 total, 12 up, 12 in 2024-05-24T11:51:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:15 smithi137 ceph-mon[47907]: pgmap v411: 97 pgs: 97 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 22 MiB/s wr, 1.78k op/s 2024-05-24T11:51:16.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:15 smithi137 ceph-mon[47907]: osdmap e98: 12 total, 12 up, 12 in 2024-05-24T11:51:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76b3c4bb80 1 Processor -- start 2024-05-24T11:51:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76b3c4bb80 1 -- start start 2024-05-24T11:51:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b8950 0x55d92a2c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 0x55d92a2ad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a2a8370 con 0x55d92a2b9270 2024-05-24T11:51:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a2bdb80 con 0x55d92a2b3480 2024-05-24T11:51:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a2d3e20 con 0x55d92a2b8950 2024-05-24T11:51:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76a602d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 0x55d92a2ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76a502b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b8950 0x55d92a2c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76a602d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 0x55d92a2ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55862/0 (socket says 172.21.15.39:55862) 2024-05-24T11:51:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76a602d700 1 -- 172.21.15.39:0/462984838 learned_addr learned my addr 172.21.15.39:0/462984838 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.250+0000 7f76a582c700 1 --2- 172.21.15.39:0/462984838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.251+0000 7f76a502b700 1 -- 172.21.15.39:0/462984838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 msgr2=0x55d92a2bc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.251+0000 7f76a502b700 1 --2- 172.21.15.39:0/462984838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.251+0000 7f76a502b700 1 -- 172.21.15.39:0/462984838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 msgr2=0x55d92a2ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.251+0000 7f76a502b700 1 --2- 172.21.15.39:0/462984838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 0x55d92a2ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.251+0000 7f76a502b700 1 -- 172.21.15.39:0/462984838 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d92a1c9500 con 0x55d92a2b8950 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.251+0000 7f76a502b700 1 --2- 172.21.15.39:0/462984838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b8950 0x55d92a2c8690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76900094f0 tx=0x7f7690009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f298c0bdcaccbbe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.251+0000 7f7694da5700 1 -- 172.21.15.39:0/462984838 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769002d070 con 0x55d92a2b8950 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.251+0000 7f7694da5700 1 -- 172.21.15.39:0/462984838 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7690020040 con 0x55d92a2b8950 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.251+0000 7f7694da5700 1 -- 172.21.15.39:0/462984838 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900044f0 con 0x55d92a2b8950 2024-05-24T11:51:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.252+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/462984838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b8950 msgr2=0x55d92a2c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.252+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/462984838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b8950 0x55d92a2c8690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76900094f0 tx=0x7f7690009400 comp rx=0 tx=0).stop 2024-05-24T11:51:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.253+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/462984838 shutdown_connections 2024-05-24T11:51:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.253+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/462984838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b9270 0x55d92a2ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.253+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/462984838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b3480 0x55d92a2bc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.253+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/462984838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b8950 0x55d92a2c8690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.253+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/462984838 >> 172.21.15.39:0/462984838 conn(0x55d92a2111b0 msgr2=0x55d92a2bbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.253+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/462984838 shutdown_connections 2024-05-24T11:51:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.253+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/462984838 wait complete. 2024-05-24T11:51:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.253+0000 7f76b3c4bb80 1 -- start start 2024-05-24T11:51:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.253+0000 7f76b3c4bb80 1 Processor -- start 2024-05-24T11:51:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b3480 0x55d92a2c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76b3c4bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 0x55d92a2b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a2b6ff0 con 0x55d92a2b3480 2024-05-24T11:51:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a2b7160 con 0x55d92a2b8950 2024-05-24T11:51:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76b3c4bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92a2b7320 con 0x55d92a2b9270 2024-05-24T11:51:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76b3c4bb80 10 client.0 _ll_register_callbacks cb 0x55d92a1a8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76a502b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76a502b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:47260/0 (socket says 172.21.15.39:47260) 2024-05-24T11:51:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76a502b700 1 -- 172.21.15.39:0/1955023270 learned_addr learned my addr 172.21.15.39:0/1955023270 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76a602d700 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 0x55d92a2b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76a502b700 1 -- 172.21.15.39:0/1955023270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 msgr2=0x55d92a2b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76a502b700 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 0x55d92a2b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76a502b700 1 -- 172.21.15.39:0/1955023270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b3480 msgr2=0x55d92a2c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76a502b700 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b3480 0x55d92a2c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.254+0000 7f76a502b700 1 -- 172.21.15.39:0/1955023270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d92a084350 con 0x55d92a2b8950 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.255+0000 7f76a502b700 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b6380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7690009d10 tx=0x7f7690005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9a0c9fe2feecdc server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.255+0000 7f76897fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.255+0000 7f76897fa700 1 -- 172.21.15.39:0/1955023270 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769002d070 con 0x55d92a2b8950 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.255+0000 7f76897fa700 1 -- 172.21.15.39:0/1955023270 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7690017070 con 0x55d92a2b8950 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.255+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1955023270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7690015070 con 0x55d92a2b8950 2024-05-24T11:51:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.255+0000 7f76897fa700 1 -- 172.21.15.39:0/1955023270 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7690028730 con 0x55d92a2b8950 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.255+0000 7f76b3c4bb80 10 client.26012 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.255+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1955023270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d92a088c70 con 0x55d92a2b8950 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.255+0000 7f76897fa700 1 -- 172.21.15.39:0/1955023270 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7690015070 con 0x55d92a2b8950 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.256+0000 7f76897fa700 1 -- 172.21.15.39:0/1955023270 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7690026040 con 0x55d92a2b8950 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.256+0000 7f76b3c4bb80 10 client.26012 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.256+0000 7f76b3c4bb80 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.256+0000 7f76b3c4bb80 20 client.26012 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.256+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d92a323110 0x55d92a325530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.256+0000 7f76b3c4bb80 4 client.26012 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.256+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1955023270 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d92a1fbe60 con 0x55d92a323110 2024-05-24T11:51:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.256+0000 7f76a582c700 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d92a323110 0x55d92a325530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.257+0000 7f76a582c700 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d92a323110 0x55d92a325530 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d2b9af17ee8343 server_cookie=962a16faf1ef7b21 in_seq=0 out_seq=0 2024-05-24T11:51:16.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.257+0000 7f76897fa700 10 client.26012 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.324+0000 7f76897fa700 1 -- 172.21.15.39:0/1955023270 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d92a1fbe60 con 0x55d92a323110 2024-05-24T11:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.324+0000 7f76897fa700 10 client.26012 handle_command_reply: tid=0 2024-05-24T11:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.324+0000 7f76b3c4bb80 1 client.26012 shutdown 2024-05-24T11:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1955023270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 msgr2=0x55d92a2b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b6380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7690009d10 tx=0x7f7690005dd0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1955023270 shutdown_connections 2024-05-24T11:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92a2b9270 0x55d92a2b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55d92a323110 0x55d92a325530 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d92a2b3480 0x55d92a2c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 --2- 172.21.15.39:0/1955023270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d92a2b8950 0x55d92a2b6380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1955023270 >> 172.21.15.39:0/1955023270 conn(0x55d92a2111b0 msgr2=0x55d92a2bbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76897fa700 0 client.26012 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1955023270 shutdown_connections 2024-05-24T11:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.325+0000 7f76b3c4bb80 1 -- 172.21.15.39:0/1955023270 wait complete. 2024-05-24T11:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.326+0000 7f76b3c4bb80 20 client.26012 trim_cache size 0 max 16384 2024-05-24T11:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:51:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:51:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 -- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f768c0a1390 msgr2=0x7f768c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f768c0a1390 0x7f768c0a3850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7698009f70 tx=0x7f769800b040 comp rx=0 tx=0).stop 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 -- 172.21.15.39:0/1312793351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0108830 msgr2=0x7f76a01106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0108830 0x7f76a01106e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f769c004760 tx=0x7f769c004790 comp rx=0 tx=0).stop 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 -- 172.21.15.39:0/1312793351 shutdown_connections 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76a010a260 0x7f76a0111e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f768c0a1390 0x7f768c0a3850 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a0108830 0x7f76a01106e0 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f769c004760 tx=0x7f769c004790 comp rx=0 tx=0).stop 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 --2- 172.21.15.39:0/1312793351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76a0109940 0x7f76a0110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.330+0000 7f76a7a90700 1 -- 172.21.15.39:0/1312793351 >> 172.21.15.39:0/1312793351 conn(0x7f76a006bf80 msgr2=0x7f76a006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.331+0000 7f76a7a90700 1 -- 172.21.15.39:0/1312793351 shutdown_connections 2024-05-24T11:51:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:16.331+0000 7f76a7a90700 1 -- 172.21.15.39:0/1312793351 wait complete. 2024-05-24T11:51:18.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:17 smithi039 ceph-mon[42919]: pgmap v413: 97 pgs: 97 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 17 MiB/s wr, 1.41k op/s 2024-05-24T11:51:18.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:17 smithi053 ceph-mon[47448]: pgmap v413: 97 pgs: 97 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 17 MiB/s wr, 1.41k op/s 2024-05-24T11:51:18.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:17 smithi137 ceph-mon[47907]: pgmap v413: 97 pgs: 97 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 17 MiB/s wr, 1.41k op/s 2024-05-24T11:51:19.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 715.0 s, 759.4 tps, lat 131.950 ms stddev 138.548 2024-05-24T11:51:20.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:19 smithi039 ceph-mon[42919]: pgmap v414: 97 pgs: 97 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 17 MiB/s wr, 1.45k op/s 2024-05-24T11:51:20.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:19 smithi053 ceph-mon[47448]: pgmap v414: 97 pgs: 97 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 17 MiB/s wr, 1.45k op/s 2024-05-24T11:51:20.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:19 smithi137 ceph-mon[47907]: pgmap v414: 97 pgs: 97 active+clean; 5.5 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 17 MiB/s wr, 1.45k op/s 2024-05-24T11:51:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:21 smithi039 ceph-mon[42919]: pgmap v415: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-24T11:51:22.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:21 smithi053 ceph-mon[47448]: pgmap v415: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-24T11:51:22.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:21 smithi137 ceph-mon[47907]: pgmap v415: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-24T11:51:24.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:23 smithi039 ceph-mon[42919]: pgmap v416: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-24T11:51:24.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 720.0 s, 721.8 tps, lat 130.533 ms stddev 47.613 2024-05-24T11:51:24.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:23 smithi053 ceph-mon[47448]: pgmap v416: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-24T11:51:24.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:23 smithi137 ceph-mon[47907]: pgmap v416: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-24T11:51:26.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:25 smithi039 ceph-mon[42919]: pgmap v417: 97 pgs: 97 active+clean; 5.6 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-05-24T11:51:26.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:25 smithi053 ceph-mon[47448]: pgmap v417: 97 pgs: 97 active+clean; 5.6 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-05-24T11:51:26.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:25 smithi137 ceph-mon[47907]: pgmap v417: 97 pgs: 97 active+clean; 5.6 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-05-24T11:51:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:27 smithi039 ceph-mon[42919]: pgmap v418: 97 pgs: 97 active+clean; 5.6 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-05-24T11:51:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:27 smithi053 ceph-mon[47448]: pgmap v418: 97 pgs: 97 active+clean; 5.6 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-05-24T11:51:28.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:27 smithi137 ceph-mon[47907]: pgmap v418: 97 pgs: 97 active+clean; 5.6 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-05-24T11:51:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 725.0 s, 723.6 tps, lat 145.489 ms stddev 113.124 2024-05-24T11:51:29.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:28 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:51:29.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:28 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:51:29.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:28 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:51:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.520+0000 7f97c010b700 1 -- 172.21.15.39:0/899587494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b810d8f0 msgr2=0x7f97b810fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.520+0000 7f97c010b700 1 --2- 172.21.15.39:0/899587494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b810d8f0 0x7f97b810fce0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f97b000b0a0 tx=0x7f97b000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.520+0000 7f97c010b700 1 -- 172.21.15.39:0/899587494 shutdown_connections 2024-05-24T11:51:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.520+0000 7f97c010b700 1 --2- 172.21.15.39:0/899587494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8069730 0x7f97b8069b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.520+0000 7f97c010b700 1 --2- 172.21.15.39:0/899587494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b810d8f0 0x7f97b810fce0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.520+0000 7f97c010b700 1 --2- 172.21.15.39:0/899587494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b806a050 0x7f97b810d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.520+0000 7f97c010b700 1 -- 172.21.15.39:0/899587494 >> 172.21.15.39:0/899587494 conn(0x7f97b8076fd0 msgr2=0x7f97b80773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.520+0000 7f97c010b700 1 -- 172.21.15.39:0/899587494 shutdown_connections 2024-05-24T11:51:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.520+0000 7f97c010b700 1 -- 172.21.15.39:0/899587494 wait complete. 2024-05-24T11:51:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97c010b700 1 Processor -- start 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97c010b700 1 -- start start 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97c010b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8069730 0x7f97b81069f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97c010b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b806a050 0x7f97b8106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97c010b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b810d8f0 0x7f97b8101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97c010b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8102740 con 0x7f97b806a050 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97c010b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b81028b0 con 0x7f97b810d8f0 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97c010b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8102a70 con 0x7f97b8069730 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97be6a8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b810d8f0 0x7f97b8101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97be6a8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b810d8f0 0x7f97b8101f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:45980/0 (socket says 172.21.15.39:45980) 2024-05-24T11:51:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.521+0000 7f97be6a8700 1 -- 172.21.15.39:0/2055690034 learned_addr learned my addr 172.21.15.39:0/2055690034 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.522+0000 7f97bdea7700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8069730 0x7f97b81069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.522+0000 7f97be6a8700 1 -- 172.21.15.39:0/2055690034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8069730 msgr2=0x7f97b81069f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.522+0000 7f97be6a8700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8069730 0x7f97b81069f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.522+0000 7f97be6a8700 1 -- 172.21.15.39:0/2055690034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b806a050 msgr2=0x7f97b8106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.522+0000 7f97be6a8700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b806a050 0x7f97b8106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.522+0000 7f97be6a8700 1 -- 172.21.15.39:0/2055690034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b0009d00 con 0x7f97b810d8f0 2024-05-24T11:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.522+0000 7f97be6a8700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b810d8f0 0x7f97b8101f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97b000b070 tx=0x7f97b000be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.523+0000 7f97aeffd700 1 -- 172.21.15.39:0/2055690034 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000e040 con 0x7f97b810d8f0 2024-05-24T11:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.523+0000 7f97aeffd700 1 -- 172.21.15.39:0/2055690034 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97b0007e20 con 0x7f97b810d8f0 2024-05-24T11:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.523+0000 7f97aeffd700 1 -- 172.21.15.39:0/2055690034 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0005040 con 0x7f97b810d8f0 2024-05-24T11:51:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.526+0000 7f97c010b700 1 -- 172.21.15.39:0/2055690034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b8102cf0 con 0x7f97b810d8f0 2024-05-24T11:51:29.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.527+0000 7f97c010b700 1 -- 172.21.15.39:0/2055690034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b810c950 con 0x7f97b810d8f0 2024-05-24T11:51:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.532+0000 7f97cc2c6b80 1 Processor -- start 2024-05-24T11:51:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.534+0000 7f97aeffd700 1 -- 172.21.15.39:0/2055690034 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f97b001d480 con 0x7f97b810d8f0 2024-05-24T11:51:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.534+0000 7f97aeffd700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f97a40a1310 0x7f97a40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.534+0000 7f97aeffd700 1 -- 172.21.15.39:0/2055690034 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f97b00cd550 con 0x7f97b810d8f0 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f97a40a1310 0x7f97a40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97cc2c6b80 1 -- start start 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c2090f0 con 0x55ec2c2b8270 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c206ab0 con 0x55ec2c2b2480 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c2023a0 con 0x55ec2c2b7950 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97bd6a6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97bd6a6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46446/0 (socket says 172.21.15.39:46446) 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97bd6a6700 1 -- 172.21.15.39:0/27644123 learned_addr learned my addr 172.21.15.39:0/27644123 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.535+0000 7f97bdea7700 1 --2- 172.21.15.39:0/27644123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97be6a8700 1 --2- 172.21.15.39:0/27644123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f97a40a1310 0x7f97a40a37d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97a8005950 tx=0x7f97a80058e0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97bd6a6700 1 -- 172.21.15.39:0/27644123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 msgr2=0x55ec2c2b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/27644123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97bd6a6700 1 -- 172.21.15.39:0/27644123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 msgr2=0x55ec2c2b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/27644123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97bd6a6700 1 -- 172.21.15.39:0/27644123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec2c083350 con 0x55ec2c2b8270 2024-05-24T11:51:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/27644123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b1f40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f97a800eae0 tx=0x7f97a8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f07b5a92a310373 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97a3fff700 1 -- 172.21.15.39:0/27644123 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80108c0 con 0x55ec2c2b8270 2024-05-24T11:51:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97a3fff700 1 -- 172.21.15.39:0/27644123 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97a801f070 con 0x55ec2c2b8270 2024-05-24T11:51:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.536+0000 7f97a3fff700 1 -- 172.21.15.39:0/27644123 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a801be00 con 0x55ec2c2b8270 2024-05-24T11:51:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.537+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/27644123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 msgr2=0x55ec2c2b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.537+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/27644123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b1f40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f97a800eae0 tx=0x7f97a8019000 comp rx=0 tx=0).stop 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.537+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/27644123 shutdown_connections 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.537+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/27644123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.537+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/27644123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.537+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/27644123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b1f40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.537+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/27644123 >> 172.21.15.39:0/27644123 conn(0x55ec2c2101b0 msgr2=0x55ec2c2ac840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.537+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/27644123 shutdown_connections 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.538+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/27644123 wait complete. 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.538+0000 7f97cc2c6b80 1 -- start start 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.538+0000 7f97cc2c6b80 1 Processor -- start 2024-05-24T11:51:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b2480 0x55ec2c2be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b7950 0x55ec2c2bb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b8270 0x55ec2c2bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c19e500 con 0x55ec2c2b7950 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c19e1a0 con 0x55ec2c2b8270 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c085640 con 0x55ec2c2b2480 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97cc2c6b80 10 client.0 _ll_register_callbacks cb 0x55ec2c1a7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97bdea7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b2480 0x55ec2c2be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97bdea7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b2480 0x55ec2c2be230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:36396/0 (socket says 172.21.15.39:36396) 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97bdea7700 1 -- 172.21.15.39:0/3673504445 learned_addr learned my addr 172.21.15.39:0/3673504445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97be6a8700 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b8270 0x55ec2c2bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b7950 0x55ec2c2bb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97bdea7700 1 -- 172.21.15.39:0/3673504445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b8270 msgr2=0x55ec2c2bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97bdea7700 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b8270 0x55ec2c2bbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97bdea7700 1 -- 172.21.15.39:0/3673504445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b7950 msgr2=0x55ec2c2bb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97bdea7700 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b7950 0x55ec2c2bb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.539+0000 7f97bdea7700 1 -- 172.21.15.39:0/3673504445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a800e790 con 0x55ec2c2b2480 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.540+0000 7f97bdea7700 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b2480 0x55ec2c2be230 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f97b4011470 tx=0x7f97b4011780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0f4fab5031ee09e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.540+0000 7f97a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.540+0000 7f97a17fa700 1 -- 172.21.15.39:0/3673504445 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4016070 con 0x55ec2c2b2480 2024-05-24T11:51:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.540+0000 7f97a17fa700 1 -- 172.21.15.39:0/3673504445 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f97b4004030 con 0x55ec2c2b2480 2024-05-24T11:51:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.540+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/3673504445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec2c15b210 con 0x55ec2c2b2480 2024-05-24T11:51:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.540+0000 7f97a17fa700 1 -- 172.21.15.39:0/3673504445 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97b40041f0 con 0x55ec2c2b2480 2024-05-24T11:51:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.540+0000 7f97cc2c6b80 10 client.25954 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.540+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/3673504445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ec2c02fb00 con 0x55ec2c2b2480 2024-05-24T11:51:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.540+0000 7f97a17fa700 1 -- 172.21.15.39:0/3673504445 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ec2c15b210 con 0x55ec2c2b2480 2024-05-24T11:51:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.541+0000 7f97a17fa700 1 -- 172.21.15.39:0/3673504445 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f97b4022070 con 0x55ec2c2b2480 2024-05-24T11:51:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.541+0000 7f97cc2c6b80 10 client.25954 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.541+0000 7f97cc2c6b80 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.541+0000 7f97cc2c6b80 20 client.25954 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.541+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ec2c2cbcc0 0x55ec2c2ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.541+0000 7f97cc2c6b80 4 client.25954 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:51:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.541+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/3673504445 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec2c2817b0 con 0x55ec2c2cbcc0 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.541+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ec2c2cbcc0 0x55ec2c2ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.542+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ec2c2cbcc0 0x55ec2c2ce0e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=981945e4e614cac1 server_cookie=a28507e90d83383f in_seq=0 out_seq=0 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.542+0000 7f97a17fa700 10 client.25954 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.543+0000 7f97a17fa700 1 -- 172.21.15.39:0/3673504445 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec2c2817b0 con 0x55ec2c2cbcc0 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.543+0000 7f97a17fa700 10 client.25954 handle_command_reply: tid=0 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.543+0000 7f97cc2c6b80 1 client.25954 shutdown 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.543+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/3673504445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b2480 msgr2=0x55ec2c2be230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.543+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b2480 0x55ec2c2be230 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f97b4011470 tx=0x7f97b4011780 comp rx=0 tx=0).stop 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.544+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/3673504445 shutdown_connections 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.544+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b2480 0x55ec2c2be230 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.544+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ec2c2cbcc0 0x55ec2c2ce0e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.544+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b8270 0x55ec2c2bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.544+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/3673504445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b7950 0x55ec2c2bb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.544+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/3673504445 >> 172.21.15.39:0/3673504445 conn(0x55ec2c2101b0 msgr2=0x55ec2c2b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.544+0000 7f97a17fa700 0 client.25954 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.544+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/3673504445 shutdown_connections 2024-05-24T11:51:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.544+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/3673504445 wait complete. 2024-05-24T11:51:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.545+0000 7f97cc2c6b80 20 client.25954 trim_cache size 0 max 16384 2024-05-24T11:51:29.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.572+0000 7f97cc2c6b80 1 Processor -- start 2024-05-24T11:51:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97cc2c6b80 1 -- start start 2024-05-24T11:51:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2ac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b8270 0x55ec2c2baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c2ad530 con 0x55ec2c2b7950 2024-05-24T11:51:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c2b2da0 con 0x55ec2c2b2480 2024-05-24T11:51:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c2ae0f0 con 0x55ec2c2b8270 2024-05-24T11:51:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97bdea7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97be6a8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b8270 0x55ec2c2baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97bdea7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2ac910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:46020/0 (socket says 172.21.15.39:46020) 2024-05-24T11:51:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.573+0000 7f97bdea7700 1 -- 172.21.15.39:0/987572891 learned_addr learned my addr 172.21.15.39:0/987572891 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97bdea7700 1 -- 172.21.15.39:0/987572891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b8270 msgr2=0x55ec2c2baab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97bdea7700 1 --2- 172.21.15.39:0/987572891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b8270 0x55ec2c2baab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97bdea7700 1 -- 172.21.15.39:0/987572891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b7950 msgr2=0x55ec2c2b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97bdea7700 1 --2- 172.21.15.39:0/987572891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97bdea7700 1 -- 172.21.15.39:0/987572891 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec2c1c8500 con 0x55ec2c2b2480 2024-05-24T11:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97be6a8700 1 --2- 172.21.15.39:0/987572891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b8270 0x55ec2c2baab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97bdea7700 1 --2- 172.21.15.39:0/987572891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2ac910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f97b4011440 tx=0x7f97b4005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fccd729c3527b7a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97acff9700 1 -- 172.21.15.39:0/987572891 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4016040 con 0x55ec2c2b2480 2024-05-24T11:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97acff9700 1 -- 172.21.15.39:0/987572891 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97b4010c10 con 0x55ec2c2b2480 2024-05-24T11:51:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.574+0000 7f97acff9700 1 -- 172.21.15.39:0/987572891 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4003c90 con 0x55ec2c2b2480 2024-05-24T11:51:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.575+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/987572891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 msgr2=0x55ec2c2ac910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.575+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/987572891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2ac910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f97b4011440 tx=0x7f97b4005700 comp rx=0 tx=0).stop 2024-05-24T11:51:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.575+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/987572891 shutdown_connections 2024-05-24T11:51:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.575+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/987572891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b8270 0x55ec2c2baab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.575+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/987572891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2ac910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.575+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/987572891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.575+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/987572891 >> 172.21.15.39:0/987572891 conn(0x55ec2c2101b0 msgr2=0x55ec2c2ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.576+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/987572891 shutdown_connections 2024-05-24T11:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.576+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/987572891 wait complete. 2024-05-24T11:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.577+0000 7f97cc2c6b80 1 -- start start 2024-05-24T11:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.577+0000 7f97cc2c6b80 1 Processor -- start 2024-05-24T11:51:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.577+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.577+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.577+0000 7f97cc2c6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.577+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c2b5e00 con 0x55ec2c2b8270 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.577+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c2b5f40 con 0x55ec2c2b2480 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.577+0000 7f97cc2c6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec2c2b6100 con 0x55ec2c2b7950 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.577+0000 7f97cc2c6b80 10 client.0 _ll_register_callbacks cb 0x55ec2c1a7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bdea7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bdea7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2c2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:46028/0 (socket says 172.21.15.39:46028) 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bdea7700 1 -- 172.21.15.39:0/1539979463 learned_addr learned my addr 172.21.15.39:0/1539979463 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97be6a8700 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bdea7700 1 -- 172.21.15.39:0/1539979463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 msgr2=0x55ec2c2b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bdea7700 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bdea7700 1 -- 172.21.15.39:0/1539979463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 msgr2=0x55ec2c2b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bdea7700 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bdea7700 1 -- 172.21.15.39:0/1539979463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b810c950 con 0x55ec2c2b2480 2024-05-24T11:51:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97bdea7700 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2c2e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97b40106e0 tx=0x7f97b4005530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c4aa6de40f92b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.578+0000 7f97a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:51:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.579+0000 7f97a17fa700 1 -- 172.21.15.39:0/1539979463 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4003680 con 0x55ec2c2b2480 2024-05-24T11:51:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.579+0000 7f97a17fa700 1 -- 172.21.15.39:0/1539979463 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f97b4003890 con 0x55ec2c2b2480 2024-05-24T11:51:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.579+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/1539979463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97a801f070 con 0x55ec2c2b2480 2024-05-24T11:51:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.579+0000 7f97a17fa700 1 -- 172.21.15.39:0/1539979463 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97b4003a50 con 0x55ec2c2b2480 2024-05-24T11:51:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.579+0000 7f97cc2c6b80 10 client.26030 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.579+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/1539979463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ec2c02fb00 con 0x55ec2c2b2480 2024-05-24T11:51:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.579+0000 7f97a17fa700 1 -- 172.21.15.39:0/1539979463 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97a801f070 con 0x55ec2c2b2480 2024-05-24T11:51:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97a17fa700 1 -- 172.21.15.39:0/1539979463 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f97b4022070 con 0x55ec2c2b2480 2024-05-24T11:51:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97cc2c6b80 10 client.26030 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97cc2c6b80 10 client.26030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97cc2c6b80 20 client.26030 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ec2c322430 0x55ec2c324850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97cc2c6b80 4 client.26030 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:51:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/1539979463 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ec2c1fae60 con 0x55ec2c322430 2024-05-24T11:51:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ec2c322430 0x55ec2c324850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97a17fa700 10 client.26030 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.580+0000 7f97bd6a6700 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ec2c322430 0x55ec2c324850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c42d37c1dc2800 server_cookie=e94eef78555f6223 in_seq=0 out_seq=0 2024-05-24T11:51:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.613+0000 7f97a17fa700 1 -- 172.21.15.39:0/1539979463 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ec2c1fae60 con 0x55ec2c322430 2024-05-24T11:51:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.613+0000 7f97a17fa700 10 client.26030 handle_command_reply: tid=0 2024-05-24T11:51:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.613+0000 7f97cc2c6b80 1 client.26030 shutdown 2024-05-24T11:51:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/1539979463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 msgr2=0x55ec2c2c2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2c2e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97b40106e0 tx=0x7f97b4005530 comp rx=0 tx=0).stop 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/1539979463 shutdown_connections 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ec2c322430 0x55ec2c324850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec2c2b8270 0x55ec2c2b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ec2c2b2480 0x55ec2c2c2e00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 --2- 172.21.15.39:0/1539979463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec2c2b7950 0x55ec2c2b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/1539979463 >> 172.21.15.39:0/1539979463 conn(0x55ec2c2101b0 msgr2=0x55ec2c2d2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97a17fa700 0 client.26030 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/1539979463 shutdown_connections 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.614+0000 7f97cc2c6b80 1 -- 172.21.15.39:0/1539979463 wait complete. 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.615+0000 7f97cc2c6b80 20 client.26030 trim_cache size 0 max 16384 2024-05-24T11:51:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:51:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:51:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:51:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:51:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.619+0000 7f97c010b700 1 -- 172.21.15.39:0/2055690034 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f97a40a1310 msgr2=0x7f97a40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.619+0000 7f97c010b700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f97a40a1310 0x7f97a40a37d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97a8005950 tx=0x7f97a80058e0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.619+0000 7f97c010b700 1 -- 172.21.15.39:0/2055690034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b810d8f0 msgr2=0x7f97b8101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.619+0000 7f97c010b700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b810d8f0 0x7f97b8101f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f97b000b070 tx=0x7f97b000be20 comp rx=0 tx=0).stop 2024-05-24T11:51:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.621+0000 7f97c010b700 1 -- 172.21.15.39:0/2055690034 shutdown_connections 2024-05-24T11:51:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.621+0000 7f97c010b700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97b8069730 0x7f97b81069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.621+0000 7f97c010b700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f97a40a1310 0x7f97a40a37d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.621+0000 7f97c010b700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b810d8f0 0x7f97b8101f70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.621+0000 7f97c010b700 1 --2- 172.21.15.39:0/2055690034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b806a050 0x7f97b8106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.621+0000 7f97c010b700 1 -- 172.21.15.39:0/2055690034 >> 172.21.15.39:0/2055690034 conn(0x7f97b8076fd0 msgr2=0x7f97b810eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.621+0000 7f97c010b700 1 -- 172.21.15.39:0/2055690034 shutdown_connections 2024-05-24T11:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:29.621+0000 7f97c010b700 1 -- 172.21.15.39:0/2055690034 wait complete. 2024-05-24T11:51:30.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:29 smithi053 ceph-mon[47448]: pgmap v419: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 18 MiB/s wr, 1.56k op/s 2024-05-24T11:51:30.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:29 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:51:30.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:29 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:51:30.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:29 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:51:30.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:29 smithi137 ceph-mon[47907]: pgmap v419: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 18 MiB/s wr, 1.56k op/s 2024-05-24T11:51:30.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:29 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:51:30.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:29 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:51:30.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:29 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:51:30.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:29 smithi039 ceph-mon[42919]: pgmap v419: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 18 MiB/s wr, 1.56k op/s 2024-05-24T11:51:30.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:29 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:51:30.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:29 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:51:30.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:29 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:51:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:31 smithi053 ceph-mon[47448]: pgmap v420: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 16 MiB/s wr, 1.38k op/s 2024-05-24T11:51:32.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:31 smithi137 ceph-mon[47907]: pgmap v420: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 16 MiB/s wr, 1.38k op/s 2024-05-24T11:51:32.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:31 smithi039 ceph-mon[42919]: pgmap v420: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 16 MiB/s wr, 1.38k op/s 2024-05-24T11:51:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 730.0 s, 844.3 tps, lat 118.998 ms stddev 42.369 2024-05-24T11:51:34.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:33 smithi053 ceph-mon[47448]: pgmap v421: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-05-24T11:51:34.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:33 smithi137 ceph-mon[47907]: pgmap v421: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-05-24T11:51:34.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:33 smithi039 ceph-mon[42919]: pgmap v421: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-05-24T11:51:36.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:35 smithi053 ceph-mon[47448]: pgmap v422: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 20 MiB/s wr, 1.72k op/s 2024-05-24T11:51:36.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:35 smithi137 ceph-mon[47907]: pgmap v422: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 20 MiB/s wr, 1.72k op/s 2024-05-24T11:51:36.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:35 smithi039 ceph-mon[42919]: pgmap v422: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 20 MiB/s wr, 1.72k op/s 2024-05-24T11:51:38.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:37 smithi053 ceph-mon[47448]: pgmap v423: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:51:38.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:37 smithi137 ceph-mon[47907]: pgmap v423: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:51:38.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:37 smithi039 ceph-mon[42919]: pgmap v423: 97 pgs: 97 active+clean; 5.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:51:39.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 735.0 s, 698.8 tps, lat 142.839 ms stddev 189.761 2024-05-24T11:51:40.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:39 smithi053 ceph-mon[47448]: pgmap v424: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-05-24T11:51:40.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:39 smithi137 ceph-mon[47907]: pgmap v424: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-05-24T11:51:40.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:39 smithi039 ceph-mon[42919]: pgmap v424: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-05-24T11:51:42.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:41 smithi053 ceph-mon[47448]: pgmap v425: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 16 MiB/s wr, 1.40k op/s 2024-05-24T11:51:42.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:41 smithi137 ceph-mon[47907]: pgmap v425: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 16 MiB/s wr, 1.40k op/s 2024-05-24T11:51:42.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:41 smithi039 ceph-mon[42919]: pgmap v425: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 16 MiB/s wr, 1.40k op/s 2024-05-24T11:51:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.829+0000 7f67986cf700 1 -- 172.21.15.39:0/733058152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67901009c0 msgr2=0x7f6790104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.829+0000 7f67986cf700 1 --2- 172.21.15.39:0/733058152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67901009c0 0x7f6790104a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6788009f90 tx=0x7f678800b330 comp rx=0 tx=0).stop 2024-05-24T11:51:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.829+0000 7f67986cf700 1 -- 172.21.15.39:0/733058152 shutdown_connections 2024-05-24T11:51:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.829+0000 7f67986cf700 1 --2- 172.21.15.39:0/733058152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6790105130 0x7f679010ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.829+0000 7f67986cf700 1 --2- 172.21.15.39:0/733058152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6790100010 0x7f67901003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.829+0000 7f67986cf700 1 --2- 172.21.15.39:0/733058152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67901009c0 0x7f6790104a00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.829+0000 7f67986cf700 1 -- 172.21.15.39:0/733058152 >> 172.21.15.39:0/733058152 conn(0x7f6790076f40 msgr2=0x7f6790077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.832+0000 7f67986cf700 1 -- 172.21.15.39:0/733058152 shutdown_connections 2024-05-24T11:51:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.832+0000 7f67986cf700 1 -- 172.21.15.39:0/733058152 wait complete. 2024-05-24T11:51:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.832+0000 7f67986cf700 1 Processor -- start 2024-05-24T11:51:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.833+0000 7f67986cf700 1 -- start start 2024-05-24T11:51:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.833+0000 7f67986cf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6790100010 0x7f6790108d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.833+0000 7f67986cf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6790105130 0x7f67901092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.833+0000 7f67986cf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6790109800 0x7f6790072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.833+0000 7f67986cf700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679010ce20 con 0x7f6790105130 2024-05-24T11:51:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.833+0000 7f67986cf700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679010cf60 con 0x7f6790100010 2024-05-24T11:51:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.833+0000 7f67986cf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6790114780 con 0x7f6790109800 2024-05-24T11:51:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.834+0000 7f6795c6a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6790105130 0x7f67901092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.834+0000 7f6795c6a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6790105130 0x7f67901092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39400/0 (socket says 172.21.15.39:39400) 2024-05-24T11:51:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.834+0000 7f6795c6a700 1 -- 172.21.15.39:0/3331136126 learned_addr learned my addr 172.21.15.39:0/3331136126 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.834+0000 7f6795c6a700 1 -- 172.21.15.39:0/3331136126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6790109800 msgr2=0x7f6790072a10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-24T11:51:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.834+0000 7f6795c6a700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6790109800 0x7f6790072a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.834+0000 7f6795c6a700 1 -- 172.21.15.39:0/3331136126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6790100010 msgr2=0x7f6790108d80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-24T11:51:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.834+0000 7f6795c6a700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6790100010 0x7f6790108d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.834+0000 7f6795c6a700 1 -- 172.21.15.39:0/3331136126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6788009c40 con 0x7f6790105130 2024-05-24T11:51:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.835+0000 7f6795c6a700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6790105130 0x7f67901092c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f678800d010 tx=0x7f6788004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.835+0000 7f67877fe700 1 -- 172.21.15.39:0/3331136126 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678800e040 con 0x7f6790105130 2024-05-24T11:51:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.835+0000 7f67877fe700 1 -- 172.21.15.39:0/3331136126 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6788007de0 con 0x7f6790105130 2024-05-24T11:51:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.835+0000 7f67986cf700 1 -- 172.21.15.39:0/3331136126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6790114a00 con 0x7f6790105130 2024-05-24T11:51:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.835+0000 7f67986cf700 1 -- 172.21.15.39:0/3331136126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6790114f60 con 0x7f6790105130 2024-05-24T11:51:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.835+0000 7f67877fe700 1 -- 172.21.15.39:0/3331136126 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678801b6b0 con 0x7f6790105130 2024-05-24T11:51:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.838+0000 7f67877fe700 1 -- 172.21.15.39:0/3331136126 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f678801b810 con 0x7f6790105130 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.839+0000 7f67877fe700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f677c0a1310 0x7f677c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.839+0000 7f67877fe700 1 -- 172.21.15.39:0/3331136126 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f67880cdff0 con 0x7f6790105130 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.840+0000 7f67a488ab80 1 Processor -- start 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.840+0000 7f67a488ab80 1 -- start start 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.841+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a4456d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.841+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a4450f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.841+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4451480 0x5620a44518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.841+0000 7f67a488ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a43a80f0 con 0x5620a4457270 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.841+0000 7f67a488ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a43a5ab0 con 0x5620a4451480 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.841+0000 7f67a488ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a43a13a0 con 0x5620a4456950 2024-05-24T11:51:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.841+0000 7f679646b700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f677c0a1310 0x7f677c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.842+0000 7f6795c6a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a4450f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.842+0000 7f6795c6a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a4450f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39410/0 (socket says 172.21.15.39:39410) 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.842+0000 7f6795c6a700 1 -- 172.21.15.39:0/3606725839 learned_addr learned my addr 172.21.15.39:0/3606725839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.842+0000 7f6796c6c700 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4451480 0x5620a44518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.842+0000 7f679646b700 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a4456d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6795c6a700 1 -- 172.21.15.39:0/3606725839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 msgr2=0x5620a4456d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6795c6a700 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a4456d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6795c6a700 1 -- 172.21.15.39:0/3606725839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4451480 msgr2=0x5620a44518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6795c6a700 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4451480 0x5620a44518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6795c6a700 1 -- 172.21.15.39:0/3606725839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a4222350 con 0x5620a4457270 2024-05-24T11:51:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6796c6c700 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4451480 0x5620a44518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:51:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6795c6a700 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a4450f40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6788004ed0 tx=0x7f6788009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de2dce57a9f1436 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f679646b700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f677c0a1310 0x7f677c0a37d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6780005fd0 tx=0x7f6780009750 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6784ff9700 1 -- 172.21.15.39:0/3606725839 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678809b860 con 0x5620a4457270 2024-05-24T11:51:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6784ff9700 1 -- 172.21.15.39:0/3606725839 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f678809b160 con 0x5620a4457270 2024-05-24T11:51:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.843+0000 7f6784ff9700 1 -- 172.21.15.39:0/3606725839 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880a10b0 con 0x5620a4457270 2024-05-24T11:51:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.845+0000 7f67a488ab80 1 -- 172.21.15.39:0/3606725839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 msgr2=0x5620a4450f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.845+0000 7f67a488ab80 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a4450f40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6788004ed0 tx=0x7f6788009920 comp rx=0 tx=0).stop 2024-05-24T11:51:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.845+0000 7f67a488ab80 1 -- 172.21.15.39:0/3606725839 shutdown_connections 2024-05-24T11:51:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.845+0000 7f67a488ab80 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a4456d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.845+0000 7f67a488ab80 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4451480 0x5620a44518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.845+0000 7f67a488ab80 1 --2- 172.21.15.39:0/3606725839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a4450f40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.845+0000 7f67a488ab80 1 -- 172.21.15.39:0/3606725839 >> 172.21.15.39:0/3606725839 conn(0x5620a43af1b0 msgr2=0x5620a444b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.845+0000 7f67a488ab80 1 -- 172.21.15.39:0/3606725839 shutdown_connections 2024-05-24T11:51:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.845+0000 7f67a488ab80 1 -- 172.21.15.39:0/3606725839 wait complete. 2024-05-24T11:51:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.846+0000 7f67a488ab80 1 -- start start 2024-05-24T11:51:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.846+0000 7f67a488ab80 1 Processor -- start 2024-05-24T11:51:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.846+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 0x5620a445d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.846+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.846+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a445abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.846+0000 7f67a488ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a433d500 con 0x5620a4457270 2024-05-24T11:51:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.846+0000 7f67a488ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a433d1a0 con 0x5620a4456950 2024-05-24T11:51:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.846+0000 7f67a488ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a4224640 con 0x5620a4451480 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f67a488ab80 10 client.0 _ll_register_callbacks cb 0x5620a4346ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f6795c6a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f6795c6a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:47220/0 (socket says 172.21.15.39:47220) 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f6795c6a700 1 -- 172.21.15.39:0/2963378343 learned_addr learned my addr 172.21.15.39:0/2963378343 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f6796c6c700 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a445abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f6795c6a700 1 -- 172.21.15.39:0/2963378343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 msgr2=0x5620a445d1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f6795c6a700 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 0x5620a445d1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f6795c6a700 1 -- 172.21.15.39:0/2963378343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 msgr2=0x5620a445abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f6795c6a700 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a445abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.847+0000 7f6795c6a700 1 -- 172.21.15.39:0/2963378343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c0092f0 con 0x5620a4456950 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.848+0000 7f6795c6a700 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445a4b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6788097660 tx=0x7f6788007420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=325a835edf644d69 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.848+0000 7f6779ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:51:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.849+0000 7f6779ffb700 1 -- 172.21.15.39:0/2963378343 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880a1240 con 0x5620a4456950 2024-05-24T11:51:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.849+0000 7f6779ffb700 1 -- 172.21.15.39:0/2963378343 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f67880a3550 con 0x5620a4456950 2024-05-24T11:51:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.849+0000 7f67a488ab80 1 -- 172.21.15.39:0/2963378343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620a42fa210 con 0x5620a4456950 2024-05-24T11:51:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.850+0000 7f6779ffb700 1 -- 172.21.15.39:0/2963378343 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f678809b3c0 con 0x5620a4456950 2024-05-24T11:51:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.850+0000 7f67a488ab80 10 client.26039 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.850+0000 7f67a488ab80 1 -- 172.21.15.39:0/2963378343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f6790114f60 con 0x5620a4456950 2024-05-24T11:51:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.850+0000 7f6779ffb700 1 -- 172.21.15.39:0/2963378343 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6788082080 con 0x5620a4456950 2024-05-24T11:51:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.851+0000 7f6779ffb700 1 -- 172.21.15.39:0/2963378343 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f67880a30c0 con 0x5620a4456950 2024-05-24T11:51:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.851+0000 7f67a488ab80 10 client.26039 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.851+0000 7f67a488ab80 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.851+0000 7f67a488ab80 20 client.26039 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.851+0000 7f67a488ab80 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5620a446a540 0x5620a446c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.851+0000 7f67a488ab80 4 client.26039 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:51:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.851+0000 7f67a488ab80 1 -- 172.21.15.39:0/2963378343 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620a44207b0 con 0x5620a446a540 2024-05-24T11:51:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.851+0000 7f679646b700 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5620a446a540 0x5620a446c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.852+0000 7f679646b700 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5620a446a540 0x5620a446c960 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb8cb0e9d552784 server_cookie=ffd1d0e1508d9e3a in_seq=0 out_seq=0 2024-05-24T11:51:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.852+0000 7f6779ffb700 10 client.26039 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.852+0000 7f6779ffb700 1 -- 172.21.15.39:0/2963378343 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620a44207b0 con 0x5620a446a540 2024-05-24T11:51:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.852+0000 7f6779ffb700 10 client.26039 handle_command_reply: tid=0 2024-05-24T11:51:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.852+0000 7f67a488ab80 1 client.26039 shutdown 2024-05-24T11:51:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 -- 172.21.15.39:0/2963378343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 msgr2=0x5620a445a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445a4b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6788097660 tx=0x7f6788007420 comp rx=0 tx=0).stop 2024-05-24T11:51:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 -- 172.21.15.39:0/2963378343 shutdown_connections 2024-05-24T11:51:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 0x5620a445d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5620a446a540 0x5620a446c960 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a445abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 --2- 172.21.15.39:0/2963378343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445a4b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f6779ffb700 0 client.26039 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 -- 172.21.15.39:0/2963378343 >> 172.21.15.39:0/2963378343 conn(0x5620a43af1b0 msgr2=0x5620a44554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 -- 172.21.15.39:0/2963378343 shutdown_connections 2024-05-24T11:51:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 1 -- 172.21.15.39:0/2963378343 wait complete. 2024-05-24T11:51:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.853+0000 7f67a488ab80 20 client.26039 trim_cache size 0 max 16384 2024-05-24T11:51:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.874+0000 7f67a488ab80 1 Processor -- start 2024-05-24T11:51:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f67a488ab80 1 -- start start 2024-05-24T11:51:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4451480 0x5620a445abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a445a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4457270 0x5620a4451860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f67a488ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a444c6a0 con 0x5620a4451480 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f67a488ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a4471b50 con 0x5620a4457270 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f67a488ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a4450100 con 0x5620a4456950 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f6795c6a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a445a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f6795c6a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a445a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:34316/0 (socket says 172.21.15.39:34316) 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f6795c6a700 1 -- 172.21.15.39:0/771078567 learned_addr learned my addr 172.21.15.39:0/771078567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f6796c6c700 1 --2- 172.21.15.39:0/771078567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4457270 0x5620a4451860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f6795c6a700 1 -- 172.21.15.39:0/771078567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4457270 msgr2=0x5620a4451860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f6795c6a700 1 --2- 172.21.15.39:0/771078567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4457270 0x5620a4451860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f6795c6a700 1 -- 172.21.15.39:0/771078567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4451480 msgr2=0x5620a445abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f6795c6a700 1 --2- 172.21.15.39:0/771078567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4451480 0x5620a445abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.875+0000 7f6795c6a700 1 -- 172.21.15.39:0/771078567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a41ceb00 con 0x5620a4456950 2024-05-24T11:51:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.876+0000 7f6795c6a700 1 --2- 172.21.15.39:0/771078567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a445a470 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f678809ae60 tx=0x7f678809f0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dce20b61858598 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.876+0000 7f67857fa700 1 -- 172.21.15.39:0/771078567 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880a3550 con 0x5620a4456950 2024-05-24T11:51:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.876+0000 7f67857fa700 1 -- 172.21.15.39:0/771078567 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5620a42fa210 con 0x5620a4456950 2024-05-24T11:51:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.876+0000 7f67857fa700 1 -- 172.21.15.39:0/771078567 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880a12a0 con 0x5620a4456950 2024-05-24T11:51:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.877+0000 7f67a488ab80 1 -- 172.21.15.39:0/771078567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 msgr2=0x5620a445a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.877+0000 7f67a488ab80 1 --2- 172.21.15.39:0/771078567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a445a470 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f678809ae60 tx=0x7f678809f0d0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.877+0000 7f67a488ab80 1 -- 172.21.15.39:0/771078567 shutdown_connections 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.877+0000 7f67a488ab80 1 --2- 172.21.15.39:0/771078567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4457270 0x5620a4451860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.877+0000 7f67a488ab80 1 --2- 172.21.15.39:0/771078567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4451480 0x5620a445abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.877+0000 7f67a488ab80 1 --2- 172.21.15.39:0/771078567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4456950 0x5620a445a470 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.877+0000 7f67a488ab80 1 -- 172.21.15.39:0/771078567 >> 172.21.15.39:0/771078567 conn(0x5620a43af1b0 msgr2=0x5620a444f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.877+0000 7f67a488ab80 1 -- 172.21.15.39:0/771078567 shutdown_connections 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 1 -- 172.21.15.39:0/771078567 wait complete. 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 1 -- start start 2024-05-24T11:51:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 1 Processor -- start 2024-05-24T11:51:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 0x5620a4461310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a445ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a445f4f0 con 0x5620a4457270 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a44bfa40 con 0x5620a4456950 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a44bfb80 con 0x5620a4451480 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.878+0000 7f67a488ab80 10 client.0 _ll_register_callbacks cb 0x5620a4346ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f679646b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 0x5620a4461310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f679646b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 0x5620a4461310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:34328/0 (socket says 172.21.15.39:34328) 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f679646b700 1 -- 172.21.15.39:0/500042283 learned_addr learned my addr 172.21.15.39:0/500042283 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f6795c6a700 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f6796c6c700 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a445ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f679646b700 1 -- 172.21.15.39:0/500042283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 msgr2=0x5620a445e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f679646b700 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f679646b700 1 -- 172.21.15.39:0/500042283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 msgr2=0x5620a445ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f679646b700 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a445ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f679646b700 1 -- 172.21.15.39:0/500042283 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6790114f60 con 0x5620a4451480 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f6795c6a700 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f679646b700 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 0x5620a4461310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f678000a010 tx=0x7f6780004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9acfeda026b63c3d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.879+0000 7f6779ffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.880+0000 7f6779ffb700 1 -- 172.21.15.39:0/500042283 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678004f070 con 0x5620a4451480 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.880+0000 7f6779ffb700 1 -- 172.21.15.39:0/500042283 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6780032d60 con 0x5620a4451480 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.880+0000 7f67a488ab80 1 -- 172.21.15.39:0/500042283 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f678809b160 con 0x5620a4451480 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.880+0000 7f6779ffb700 1 -- 172.21.15.39:0/500042283 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6780004830 con 0x5620a4451480 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.880+0000 7f67a488ab80 10 client.25969 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.880+0000 7f67a488ab80 1 -- 172.21.15.39:0/500042283 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5620a4367500 con 0x5620a4451480 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.880+0000 7f6779ffb700 1 -- 172.21.15.39:0/500042283 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f678809b160 con 0x5620a4451480 2024-05-24T11:51:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.880+0000 7f6779ffb700 1 -- 172.21.15.39:0/500042283 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6780052020 con 0x5620a4451480 2024-05-24T11:51:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.881+0000 7f67a488ab80 10 client.25969 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.881+0000 7f67a488ab80 10 client.25969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.881+0000 7f67a488ab80 20 client.25969 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.881+0000 7f67a488ab80 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5620a44c1370 0x5620a44c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.881+0000 7f67a488ab80 4 client.25969 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:51:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.881+0000 7f67a488ab80 1 -- 172.21.15.39:0/500042283 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5620a4399e60 con 0x5620a44c1370 2024-05-24T11:51:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.881+0000 7f6795c6a700 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5620a44c1370 0x5620a44c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.881+0000 7f6779ffb700 10 client.25969 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.881+0000 7f6795c6a700 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5620a44c1370 0x5620a44c3790 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab7faa0366e9ac9e server_cookie=47a2bbe77b33177f in_seq=0 out_seq=0 2024-05-24T11:51:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.936+0000 7f6779ffb700 1 -- 172.21.15.39:0/500042283 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5620a4399e60 con 0x5620a44c1370 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.936+0000 7f6779ffb700 10 client.25969 handle_command_reply: tid=0 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.936+0000 7f67a488ab80 1 client.25969 shutdown 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 -- 172.21.15.39:0/500042283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 msgr2=0x5620a4461310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 0x5620a4461310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f678000a010 tx=0x7f6780004750 comp rx=0 tx=0).stop 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 -- 172.21.15.39:0/500042283 shutdown_connections 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620a4451480 0x5620a4461310 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5620a44c1370 0x5620a44c3790 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620a4457270 0x5620a445ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 --2- 172.21.15.39:0/500042283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5620a4456950 0x5620a445e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 -- 172.21.15.39:0/500042283 >> 172.21.15.39:0/500042283 conn(0x5620a43af1b0 msgr2=0x5620a4450510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f6779ffb700 0 client.25969 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 -- 172.21.15.39:0/500042283 shutdown_connections 2024-05-24T11:51:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 1 -- 172.21.15.39:0/500042283 wait complete. 2024-05-24T11:51:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.938+0000 7f67a488ab80 20 client.25969 trim_cache size 0 max 16384 2024-05-24T11:51:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:51:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:51:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:51:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 -- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f677c0a1310 msgr2=0x7f677c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f677c0a1310 0x7f677c0a37d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6780005fd0 tx=0x7f6780009750 comp rx=0 tx=0).stop 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 -- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6790105130 msgr2=0x7f67901092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6790105130 0x7f67901092c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f678800d010 tx=0x7f6788004db0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 -- 172.21.15.39:0/3331136126 shutdown_connections 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6790109800 0x7f6790072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f677c0a1310 0x7f677c0a37d0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6780005fd0 tx=0x7f6780009750 comp rx=0 tx=0).stop 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6790100010 0x7f6790108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 --2- 172.21.15.39:0/3331136126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6790105130 0x7f67901092c0 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f678800d010 tx=0x7f6788004db0 comp rx=0 tx=0).stop 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.939+0000 7f67986cf700 1 -- 172.21.15.39:0/3331136126 >> 172.21.15.39:0/3331136126 conn(0x7f6790076f40 msgr2=0x7f67900fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.940+0000 7f67986cf700 1 -- 172.21.15.39:0/3331136126 shutdown_connections 2024-05-24T11:51:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:42.942+0000 7f67986cf700 1 -- 172.21.15.39:0/3331136126 wait complete. 2024-05-24T11:51:44.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 740.0 s, 720.8 tps, lat 139.105 ms stddev 48.679 2024-05-24T11:51:44.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:43 smithi053 ceph-mon[47448]: pgmap v426: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:51:44.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:43 smithi137 ceph-mon[47907]: pgmap v426: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:51:44.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:43 smithi039 ceph-mon[42919]: pgmap v426: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-05-24T11:51:46.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:45 smithi053 ceph-mon[47448]: pgmap v427: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-05-24T11:51:46.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:45 smithi137 ceph-mon[47907]: pgmap v427: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-05-24T11:51:46.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:45 smithi039 ceph-mon[42919]: pgmap v427: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 20 MiB/s wr, 1.68k op/s 2024-05-24T11:51:48.243 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:47 smithi053 ceph-mon[47448]: pgmap v428: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 13 MiB/s wr, 1.13k op/s 2024-05-24T11:51:48.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:47 smithi137 ceph-mon[47907]: pgmap v428: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 13 MiB/s wr, 1.13k op/s 2024-05-24T11:51:48.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:47 smithi039 ceph-mon[42919]: pgmap v428: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 13 MiB/s wr, 1.13k op/s 2024-05-24T11:51:49.117 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 745.0 s, 751.0 tps, lat 132.654 ms stddev 41.410 2024-05-24T11:51:50.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:49 smithi053 ceph-mon[47448]: pgmap v429: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-05-24T11:51:50.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:49 smithi137 ceph-mon[47907]: pgmap v429: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-05-24T11:51:50.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:49 smithi039 ceph-mon[42919]: pgmap v429: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-05-24T11:51:52.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:51 smithi053 ceph-mon[47448]: pgmap v430: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:51:52.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:51 smithi137 ceph-mon[47907]: pgmap v430: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:51:52.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:51 smithi039 ceph-mon[42919]: pgmap v430: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 14 MiB/s wr, 1.28k op/s 2024-05-24T11:51:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 750.0 s, 760.0 tps, lat 131.899 ms stddev 47.477 2024-05-24T11:51:54.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:53 smithi053 ceph-mon[47448]: pgmap v431: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 12 MiB/s wr, 1.05k op/s 2024-05-24T11:51:54.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:53 smithi137 ceph-mon[47907]: pgmap v431: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 12 MiB/s wr, 1.05k op/s 2024-05-24T11:51:54.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:53 smithi039 ceph-mon[42919]: pgmap v431: 97 pgs: 97 active+clean; 5.8 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 12 MiB/s wr, 1.05k op/s 2024-05-24T11:51:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.119+0000 7f910e977700 1 -- 172.21.15.39:0/3075398687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910810d5a0 msgr2=0x7f910810d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.119+0000 7f910e977700 1 --2- 172.21.15.39:0/3075398687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910810d5a0 0x7f910810d980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90f80098f0 tx=0x7f90f8009c00 comp rx=0 tx=0).stop 2024-05-24T11:51:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.119+0000 7f910e977700 1 -- 172.21.15.39:0/3075398687 shutdown_connections 2024-05-24T11:51:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.119+0000 7f910e977700 1 --2- 172.21.15.39:0/3075398687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910810d5a0 0x7f910810d980 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.119+0000 7f910e977700 1 --2- 172.21.15.39:0/3075398687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9108107ea0 0x7f9108114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.119+0000 7f910e977700 1 --2- 172.21.15.39:0/3075398687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91081074e0 0x7f9108107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.119+0000 7f910e977700 1 -- 172.21.15.39:0/3075398687 >> 172.21.15.39:0/3075398687 conn(0x7f9108076dc0 msgr2=0x7f9108105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.121+0000 7f910e977700 1 -- 172.21.15.39:0/3075398687 shutdown_connections 2024-05-24T11:51:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.121+0000 7f910e977700 1 -- 172.21.15.39:0/3075398687 wait complete. 2024-05-24T11:51:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.122+0000 7f910e977700 1 Processor -- start 2024-05-24T11:51:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.122+0000 7f910e977700 1 -- start start 2024-05-24T11:51:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f910e977700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91081074e0 0x7f9108081470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f910e977700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9108107ea0 0x7f91080819f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f910e977700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f910810d5a0 0x7f910807a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f910e977700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910807ae80 con 0x7f910810d5a0 2024-05-24T11:51:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f910e977700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910807aff0 con 0x7f91081074e0 2024-05-24T11:51:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f910e977700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910807b1b0 con 0x7f9108107ea0 2024-05-24T11:51:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f910cf14700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f910810d5a0 0x7f910807a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f910cf14700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f910810d5a0 0x7f910807a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57378/0 (socket says 172.21.15.39:57378) 2024-05-24T11:51:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f910cf14700 1 -- 172.21.15.39:0/3899187755 learned_addr learned my addr 172.21.15.39:0/3899187755 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.123+0000 7f9107fff700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91081074e0 0x7f9108081470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.124+0000 7f9107fff700 1 -- 172.21.15.39:0/3899187755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9108107ea0 msgr2=0x7f91080819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.124+0000 7f9107fff700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9108107ea0 0x7f91080819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.124+0000 7f9107fff700 1 -- 172.21.15.39:0/3899187755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f910810d5a0 msgr2=0x7f910807a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.124+0000 7f9107fff700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f910810d5a0 0x7f910807a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.124+0000 7f9107fff700 1 -- 172.21.15.39:0/3899187755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f8009520 con 0x7f91081074e0 2024-05-24T11:51:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.125+0000 7f9107fff700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91081074e0 0x7f9108081470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009fd0 tx=0x7f90f800b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.125+0000 7f91057fa700 1 -- 172.21.15.39:0/3899187755 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f800ba10 con 0x7f91081074e0 2024-05-24T11:51:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.125+0000 7f910e977700 1 -- 172.21.15.39:0/3899187755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910807f000 con 0x7f91081074e0 2024-05-24T11:51:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.125+0000 7f91057fa700 1 -- 172.21.15.39:0/3899187755 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90f800bb70 con 0x7f91081074e0 2024-05-24T11:51:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.125+0000 7f91057fa700 1 -- 172.21.15.39:0/3899187755 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8003cc0 con 0x7f91081074e0 2024-05-24T11:51:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.125+0000 7f910e977700 1 -- 172.21.15.39:0/3899187755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910807f500 con 0x7f91081074e0 2024-05-24T11:51:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.128+0000 7f91057fa700 1 -- 172.21.15.39:0/3899187755 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f90f8005040 con 0x7f91081074e0 2024-05-24T11:51:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.130+0000 7f911ab32b80 1 Processor -- start 2024-05-24T11:51:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.130+0000 7f91057fa700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f90f00a1390 0x7f90f00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.131+0000 7f91057fa700 1 -- 172.21.15.39:0/3899187755 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f90f80cdde0 con 0x7f91081074e0 2024-05-24T11:51:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.131+0000 7f910cf14700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f90f00a1390 0x7f90f00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.131+0000 7f911ab32b80 1 -- start start 2024-05-24T11:51:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.131+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb08dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.131+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb087f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.131+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0888a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.131+0000 7f911ab32b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fafdf0f0 con 0x5590fb088480 2024-05-24T11:51:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.131+0000 7f911ab32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fafdcab0 con 0x5590fb08e270 2024-05-24T11:51:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.131+0000 7f911ab32b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fafd83a0 con 0x5590fb08d950 2024-05-24T11:51:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0888a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0888a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57394/0 (socket says 172.21.15.39:57394) 2024-05-24T11:51:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 -- 172.21.15.39:0/1742417849 learned_addr learned my addr 172.21.15.39:0/1742417849 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f91077fe700 1 --2- 172.21.15.39:0/1742417849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb087f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f90f00a1390 0x7f90f00a3850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9100009fd0 tx=0x7f91000099e0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f9107fff700 1 --2- 172.21.15.39:0/1742417849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb08dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 -- 172.21.15.39:0/1742417849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 msgr2=0x5590fb08dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 --2- 172.21.15.39:0/1742417849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb08dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 -- 172.21.15.39:0/1742417849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 msgr2=0x5590fb087f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 --2- 172.21.15.39:0/1742417849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb087f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 -- 172.21.15.39:0/1742417849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590fae59350 con 0x5590fb088480 2024-05-24T11:51:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.132+0000 7f910cf14700 1 --2- 172.21.15.39:0/1742417849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0888a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f910001a7e0 tx=0x7f910001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca078e57c4094b83 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.133+0000 7f90ee7fc700 1 -- 172.21.15.39:0/1742417849 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100004db0 con 0x5590fb088480 2024-05-24T11:51:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.133+0000 7f90ee7fc700 1 -- 172.21.15.39:0/1742417849 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f910000f040 con 0x5590fb088480 2024-05-24T11:51:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.133+0000 7f90ee7fc700 1 -- 172.21.15.39:0/1742417849 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100022690 con 0x5590fb088480 2024-05-24T11:51:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.134+0000 7f911ab32b80 1 -- 172.21.15.39:0/1742417849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 msgr2=0x5590fb0888a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.134+0000 7f911ab32b80 1 --2- 172.21.15.39:0/1742417849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0888a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f910001a7e0 tx=0x7f910001c000 comp rx=0 tx=0).stop 2024-05-24T11:51:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.134+0000 7f911ab32b80 1 -- 172.21.15.39:0/1742417849 shutdown_connections 2024-05-24T11:51:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.134+0000 7f911ab32b80 1 --2- 172.21.15.39:0/1742417849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb08dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.134+0000 7f911ab32b80 1 --2- 172.21.15.39:0/1742417849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0888a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.134+0000 7f911ab32b80 1 --2- 172.21.15.39:0/1742417849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb087f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.134+0000 7f911ab32b80 1 -- 172.21.15.39:0/1742417849 >> 172.21.15.39:0/1742417849 conn(0x5590fafe61b0 msgr2=0x5590fb082840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.135+0000 7f911ab32b80 1 -- 172.21.15.39:0/1742417849 shutdown_connections 2024-05-24T11:51:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.135+0000 7f911ab32b80 1 -- 172.21.15.39:0/1742417849 wait complete. 2024-05-24T11:51:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.135+0000 7f911ab32b80 1 -- start start 2024-05-24T11:51:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f911ab32b80 1 Processor -- start 2024-05-24T11:51:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb091af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb092030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f911ab32b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590faf74500 con 0x5590fb088480 2024-05-24T11:51:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f911ab32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590faf741a0 con 0x5590fb08e270 2024-05-24T11:51:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f911ab32b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fae5b640 con 0x5590fb08d950 2024-05-24T11:51:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f911ab32b80 10 client.0 _ll_register_callbacks cb 0x5590faf7dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f910cf14700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb092030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f91077fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb091af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f910cf14700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb092030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:43784/0 (socket says 172.21.15.39:43784) 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f91077fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb091af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59364/0 (socket says 172.21.15.39:59364) 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f910cf14700 1 -- 172.21.15.39:0/3513215686 learned_addr learned my addr 172.21.15.39:0/3513215686 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.136+0000 7f9107fff700 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f91077fe700 1 -- 172.21.15.39:0/3513215686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 msgr2=0x5590fb092030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f91077fe700 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb092030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f91077fe700 1 -- 172.21.15.39:0/3513215686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 msgr2=0x5590fb0915b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f91077fe700 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0915b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f91077fe700 1 -- 172.21.15.39:0/3513215686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910001a490 con 0x5590fb08d950 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f910cf14700 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb092030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f91077fe700 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb091af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005d90 tx=0x7f90fc003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3433ea66a7f4edf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f90e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:51:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f90e77fe700 1 -- 172.21.15.39:0/3513215686 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00bce0 con 0x5590fb08d950 2024-05-24T11:51:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f90e77fe700 1 -- 172.21.15.39:0/3513215686 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f90fc00be40 con 0x5590fb08d950 2024-05-24T11:51:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f911ab32b80 1 -- 172.21.15.39:0/3513215686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590faf31210 con 0x5590fb08d950 2024-05-24T11:51:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.137+0000 7f90e77fe700 1 -- 172.21.15.39:0/3513215686 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90fc01a470 con 0x5590fb08d950 2024-05-24T11:51:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.138+0000 7f911ab32b80 10 client.25978 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.138+0000 7f911ab32b80 1 -- 172.21.15.39:0/3513215686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5590fae05b00 con 0x5590fb08d950 2024-05-24T11:51:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.138+0000 7f90e77fe700 1 -- 172.21.15.39:0/3513215686 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5590faf31210 con 0x5590fb08d950 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.139+0000 7f90e77fe700 1 -- 172.21.15.39:0/3513215686 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f90fc01a5d0 con 0x5590fb08d950 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.139+0000 7f911ab32b80 10 client.25978 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.139+0000 7f911ab32b80 10 client.25978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.139+0000 7f911ab32b80 20 client.25978 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.139+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5590fb0a1bc0 0x5590fb0a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.139+0000 7f911ab32b80 4 client.25978 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.139+0000 7f911ab32b80 1 -- 172.21.15.39:0/3513215686 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590fb0577b0 con 0x5590fb0a1bc0 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.139+0000 7f9107fff700 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5590fb0a1bc0 0x5590fb0a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.139+0000 7f9107fff700 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5590fb0a1bc0 0x5590fb0a3fe0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669f5c30b63f2907 server_cookie=293d834065f29fbf in_seq=0 out_seq=0 2024-05-24T11:51:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.140+0000 7f90e77fe700 10 client.25978 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.140+0000 7f90e77fe700 1 -- 172.21.15.39:0/3513215686 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590fb0577b0 con 0x5590fb0a1bc0 2024-05-24T11:51:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.140+0000 7f90e77fe700 10 client.25978 handle_command_reply: tid=0 2024-05-24T11:51:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.143+0000 7f911ab32b80 1 client.25978 shutdown 2024-05-24T11:51:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.143+0000 7f911ab32b80 1 -- 172.21.15.39:0/3513215686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 msgr2=0x5590fb091af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.143+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb091af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005d90 tx=0x7f90fc003c90 comp rx=0 tx=0).stop 2024-05-24T11:51:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f911ab32b80 1 -- 172.21.15.39:0/3513215686 shutdown_connections 2024-05-24T11:51:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5590fb0a1bc0 0x5590fb0a3fe0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb092030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb0915b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3513215686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb091af0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f911ab32b80 1 -- 172.21.15.39:0/3513215686 >> 172.21.15.39:0/3513215686 conn(0x5590fafe61b0 msgr2=0x5590fb08c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f90e77fe700 0 client.25978 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f911ab32b80 1 -- 172.21.15.39:0/3513215686 shutdown_connections 2024-05-24T11:51:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f911ab32b80 1 -- 172.21.15.39:0/3513215686 wait complete. 2024-05-24T11:51:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.145+0000 7f911ab32b80 20 client.25978 trim_cache size 0 max 16384 2024-05-24T11:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.170+0000 7f911ab32b80 1 Processor -- start 2024-05-24T11:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.170+0000 7f911ab32b80 1 -- start start 2024-05-24T11:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.170+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb088480 0x5590fb09d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.170+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb08d950 0x5590fb082910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.170+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08e270 0x5590fb084540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.170+0000 7f911ab32b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fb092c20 con 0x5590fb08d950 2024-05-24T11:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.170+0000 7f911ab32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fb083330 con 0x5590fb088480 2024-05-24T11:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.170+0000 7f911ab32b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fb0834f0 con 0x5590fb08e270 2024-05-24T11:51:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f91077fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb08d950 0x5590fb082910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f91077fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb08d950 0x5590fb082910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57406/0 (socket says 172.21.15.39:57406) 2024-05-24T11:51:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f91077fe700 1 -- 172.21.15.39:0/581945599 learned_addr learned my addr 172.21.15.39:0/581945599 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f9107fff700 1 --2- 172.21.15.39:0/581945599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb088480 0x5590fb09d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f910cf14700 1 --2- 172.21.15.39:0/581945599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08e270 0x5590fb084540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f9107fff700 1 -- 172.21.15.39:0/581945599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08e270 msgr2=0x5590fb084540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f9107fff700 1 --2- 172.21.15.39:0/581945599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08e270 0x5590fb084540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f9107fff700 1 -- 172.21.15.39:0/581945599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb08d950 msgr2=0x5590fb082910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f9107fff700 1 --2- 172.21.15.39:0/581945599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb08d950 0x5590fb082910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.171+0000 7f9107fff700 1 -- 172.21.15.39:0/581945599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590faf9e500 con 0x5590fb088480 2024-05-24T11:51:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.172+0000 7f9107fff700 1 --2- 172.21.15.39:0/581945599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb088480 0x5590fb09d660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f90f80cb4e0 tx=0x7f90f8023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4add0dc2c22c0b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.172+0000 7f90eeffd700 1 -- 172.21.15.39:0/581945599 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f809b630 con 0x5590fb088480 2024-05-24T11:51:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.172+0000 7f90eeffd700 1 -- 172.21.15.39:0/581945599 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90f809af30 con 0x5590fb088480 2024-05-24T11:51:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.172+0000 7f90eeffd700 1 -- 172.21.15.39:0/581945599 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f80052f0 con 0x5590fb088480 2024-05-24T11:51:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.173+0000 7f911ab32b80 1 -- 172.21.15.39:0/581945599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb088480 msgr2=0x5590fb09d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.173+0000 7f911ab32b80 1 --2- 172.21.15.39:0/581945599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb088480 0x5590fb09d660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f90f80cb4e0 tx=0x7f90f8023040 comp rx=0 tx=0).stop 2024-05-24T11:51:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.173+0000 7f911ab32b80 1 -- 172.21.15.39:0/581945599 shutdown_connections 2024-05-24T11:51:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.173+0000 7f911ab32b80 1 --2- 172.21.15.39:0/581945599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08e270 0x5590fb084540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.173+0000 7f911ab32b80 1 --2- 172.21.15.39:0/581945599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb088480 0x5590fb09d660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.173+0000 7f911ab32b80 1 --2- 172.21.15.39:0/581945599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb08d950 0x5590fb082910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.173+0000 7f911ab32b80 1 -- 172.21.15.39:0/581945599 >> 172.21.15.39:0/581945599 conn(0x5590fafe61b0 msgr2=0x5590fb07ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:56.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 -- 172.21.15.39:0/581945599 shutdown_connections 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 -- 172.21.15.39:0/581945599 wait complete. 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 -- start start 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 Processor -- start 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb098bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb0990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb08b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fb08bbf0 con 0x5590fb088480 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fb08bd60 con 0x5590fb08e270 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.174+0000 7f911ab32b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fb08bf20 con 0x5590fb08d950 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.175+0000 7f911ab32b80 10 client.0 _ll_register_callbacks cb 0x5590faf7dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.175+0000 7f91077fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb0990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.175+0000 7f91077fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb0990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59390/0 (socket says 172.21.15.39:59390) 2024-05-24T11:51:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.175+0000 7f91077fe700 1 -- 172.21.15.39:0/3217567833 learned_addr learned my addr 172.21.15.39:0/3217567833 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:51:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.175+0000 7f9107fff700 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb098bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.175+0000 7f910cf14700 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb08b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.175+0000 7f91077fe700 1 -- 172.21.15.39:0/3217567833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 msgr2=0x5590fb08b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.175+0000 7f91077fe700 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb08b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.176+0000 7f91077fe700 1 -- 172.21.15.39:0/3217567833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 msgr2=0x5590fb098bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.176+0000 7f91077fe700 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb098bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.176+0000 7f91077fe700 1 -- 172.21.15.39:0/3217567833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910807f500 con 0x5590fb08d950 2024-05-24T11:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.176+0000 7f91077fe700 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb0990f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005d60 tx=0x7f90fc01a720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d9eafd823f515e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.176+0000 7f90e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.176+0000 7f90e77fe700 1 -- 172.21.15.39:0/3217567833 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc031070 con 0x5590fb08d950 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.176+0000 7f90e77fe700 1 -- 172.21.15.39:0/3217567833 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f90fc0058e0 con 0x5590fb08d950 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.176+0000 7f911ab32b80 1 -- 172.21.15.39:0/3217567833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f910000f040 con 0x5590fb08d950 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f90e77fe700 1 -- 172.21.15.39:0/3217567833 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90fc003fc0 con 0x5590fb08d950 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f911ab32b80 10 client.25981 fetch_fsmap learned FSMap version 34 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f911ab32b80 1 -- 172.21.15.39:0/3217567833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5590fae5dc70 con 0x5590fb08d950 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f90e77fe700 1 -- 172.21.15.39:0/3217567833 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f910000f040 con 0x5590fb08d950 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f90e77fe700 1 -- 172.21.15.39:0/3217567833 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f90fc034020 con 0x5590fb08d950 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f911ab32b80 10 client.25981 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f911ab32b80 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f911ab32b80 20 client.25981 populate_metadata read hostname 'smithi039' 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5590fb0f8520 0x5590fb0fa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f911ab32b80 4 client.25981 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.177+0000 7f911ab32b80 1 -- 172.21.15.39:0/3217567833 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5590fafd0e60 con 0x5590fb0f8520 2024-05-24T11:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.178+0000 7f9107fff700 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5590fb0f8520 0x5590fb0fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.178+0000 7f90e77fe700 10 client.25981 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:56.179+0000 7f9107fff700 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5590fb0f8520 0x5590fb0fa940 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b087573448c1d4f1 server_cookie=4c50149689b04be1 in_seq=0 out_seq=0 2024-05-24T11:51:56.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:55 smithi053 ceph-mon[47448]: pgmap v432: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-05-24T11:51:56.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:55 smithi137 ceph-mon[47907]: pgmap v432: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-05-24T11:51:56.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:55 smithi039 ceph-mon[42919]: pgmap v432: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-05-24T11:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f90e77fe700 1 -- 172.21.15.39:0/3217567833 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5590fafd0e60 con 0x5590fb0f8520 2024-05-24T11:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f90e77fe700 10 client.25981 handle_command_reply: tid=0 2024-05-24T11:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f911ab32b80 1 client.25981 shutdown 2024-05-24T11:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f911ab32b80 1 -- 172.21.15.39:0/3217567833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 msgr2=0x5590fb0990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb0990f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005d60 tx=0x7f90fc01a720 comp rx=0 tx=0).stop 2024-05-24T11:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f911ab32b80 1 -- 172.21.15.39:0/3217567833 shutdown_connections 2024-05-24T11:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5590fb0f8520 0x5590fb0fa940 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5590fb08e270 0x5590fb08b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5590fb088480 0x5590fb098bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f911ab32b80 1 --2- 172.21.15.39:0/3217567833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590fb08d950 0x5590fb0990f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f911ab32b80 1 -- 172.21.15.39:0/3217567833 >> 172.21.15.39:0/3217567833 conn(0x5590fafe61b0 msgr2=0x5590fb086670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.719+0000 7f90e77fe700 0 client.25981 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.720+0000 7f911ab32b80 1 -- 172.21.15.39:0/3217567833 shutdown_connections 2024-05-24T11:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.720+0000 7f911ab32b80 1 -- 172.21.15.39:0/3217567833 wait complete. 2024-05-24T11:51:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.724+0000 7f911ab32b80 20 client.25981 trim_cache size 0 max 16384 2024-05-24T11:51:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:51:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:51:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:51:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:51:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.724+0000 7f910e977700 1 -- 172.21.15.39:0/3899187755 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f90f00a1390 msgr2=0x7f90f00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.724+0000 7f910e977700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f90f00a1390 0x7f90f00a3850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9100009fd0 tx=0x7f91000099e0 comp rx=0 tx=0).stop 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.724+0000 7f910e977700 1 -- 172.21.15.39:0/3899187755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91081074e0 msgr2=0x7f9108081470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.724+0000 7f910e977700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91081074e0 0x7f9108081470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009fd0 tx=0x7f90f800b560 comp rx=0 tx=0).stop 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.724+0000 7f910e977700 1 -- 172.21.15.39:0/3899187755 shutdown_connections 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.725+0000 7f910e977700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f90f00a1390 0x7f90f00a3850 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.725+0000 7f910e977700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f910810d5a0 0x7f910807a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.725+0000 7f910e977700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91081074e0 0x7f9108081470 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009fd0 tx=0x7f90f800b560 comp rx=0 tx=0).stop 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.725+0000 7f910e977700 1 --2- 172.21.15.39:0/3899187755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9108107ea0 0x7f91080819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.725+0000 7f910e977700 1 -- 172.21.15.39:0/3899187755 >> 172.21.15.39:0/3899187755 conn(0x7f9108076dc0 msgr2=0x7f9108078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.725+0000 7f910e977700 1 -- 172.21.15.39:0/3899187755 shutdown_connections 2024-05-24T11:51:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:51:57.725+0000 7f910e977700 1 -- 172.21.15.39:0/3899187755 wait complete. 2024-05-24T11:51:58.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:57 smithi053 ceph-mon[47448]: pgmap v433: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-05-24T11:51:58.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:57 smithi137 ceph-mon[47907]: pgmap v433: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-05-24T11:51:58.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:57 smithi039 ceph-mon[42919]: pgmap v433: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2024-05-24T11:51:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 755.0 s, 850.8 tps, lat 117.369 ms stddev 39.343 2024-05-24T11:52:00.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:51:59 smithi053 ceph-mon[47448]: pgmap v434: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2024-05-24T11:52:00.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:51:59 smithi137 ceph-mon[47907]: pgmap v434: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2024-05-24T11:52:00.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:51:59 smithi039 ceph-mon[42919]: pgmap v434: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2024-05-24T11:52:02.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:01 smithi053 ceph-mon[47448]: pgmap v435: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2024-05-24T11:52:02.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:01 smithi137 ceph-mon[47907]: pgmap v435: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2024-05-24T11:52:02.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:01 smithi039 ceph-mon[42919]: pgmap v435: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2024-05-24T11:52:04.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 760.0 s, 693.6 tps, lat 144.051 ms stddev 75.762 2024-05-24T11:52:04.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:03 smithi053 ceph-mon[47448]: pgmap v436: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-05-24T11:52:04.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:03 smithi053 ceph-mon[47448]: osdmap e99: 12 total, 12 up, 12 in 2024-05-24T11:52:04.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:03 smithi137 ceph-mon[47907]: pgmap v436: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-05-24T11:52:04.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:03 smithi137 ceph-mon[47907]: osdmap e99: 12 total, 12 up, 12 in 2024-05-24T11:52:04.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:03 smithi039 ceph-mon[42919]: pgmap v436: 97 pgs: 97 active+clean; 6.0 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-05-24T11:52:04.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:03 smithi039 ceph-mon[42919]: osdmap e99: 12 total, 12 up, 12 in 2024-05-24T11:52:06.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:05 smithi053 ceph-mon[47448]: pgmap v438: 97 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 63 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-05-24T11:52:06.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:05 smithi137 ceph-mon[47907]: pgmap v438: 97 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 63 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-05-24T11:52:06.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:05 smithi039 ceph-mon[42919]: pgmap v438: 97 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 63 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-05-24T11:52:08.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:07 smithi053 ceph-mon[47448]: pgmap v439: 97 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 63 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-05-24T11:52:08.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:07 smithi137 ceph-mon[47907]: pgmap v439: 97 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 63 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-05-24T11:52:08.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:07 smithi039 ceph-mon[42919]: pgmap v439: 97 pgs: 10 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 63 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2024-05-24T11:52:09.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 765.0 s, 752.8 tps, lat 132.876 ms stddev 48.008 2024-05-24T11:52:10.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:09 smithi053 ceph-mon[47448]: pgmap v440: 97 pgs: 6 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 73 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-05-24T11:52:10.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:09 smithi137 ceph-mon[47907]: pgmap v440: 97 pgs: 6 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 73 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-05-24T11:52:10.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:09 smithi039 ceph-mon[42919]: pgmap v440: 97 pgs: 6 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 73 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-05-24T11:52:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.914+0000 7f8884ade700 1 -- 172.21.15.39:0/1647267082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8880105e90 msgr2=0x7f8880106270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.914+0000 7f8884ade700 1 --2- 172.21.15.39:0/1647267082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8880105e90 0x7f8880106270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8870009990 tx=0x7f8870009ca0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.915+0000 7f8884ade700 1 -- 172.21.15.39:0/1647267082 shutdown_connections 2024-05-24T11:52:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.915+0000 7f8884ade700 1 --2- 172.21.15.39:0/1647267082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8880105e90 0x7f8880106270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.915+0000 7f8884ade700 1 --2- 172.21.15.39:0/1647267082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8880100790 0x7f888010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.915+0000 7f8884ade700 1 --2- 172.21.15.39:0/1647267082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88800ffdd0 0x7f8880100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.915+0000 7f8884ade700 1 -- 172.21.15.39:0/1647267082 >> 172.21.15.39:0/1647267082 conn(0x7f8880076fd0 msgr2=0x7f88800773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.915+0000 7f8884ade700 1 -- 172.21.15.39:0/1647267082 shutdown_connections 2024-05-24T11:52:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.915+0000 7f8884ade700 1 -- 172.21.15.39:0/1647267082 wait complete. 2024-05-24T11:52:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.915+0000 7f8884ade700 1 Processor -- start 2024-05-24T11:52:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f8884ade700 1 -- start start 2024-05-24T11:52:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f8884ade700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800ffdd0 0x7f8880194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f8884ade700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8880100790 0x7f88801954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f8884ade700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8880105e90 0x7f8880199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f8884ade700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8880199e90 con 0x7f88800ffdd0 2024-05-24T11:52:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f8884ade700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888019a000 con 0x7f8880105e90 2024-05-24T11:52:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f8884ade700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888019a1c0 con 0x7f8880100790 2024-05-24T11:52:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f887e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800ffdd0 0x7f8880194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f887ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8880105e90 0x7f8880199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f887e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800ffdd0 0x7f8880194f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59548/0 (socket says 172.21.15.39:59548) 2024-05-24T11:52:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f887e59c700 1 -- 172.21.15.39:0/2081053944 learned_addr learned my addr 172.21.15.39:0/2081053944 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.916+0000 7f887dd9b700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8880100790 0x7f88801954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.917+0000 7f887ed9d700 1 -- 172.21.15.39:0/2081053944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8880100790 msgr2=0x7f88801954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.917+0000 7f887ed9d700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8880100790 0x7f88801954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.917+0000 7f887ed9d700 1 -- 172.21.15.39:0/2081053944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800ffdd0 msgr2=0x7f8880194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.917+0000 7f887ed9d700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800ffdd0 0x7f8880194f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.917+0000 7f887ed9d700 1 -- 172.21.15.39:0/2081053944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8874009520 con 0x7f8880105e90 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.917+0000 7f887dd9b700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8880100790 0x7f88801954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.917+0000 7f887ed9d700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8880105e90 0x7f8880199850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f887800cba0 tx=0x7f887800cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.917+0000 7f886f7fe700 1 -- 172.21.15.39:0/2081053944 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887800ab40 con 0x7f8880105e90 2024-05-24T11:52:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.917+0000 7f8884ade700 1 -- 172.21.15.39:0/2081053944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88700095f0 con 0x7f8880105e90 2024-05-24T11:52:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.918+0000 7f886f7fe700 1 -- 172.21.15.39:0/2081053944 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f887800aca0 con 0x7f8880105e90 2024-05-24T11:52:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.918+0000 7f886f7fe700 1 -- 172.21.15.39:0/2081053944 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88780175a0 con 0x7f8880105e90 2024-05-24T11:52:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.918+0000 7f8884ade700 1 -- 172.21.15.39:0/2081053944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888019a690 con 0x7f8880105e90 2024-05-24T11:52:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.922+0000 7f8890c99b80 1 Processor -- start 2024-05-24T11:52:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.922+0000 7f8890c99b80 1 -- start start 2024-05-24T11:52:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.922+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c21d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c1bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c1c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f8890c99b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049b730f0 con 0x563049c1c480 2024-05-24T11:52:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f8890c99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049b70ab0 con 0x563049c22270 2024-05-24T11:52:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f8890c99b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049b6c3a0 con 0x563049c21950 2024-05-24T11:52:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f886f7fe700 1 -- 172.21.15.39:0/2081053944 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8878017700 con 0x7f8880105e90 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f887e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c1bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f887e59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c1bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:51418/0 (socket says 172.21.15.39:51418) 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f887e59c700 1 -- 172.21.15.39:0/1912215297 learned_addr learned my addr 172.21.15.39:0/1912215297 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f887e59c700 1 -- 172.21.15.39:0/1912215297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 msgr2=0x563049c21d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f887e59c700 1 --2- 172.21.15.39:0/1912215297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c21d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f887e59c700 1 -- 172.21.15.39:0/1912215297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 msgr2=0x563049c1c8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f887e59c700 1 --2- 172.21.15.39:0/1912215297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c1c8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f887e59c700 1 -- 172.21.15.39:0/1912215297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630499ed350 con 0x563049c22270 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.923+0000 7f887e59c700 1 --2- 172.21.15.39:0/1912215297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c1bf40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8870005110 tx=0x7f8870005140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a814fbd852f30e09 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.924+0000 7f886cff9700 1 -- 172.21.15.39:0/1912215297 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887000b810 con 0x563049c22270 2024-05-24T11:52:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.924+0000 7f886f7fe700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f88680a9b90 0x7f88680ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.924+0000 7f886f7fe700 1 -- 172.21.15.39:0/2081053944 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f88780cc520 con 0x7f8880105e90 2024-05-24T11:52:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.924+0000 7f887dd9b700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f88680a9b90 0x7f88680ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.925+0000 7f886cff9700 1 -- 172.21.15.39:0/1912215297 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f887001e040 con 0x563049c22270 2024-05-24T11:52:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.925+0000 7f886cff9700 1 -- 172.21.15.39:0/1912215297 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8870004c50 con 0x563049c22270 2024-05-24T11:52:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 -- 172.21.15.39:0/1912215297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 msgr2=0x563049c1bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1912215297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c1bf40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8870005110 tx=0x7f8870005140 comp rx=0 tx=0).stop 2024-05-24T11:52:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f887dd9b700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f88680a9b90 0x7f88680ac050 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f88801964c0 tx=0x7f8874009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 -- 172.21.15.39:0/1912215297 shutdown_connections 2024-05-24T11:52:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1912215297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c21d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1912215297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c1c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1912215297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c1bf40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 -- 172.21.15.39:0/1912215297 >> 172.21.15.39:0/1912215297 conn(0x563049b7a1b0 msgr2=0x563049c16840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 -- 172.21.15.39:0/1912215297 shutdown_connections 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 -- 172.21.15.39:0/1912215297 wait complete. 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.926+0000 7f8890c99b80 1 -- start start 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f8890c99b80 1 Processor -- start 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c25530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c25c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f8890c99b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049b08500 con 0x563049c1c480 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f8890c99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049b081a0 con 0x563049c22270 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f8890c99b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630499ef640 con 0x563049c21950 2024-05-24T11:52:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f8890c99b80 10 client.0 _ll_register_callbacks cb 0x563049b11ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f887e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f887ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c25c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f887ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c25c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:51430/0 (socket says 172.21.15.39:51430) 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f887ed9d700 1 -- 172.21.15.39:0/658673823 learned_addr learned my addr 172.21.15.39:0/658673823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.927+0000 7f887dd9b700 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c25530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f887ed9d700 1 -- 172.21.15.39:0/658673823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 msgr2=0x563049c25530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f887ed9d700 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c25530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f887ed9d700 1 -- 172.21.15.39:0/658673823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 msgr2=0x563049c281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f887ed9d700 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f887ed9d700 1 -- 172.21.15.39:0/658673823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888010d520 con 0x563049c22270 2024-05-24T11:52:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f887ed9d700 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c25c70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f88780960a0 tx=0x7f88780ca160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2316d704d4cf7fe server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f8865ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:52:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f8865ffb700 1 -- 172.21.15.39:0/658673823 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878099db0 con 0x563049c22270 2024-05-24T11:52:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f8865ffb700 1 -- 172.21.15.39:0/658673823 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f887800ae10 con 0x563049c22270 2024-05-24T11:52:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.928+0000 7f8890c99b80 1 -- 172.21.15.39:0/658673823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563049ac5210 con 0x563049c22270 2024-05-24T11:52:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8865ffb700 1 -- 172.21.15.39:0/658673823 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88780b90e0 con 0x563049c22270 2024-05-24T11:52:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8890c99b80 10 client.26075 fetch_fsmap learned FSMap version 34 2024-05-24T11:52:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8890c99b80 1 -- 172.21.15.39:0/658673823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563049999b00 con 0x563049c22270 2024-05-24T11:52:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8865ffb700 1 -- 172.21.15.39:0/658673823 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563049ac5210 con 0x563049c22270 2024-05-24T11:52:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8865ffb700 1 -- 172.21.15.39:0/658673823 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f88780a1620 con 0x563049c22270 2024-05-24T11:52:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8890c99b80 10 client.26075 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8890c99b80 10 client.26075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8890c99b80 20 client.26075 populate_metadata read hostname 'smithi039' 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8890c99b80 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563049c46c70 0x563049c49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8890c99b80 4 client.26075 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f8890c99b80 1 -- 172.21.15.39:0/658673823 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563049beb7b0 con 0x563049c46c70 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.929+0000 7f887e59c700 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563049c46c70 0x563049c49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.930+0000 7f887e59c700 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563049c46c70 0x563049c49090 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c463b561cacd659 server_cookie=2064fc408a4690ac in_seq=0 out_seq=0 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.930+0000 7f8865ffb700 10 client.26075 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.930+0000 7f8865ffb700 1 -- 172.21.15.39:0/658673823 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563049beb7b0 con 0x563049c46c70 2024-05-24T11:52:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.930+0000 7f8865ffb700 10 client.26075 handle_command_reply: tid=0 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.930+0000 7f8890c99b80 1 client.26075 shutdown 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 -- 172.21.15.39:0/658673823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 msgr2=0x563049c25c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c25c70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f88780960a0 tx=0x7f88780ca160 comp rx=0 tx=0).stop 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 -- 172.21.15.39:0/658673823 shutdown_connections 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563049c46c70 0x563049c49090 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c25c70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8865ffb700 0 client.26075 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 --2- 172.21.15.39:0/658673823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c25530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 -- 172.21.15.39:0/658673823 >> 172.21.15.39:0/658673823 conn(0x563049b7a1b0 msgr2=0x563049c204a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 -- 172.21.15.39:0/658673823 shutdown_connections 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 1 -- 172.21.15.39:0/658673823 wait complete. 2024-05-24T11:52:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.931+0000 7f8890c99b80 20 client.26075 trim_cache size 0 max 16384 2024-05-24T11:52:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.950+0000 7f8890c99b80 1 Processor -- start 2024-05-24T11:52:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.950+0000 7f8890c99b80 1 -- start start 2024-05-24T11:52:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.950+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.950+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c16910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.950+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c24e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.950+0000 7f8890c99b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049c25350 con 0x563049c1c480 2024-05-24T11:52:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.950+0000 7f8890c99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049c26ae0 con 0x563049c22270 2024-05-24T11:52:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.950+0000 7f8890c99b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049c26270 con 0x563049c21950 2024-05-24T11:52:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887ed9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c24e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59570/0 (socket says 172.21.15.39:59570) 2024-05-24T11:52:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887dd9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c16910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887e59c700 1 -- 172.21.15.39:0/2109974249 learned_addr learned my addr 172.21.15.39:0/2109974249 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887ed9d700 1 -- 172.21.15.39:0/2109974249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 msgr2=0x563049c16910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887ed9d700 1 --2- 172.21.15.39:0/2109974249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c16910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887ed9d700 1 -- 172.21.15.39:0/2109974249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 msgr2=0x563049c426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887ed9d700 1 --2- 172.21.15.39:0/2109974249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887ed9d700 1 -- 172.21.15.39:0/2109974249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563049b32500 con 0x563049c22270 2024-05-24T11:52:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887e59c700 1 --2- 172.21.15.39:0/2109974249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:52:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f887ed9d700 1 --2- 172.21.15.39:0/2109974249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c24e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8878097980 tx=0x7f88780cc450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=439212d747dec30e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.951+0000 7f886d7fa700 1 -- 172.21.15.39:0/2109974249 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88780c9cd0 con 0x563049c22270 2024-05-24T11:52:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.952+0000 7f886d7fa700 1 -- 172.21.15.39:0/2109974249 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88780b90b0 con 0x563049c22270 2024-05-24T11:52:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.952+0000 7f886d7fa700 1 -- 172.21.15.39:0/2109974249 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878099630 con 0x563049c22270 2024-05-24T11:52:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.952+0000 7f8890c99b80 1 -- 172.21.15.39:0/2109974249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 msgr2=0x563049c24e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.952+0000 7f8890c99b80 1 --2- 172.21.15.39:0/2109974249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c24e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8878097980 tx=0x7f88780cc450 comp rx=0 tx=0).stop 2024-05-24T11:52:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.952+0000 7f8890c99b80 1 -- 172.21.15.39:0/2109974249 shutdown_connections 2024-05-24T11:52:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.952+0000 7f8890c99b80 1 --2- 172.21.15.39:0/2109974249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c22270 0x563049c24e10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.952+0000 7f8890c99b80 1 --2- 172.21.15.39:0/2109974249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c1c480 0x563049c426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.952+0000 7f8890c99b80 1 --2- 172.21.15.39:0/2109974249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c21950 0x563049c16910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.952+0000 7f8890c99b80 1 -- 172.21.15.39:0/2109974249 >> 172.21.15.39:0/2109974249 conn(0x563049b7a1b0 msgr2=0x563049c12e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.954+0000 7f8890c99b80 1 -- 172.21.15.39:0/2109974249 shutdown_connections 2024-05-24T11:52:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.956+0000 7f8890c99b80 1 -- 172.21.15.39:0/2109974249 wait complete. 2024-05-24T11:52:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.956+0000 7f8890c99b80 1 -- start start 2024-05-24T11:52:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f8890c99b80 1 Processor -- start 2024-05-24T11:52:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c1c480 0x563049c3dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c21950 0x563049c3bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f8890c99b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c22270 0x563049c3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f8890c99b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049c3c760 con 0x563049c22270 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f8890c99b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049c3c8d0 con 0x563049c21950 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f8890c99b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563049c3ca90 con 0x563049c1c480 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f8890c99b80 10 client.0 _ll_register_callbacks cb 0x563049b11ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f887dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c21950 0x563049c3bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f887dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c21950 0x563049c3bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:51456/0 (socket says 172.21.15.39:51456) 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f887dd9b700 1 -- 172.21.15.39:0/1466334145 learned_addr learned my addr 172.21.15.39:0/1466334145 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.957+0000 7f887e59c700 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c1c480 0x563049c3dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f887ed9d700 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c22270 0x563049c3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f887dd9b700 1 -- 172.21.15.39:0/1466334145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c1c480 msgr2=0x563049c3dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f887dd9b700 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c1c480 0x563049c3dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f887dd9b700 1 -- 172.21.15.39:0/1466334145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c22270 msgr2=0x563049c3c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f887dd9b700 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c22270 0x563049c3c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f887dd9b700 1 -- 172.21.15.39:0/1466334145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630499ed350 con 0x563049c21950 2024-05-24T11:52:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f887dd9b700 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c21950 0x563049c3bc00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8874012c10 tx=0x7f88740047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab82ba64f92e4d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f8865ffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:52:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f8865ffb700 1 -- 172.21.15.39:0/1466334145 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874025070 con 0x563049c21950 2024-05-24T11:52:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f8865ffb700 1 -- 172.21.15.39:0/1466334145 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8874018070 con 0x563049c21950 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.958+0000 7f8890c99b80 1 -- 172.21.15.39:0/1466334145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f887001e040 con 0x563049c21950 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.959+0000 7f8865ffb700 1 -- 172.21.15.39:0/1466334145 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8874004020 con 0x563049c21950 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.959+0000 7f8890c99b80 10 client.26081 fetch_fsmap learned FSMap version 34 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.959+0000 7f8890c99b80 1 -- 172.21.15.39:0/1466334145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563049999b00 con 0x563049c21950 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.960+0000 7f8865ffb700 1 -- 172.21.15.39:0/1466334145 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f887001e040 con 0x563049c21950 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.960+0000 7f8865ffb700 1 -- 172.21.15.39:0/1466334145 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8874020640 con 0x563049c21950 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.960+0000 7f8890c99b80 10 client.26081 fetch_fsmap finished waiting for FSMap version 34 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.960+0000 7f8890c99b80 10 client.26081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.960+0000 7f8890c99b80 20 client.26081 populate_metadata read hostname 'smithi039' 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.960+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563049cae8d0 0x563049cb0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.960+0000 7f8890c99b80 4 client.26081 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:52:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.960+0000 7f8890c99b80 1 -- 172.21.15.39:0/1466334145 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563049b64e60 con 0x563049cae8d0 2024-05-24T11:52:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.960+0000 7f887e59c700 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563049cae8d0 0x563049cb0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.961+0000 7f887e59c700 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563049cae8d0 0x563049cb0cf0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd2576f019031e server_cookie=8a628ed16f94d68d in_seq=0 out_seq=0 2024-05-24T11:52:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:10.961+0000 7f8865ffb700 10 client.26081 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8865ffb700 1 -- 172.21.15.39:0/1466334145 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563049b64e60 con 0x563049cae8d0 2024-05-24T11:52:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8865ffb700 10 client.26081 handle_command_reply: tid=0 2024-05-24T11:52:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 client.26081 shutdown 2024-05-24T11:52:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 -- 172.21.15.39:0/1466334145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c21950 msgr2=0x563049c3bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c21950 0x563049c3bc00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8874012c10 tx=0x7f88740047c0 comp rx=0 tx=0).stop 2024-05-24T11:52:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 -- 172.21.15.39:0/1466334145 shutdown_connections 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563049c1c480 0x563049c3dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563049cae8d0 0x563049cb0cf0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563049c22270 0x563049c3c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 --2- 172.21.15.39:0/1466334145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563049c21950 0x563049c3bc00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 -- 172.21.15.39:0/1466334145 >> 172.21.15.39:0/1466334145 conn(0x563049b7a1b0 msgr2=0x563049c1b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8865ffb700 0 client.26081 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 -- 172.21.15.39:0/1466334145 shutdown_connections 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.030+0000 7f8890c99b80 1 -- 172.21.15.39:0/1466334145 wait complete. 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8890c99b80 20 client.26081 trim_cache size 0 max 16384 2024-05-24T11:52:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:52:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:52:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:52:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:52:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 -- 172.21.15.39:0/2081053944 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f88680a9b90 msgr2=0x7f88680ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f88680a9b90 0x7f88680ac050 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f88801964c0 tx=0x7f8874009260 comp rx=0 tx=0).stop 2024-05-24T11:52:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 -- 172.21.15.39:0/2081053944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8880105e90 msgr2=0x7f8880199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8880105e90 0x7f8880199850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f887800cba0 tx=0x7f887800cf60 comp rx=0 tx=0).stop 2024-05-24T11:52:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 -- 172.21.15.39:0/2081053944 shutdown_connections 2024-05-24T11:52:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f88680a9b90 0x7f88680ac050 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8880105e90 0x7f8880199850 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800ffdd0 0x7f8880194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 --2- 172.21.15.39:0/2081053944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8880100790 0x7f88801954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.031+0000 7f8884ade700 1 -- 172.21.15.39:0/2081053944 >> 172.21.15.39:0/2081053944 conn(0x7f8880076fd0 msgr2=0x7f88800fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.032+0000 7f8884ade700 1 -- 172.21.15.39:0/2081053944 shutdown_connections 2024-05-24T11:52:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:11.032+0000 7f8884ade700 1 -- 172.21.15.39:0/2081053944 wait complete. 2024-05-24T11:52:11.982 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:52:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setuser ceph since I am not root 2024-05-24T11:52:11.982 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:52:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setgroup ceph since I am not root 2024-05-24T11:52:11.982 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:52:11 smithi137 ceph-mds[84908]: -- 172.21.15.137:0/1518289005 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56014e4ce1a0 con 0x56014e4a4c00 2024-05-24T11:52:11.983 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:52:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: starting mds.f at 2024-05-24T11:52:12.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:11 smithi053 ceph-mon[47448]: pgmap v441: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 19 MiB/s wr, 1.35k op/s 2024-05-24T11:52:12.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:11 smithi053 ceph-mon[47448]: daemon mds.f restarted 2024-05-24T11:52:12.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:11 smithi053 ceph-mon[47448]: osdmap e100: 12 total, 12 up, 12 in 2024-05-24T11:52:12.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:11 smithi137 ceph-mon[47907]: pgmap v441: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 19 MiB/s wr, 1.35k op/s 2024-05-24T11:52:12.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:11 smithi137 ceph-mon[47907]: daemon mds.f restarted 2024-05-24T11:52:12.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:11 smithi137 ceph-mon[47907]: osdmap e100: 12 total, 12 up, 12 in 2024-05-24T11:52:12.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:11 smithi039 ceph-mon[42919]: pgmap v441: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 19 MiB/s wr, 1.35k op/s 2024-05-24T11:52:12.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:11 smithi039 ceph-mon[42919]: daemon mds.f restarted 2024-05-24T11:52:12.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:11 smithi039 ceph-mon[42919]: osdmap e100: 12 total, 12 up, 12 in 2024-05-24T11:52:13.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:12 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6834/2260432902,v1:172.21.15.137:6835/2260432902] up:boot 2024-05-24T11:52:13.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:12 smithi137 ceph-mon[47907]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:52:13.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:12 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:52:13.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:12 smithi137 ceph-mon[47907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:52:13.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:12 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6834/2260432902,v1:172.21.15.137:6835/2260432902] up:boot 2024-05-24T11:52:13.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:12 smithi039 ceph-mon[42919]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:52:13.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:12 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:52:13.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:12 smithi039 ceph-mon[42919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:52:13.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:12 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6834/2260432902,v1:172.21.15.137:6835/2260432902] up:boot 2024-05-24T11:52:13.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:12 smithi053 ceph-mon[47448]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:52:13.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:12 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:52:13.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:12 smithi053 ceph-mon[47448]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:52:14.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 770.0 s, 804.6 tps, lat 124.511 ms stddev 38.818 2024-05-24T11:52:14.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:13 smithi137 ceph-mon[47907]: pgmap v443: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 20 MiB/s wr, 1.41k op/s 2024-05-24T11:52:14.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:13 smithi039 ceph-mon[42919]: pgmap v443: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 20 MiB/s wr, 1.41k op/s 2024-05-24T11:52:14.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:13 smithi053 ceph-mon[47448]: pgmap v443: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 20 MiB/s wr, 1.41k op/s 2024-05-24T11:52:16.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:15 smithi039 ceph-mon[42919]: pgmap v444: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-05-24T11:52:16.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:15 smithi039 ceph-mon[42919]: osdmap e101: 12 total, 12 up, 12 in 2024-05-24T11:52:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:15 smithi053 ceph-mon[47448]: pgmap v444: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-05-24T11:52:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:15 smithi053 ceph-mon[47448]: osdmap e101: 12 total, 12 up, 12 in 2024-05-24T11:52:16.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:15 smithi137 ceph-mon[47907]: pgmap v444: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 18 MiB/s wr, 1.40k op/s 2024-05-24T11:52:16.245 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:15 smithi137 ceph-mon[47907]: osdmap e101: 12 total, 12 up, 12 in 2024-05-24T11:52:18.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:17 smithi039 ceph-mon[42919]: pgmap v446: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 17 MiB/s wr, 1.34k op/s 2024-05-24T11:52:18.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:17 smithi053 ceph-mon[47448]: pgmap v446: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 17 MiB/s wr, 1.34k op/s 2024-05-24T11:52:18.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:17 smithi137 ceph-mon[47907]: pgmap v446: 97 pgs: 97 active+clean; 5.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 17 MiB/s wr, 1.34k op/s 2024-05-24T11:52:19.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 775.0 s, 725.6 tps, lat 123.320 ms stddev 53.112 2024-05-24T11:52:20.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:19 smithi039 ceph-mon[42919]: pgmap v447: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2024-05-24T11:52:20.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:19 smithi053 ceph-mon[47448]: pgmap v447: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2024-05-24T11:52:20.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:19 smithi137 ceph-mon[47907]: pgmap v447: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2024-05-24T11:52:22.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:21 smithi039 ceph-mon[42919]: pgmap v448: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 21 MiB/s wr, 1.65k op/s 2024-05-24T11:52:22.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:21 smithi053 ceph-mon[47448]: pgmap v448: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 21 MiB/s wr, 1.65k op/s 2024-05-24T11:52:22.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:21 smithi137 ceph-mon[47907]: pgmap v448: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 21 MiB/s wr, 1.65k op/s 2024-05-24T11:52:24.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:23 smithi039 ceph-mon[42919]: pgmap v449: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2024-05-24T11:52:24.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 780.0 s, 823.0 tps, lat 134.800 ms stddev 104.014 2024-05-24T11:52:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.215+0000 7f607fccd700 1 -- 172.21.15.39:0/948499724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60780ffee0 msgr2=0x7f607810c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.215+0000 7f607fccd700 1 --2- 172.21.15.39:0/948499724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60780ffee0 0x7f607810c3a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6068009990 tx=0x7f6068009ca0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.215+0000 7f607fccd700 1 -- 172.21.15.39:0/948499724 shutdown_connections 2024-05-24T11:52:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.215+0000 7f607fccd700 1 --2- 172.21.15.39:0/948499724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60780ffee0 0x7f607810c3a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.215+0000 7f607fccd700 1 --2- 172.21.15.39:0/948499724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60781055e0 0x7f60781059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.215+0000 7f607fccd700 1 --2- 172.21.15.39:0/948499724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60780ff520 0x7f60780ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.215+0000 7f607fccd700 1 -- 172.21.15.39:0/948499724 >> 172.21.15.39:0/948499724 conn(0x7f60780763a0 msgr2=0x7f60780fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.215+0000 7f607fccd700 1 -- 172.21.15.39:0/948499724 shutdown_connections 2024-05-24T11:52:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.216+0000 7f607fccd700 1 -- 172.21.15.39:0/948499724 wait complete. 2024-05-24T11:52:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607fccd700 1 Processor -- start 2024-05-24T11:52:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607fccd700 1 -- start start 2024-05-24T11:52:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607fccd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60780ff520 0x7f607810a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607fccd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60780ffee0 0x7f607810ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607fccd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60781055e0 0x7f60781a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607fccd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607810b5d0 con 0x7f60780ff520 2024-05-24T11:52:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607fccd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60781121f0 con 0x7f60780ffee0 2024-05-24T11:52:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607fccd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60781123b0 con 0x7f60781055e0 2024-05-24T11:52:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607e26a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60781055e0 0x7f60781a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607e26a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60781055e0 0x7f60781a9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:58698/0 (socket says 172.21.15.39:58698) 2024-05-24T11:52:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.217+0000 7f607e26a700 1 -- 172.21.15.39:0/522400206 learned_addr learned my addr 172.21.15.39:0/522400206 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f607d268700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60780ffee0 0x7f607810ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f607da69700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60780ff520 0x7f607810a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f607e26a700 1 -- 172.21.15.39:0/522400206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60780ffee0 msgr2=0x7f607810ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f607e26a700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60780ffee0 0x7f607810ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f607e26a700 1 -- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60780ff520 msgr2=0x7f607810a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f607e26a700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60780ff520 0x7f607810a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f607e26a700 1 -- 172.21.15.39:0/522400206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60680095f0 con 0x7f60781055e0 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f607e26a700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60781055e0 0x7f60781a9780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f606800c010 tx=0x7f6068005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f606effd700 1 -- 172.21.15.39:0/522400206 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606801d070 con 0x7f60781055e0 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.218+0000 7f607fccd700 1 -- 172.21.15.39:0/522400206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6078112630 con 0x7f60781055e0 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.219+0000 7f607fccd700 1 -- 172.21.15.39:0/522400206 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6078112b30 con 0x7f60781055e0 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.219+0000 7f606effd700 1 -- 172.21.15.39:0/522400206 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f606800fb00 con 0x7f60781055e0 2024-05-24T11:52:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.219+0000 7f606effd700 1 -- 172.21.15.39:0/522400206 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068022b50 con 0x7f60781055e0 2024-05-24T11:52:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.221+0000 7f606effd700 1 -- 172.21.15.39:0/522400206 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f606802c440 con 0x7f60781055e0 2024-05-24T11:52:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.222+0000 7f606effd700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f60640a1370 0x7f60640a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.222+0000 7f606effd700 1 -- 172.21.15.39:0/522400206 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f60680ceeb0 con 0x7f60781055e0 2024-05-24T11:52:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.222+0000 7f607da69700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f60640a1370 0x7f60640a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.223+0000 7f608be88b80 1 Processor -- start 2024-05-24T11:52:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.223+0000 7f607da69700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f60640a1370 0x7f60640a3830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f607000a400 tx=0x7f607000d040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.223+0000 7f608be88b80 1 -- start start 2024-05-24T11:52:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.223+0000 7f608be88b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539236950 0x560539236d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.223+0000 7f608be88b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539230f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.223+0000 7f608be88b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 0x5605392318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.223+0000 7f608be88b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605391880f0 con 0x560539237270 2024-05-24T11:52:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.223+0000 7f608be88b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560539185ab0 con 0x560539236950 2024-05-24T11:52:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.223+0000 7f608be88b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605391813a0 con 0x560539231480 2024-05-24T11:52:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607d268700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539230f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607d268700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539230f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52182/0 (socket says 172.21.15.39:52182) 2024-05-24T11:52:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607d268700 1 -- 172.21.15.39:0/854685500 learned_addr learned my addr 172.21.15.39:0/854685500 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607e26a700 1 --2- 172.21.15.39:0/854685500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 0x5605392318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607da69700 1 --2- 172.21.15.39:0/854685500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539236950 0x560539236d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607da69700 1 -- 172.21.15.39:0/854685500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 msgr2=0x5605392318a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607da69700 1 --2- 172.21.15.39:0/854685500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 0x5605392318a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607da69700 1 -- 172.21.15.39:0/854685500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 msgr2=0x560539230f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607da69700 1 --2- 172.21.15.39:0/854685500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539230f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607da69700 1 -- 172.21.15.39:0/854685500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560539002350 con 0x560539236950 2024-05-24T11:52:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f607da69700 1 --2- 172.21.15.39:0/854685500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539236950 0x560539236d30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f607000afd0 tx=0x7f6070011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748c695c0a824863 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.224+0000 7f6063fff700 1 -- 172.21.15.39:0/854685500 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070010040 con 0x560539236950 2024-05-24T11:52:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.225+0000 7f6063fff700 1 -- 172.21.15.39:0/854685500 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f607001c040 con 0x560539236950 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.225+0000 7f6063fff700 1 -- 172.21.15.39:0/854685500 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070015790 con 0x560539236950 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.226+0000 7f608be88b80 1 -- 172.21.15.39:0/854685500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539236950 msgr2=0x560539236d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.226+0000 7f608be88b80 1 --2- 172.21.15.39:0/854685500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539236950 0x560539236d30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f607000afd0 tx=0x7f6070011000 comp rx=0 tx=0).stop 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.226+0000 7f608be88b80 1 -- 172.21.15.39:0/854685500 shutdown_connections 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.226+0000 7f608be88b80 1 --2- 172.21.15.39:0/854685500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 0x5605392318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.226+0000 7f608be88b80 1 --2- 172.21.15.39:0/854685500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539236950 0x560539236d30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.226+0000 7f608be88b80 1 --2- 172.21.15.39:0/854685500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539230f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.226+0000 7f608be88b80 1 -- 172.21.15.39:0/854685500 >> 172.21.15.39:0/854685500 conn(0x56053918f1b0 msgr2=0x56053922b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.226+0000 7f608be88b80 1 -- 172.21.15.39:0/854685500 shutdown_connections 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.226+0000 7f608be88b80 1 -- 172.21.15.39:0/854685500 wait complete. 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.227+0000 7f608be88b80 1 -- start start 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.227+0000 7f608be88b80 1 Processor -- start 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f608be88b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539231480 0x56053923d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f608be88b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 0x56053923a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f608be88b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x56053923acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f608be88b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053911d500 con 0x560539231480 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f608be88b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053911d1a0 con 0x560539237270 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f608be88b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560539004640 con 0x560539236950 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f608be88b80 10 client.0 _ll_register_callbacks cb 0x560539126ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f607e26a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x56053923acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f607d268700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 0x56053923a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f607e26a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x56053923acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:33686/0 (socket says 172.21.15.39:33686) 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f607e26a700 1 -- 172.21.15.39:0/673635165 learned_addr learned my addr 172.21.15.39:0/673635165 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f607e26a700 1 -- 172.21.15.39:0/673635165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 msgr2=0x56053923a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f607e26a700 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 0x56053923a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f607e26a700 1 -- 172.21.15.39:0/673635165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539231480 msgr2=0x56053923d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f607e26a700 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539231480 0x56053923d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.228+0000 7f607e26a700 1 -- 172.21.15.39:0/673635165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607000ae40 con 0x560539237270 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.229+0000 7f607e26a700 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x56053923acd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f60680cc610 tx=0x7f606800ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d28e8e3bd29133 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.229+0000 7f60617fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.229+0000 7f60617fa700 1 -- 172.21.15.39:0/673635165 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606809fe30 con 0x560539237270 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.229+0000 7f608be88b80 1 -- 172.21.15.39:0/673635165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605390da210 con 0x560539237270 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.229+0000 7f60617fa700 1 -- 172.21.15.39:0/673635165 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f60680afe40 con 0x560539237270 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.229+0000 7f60617fa700 1 -- 172.21.15.39:0/673635165 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60680a43f0 con 0x560539237270 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.229+0000 7f608be88b80 10 client.26099 fetch_fsmap learned FSMap version 36 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.229+0000 7f608be88b80 1 -- 172.21.15.39:0/673635165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560538faeb00 con 0x560539237270 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.229+0000 7f60617fa700 1 -- 172.21.15.39:0/673635165 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5605390da210 con 0x560539237270 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.230+0000 7f60617fa700 1 -- 172.21.15.39:0/673635165 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f606809b4f0 con 0x560539237270 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.230+0000 7f608be88b80 10 client.26099 fetch_fsmap finished waiting for FSMap version 36 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.230+0000 7f608be88b80 10 client.26099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:52:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.230+0000 7f608be88b80 20 client.26099 populate_metadata read hostname 'smithi039' 2024-05-24T11:52:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.230+0000 7f608be88b80 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560539240140 0x5605392425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.230+0000 7f608be88b80 4 client.26099 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:52:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.230+0000 7f608be88b80 1 -- 172.21.15.39:0/673635165 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605392007b0 con 0x560539240140 2024-05-24T11:52:24.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:23 smithi053 ceph-mon[47448]: pgmap v449: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2024-05-24T11:52:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.230+0000 7f607da69700 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560539240140 0x5605392425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.233+0000 7f607da69700 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560539240140 0x5605392425a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6edc3f32c1f0bd server_cookie=a855ad6f8086ea97 in_seq=0 out_seq=0 2024-05-24T11:52:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.233+0000 7f60617fa700 10 client.26099 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.235+0000 7f60617fa700 1 -- 172.21.15.39:0/673635165 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605392007b0 con 0x560539240140 2024-05-24T11:52:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.235+0000 7f60617fa700 10 client.26099 handle_command_reply: tid=0 2024-05-24T11:52:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.235+0000 7f608be88b80 1 client.26099 shutdown 2024-05-24T11:52:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.236+0000 7f608be88b80 1 -- 172.21.15.39:0/673635165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 msgr2=0x56053923acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.236+0000 7f608be88b80 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x56053923acd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f60680cc610 tx=0x7f606800ff50 comp rx=0 tx=0).stop 2024-05-24T11:52:24.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:23 smithi137 ceph-mon[47907]: pgmap v449: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2024-05-24T11:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.241+0000 7f608be88b80 1 -- 172.21.15.39:0/673635165 shutdown_connections 2024-05-24T11:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.241+0000 7f608be88b80 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560539240140 0x5605392425a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.241+0000 7f608be88b80 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x56053923acd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.241+0000 7f608be88b80 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539231480 0x56053923d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.241+0000 7f608be88b80 1 --2- 172.21.15.39:0/673635165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 0x56053923a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.241+0000 7f608be88b80 1 -- 172.21.15.39:0/673635165 >> 172.21.15.39:0/673635165 conn(0x56053918f1b0 msgr2=0x5605392354c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.244+0000 7f60617fa700 0 client.26099 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.244+0000 7f608be88b80 1 -- 172.21.15.39:0/673635165 shutdown_connections 2024-05-24T11:52:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.247+0000 7f608be88b80 1 -- 172.21.15.39:0/673635165 wait complete. 2024-05-24T11:52:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.247+0000 7f608be88b80 20 client.26099 trim_cache size 0 max 16384 2024-05-24T11:52:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.269+0000 7f608be88b80 1 Processor -- start 2024-05-24T11:52:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.269+0000 7f608be88b80 1 -- start start 2024-05-24T11:52:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.269+0000 7f608be88b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539231480 0x56053922b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.269+0000 7f608be88b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 0x560539250440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.269+0000 7f608be88b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539250980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.269+0000 7f608be88b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053923b2d0 con 0x560539237270 2024-05-24T11:52:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.269+0000 7f608be88b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560539247790 con 0x560539231480 2024-05-24T11:52:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.269+0000 7f608be88b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560539269020 con 0x560539236950 2024-05-24T11:52:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.269+0000 7f607d268700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 0x560539250440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607e26a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539250980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607da69700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539231480 0x56053922b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607d268700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 0x560539250440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:58740/0 (socket says 172.21.15.39:58740) 2024-05-24T11:52:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607e26a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539250980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52194/0 (socket says 172.21.15.39:52194) 2024-05-24T11:52:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607d268700 1 -- 172.21.15.39:0/244174817 learned_addr learned my addr 172.21.15.39:0/244174817 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607e26a700 1 -- 172.21.15.39:0/244174817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 msgr2=0x560539250440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607e26a700 1 --2- 172.21.15.39:0/244174817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 0x560539250440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607e26a700 1 -- 172.21.15.39:0/244174817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539231480 msgr2=0x56053922b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607e26a700 1 --2- 172.21.15.39:0/244174817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539231480 0x56053922b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607e26a700 1 -- 172.21.15.39:0/244174817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560539147500 con 0x560539237270 2024-05-24T11:52:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f607e26a700 1 --2- 172.21.15.39:0/244174817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539250980 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6068098a70 tx=0x7f606809c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7368fda19624b3cf server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.270+0000 7f606cff9700 1 -- 172.21.15.39:0/244174817 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606809fe30 con 0x560539237270 2024-05-24T11:52:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.271+0000 7f606cff9700 1 -- 172.21.15.39:0/244174817 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60680a43f0 con 0x560539237270 2024-05-24T11:52:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.271+0000 7f606cff9700 1 -- 172.21.15.39:0/244174817 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606809c260 con 0x560539237270 2024-05-24T11:52:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.272+0000 7f608be88b80 1 -- 172.21.15.39:0/244174817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 msgr2=0x560539250980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.272+0000 7f608be88b80 1 --2- 172.21.15.39:0/244174817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539250980 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6068098a70 tx=0x7f606809c140 comp rx=0 tx=0).stop 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.272+0000 7f608be88b80 1 -- 172.21.15.39:0/244174817 shutdown_connections 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.272+0000 7f608be88b80 1 --2- 172.21.15.39:0/244174817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539237270 0x560539250980 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.272+0000 7f608be88b80 1 --2- 172.21.15.39:0/244174817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539231480 0x56053922b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.272+0000 7f608be88b80 1 --2- 172.21.15.39:0/244174817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539236950 0x560539250440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.272+0000 7f608be88b80 1 -- 172.21.15.39:0/244174817 >> 172.21.15.39:0/244174817 conn(0x56053918f1b0 msgr2=0x56053922f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.272+0000 7f608be88b80 1 -- 172.21.15.39:0/244174817 shutdown_connections 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.272+0000 7f608be88b80 1 -- 172.21.15.39:0/244174817 wait complete. 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.273+0000 7f608be88b80 1 -- start start 2024-05-24T11:52:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.273+0000 7f608be88b80 1 Processor -- start 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f608be88b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 0x5605392687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f608be88b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539236950 0x560539250450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f608be88b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x560539266f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f608be88b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560539250d60 con 0x560539236950 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f608be88b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605392674d0 con 0x560539237270 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f608be88b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560539267640 con 0x560539231480 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f608be88b80 10 client.0 _ll_register_callbacks cb 0x560539126ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f607d268700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539236950 0x560539250450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f607e26a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x560539266f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f607d268700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539236950 0x560539250450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52200/0 (socket says 172.21.15.39:52200) 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f607d268700 1 -- 172.21.15.39:0/1090841735 learned_addr learned my addr 172.21.15.39:0/1090841735 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.274+0000 7f607da69700 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 0x5605392687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f607e26a700 1 -- 172.21.15.39:0/1090841735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 msgr2=0x5605392687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f607e26a700 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 0x5605392687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f607e26a700 1 -- 172.21.15.39:0/1090841735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539236950 msgr2=0x560539250450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f607e26a700 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539236950 0x560539250450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f607e26a700 1 -- 172.21.15.39:0/1090841735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560538faeb00 con 0x560539237270 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f607d268700 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539236950 0x560539250450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f607e26a700 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x560539266f90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60680057b0 tx=0x7f606809c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20f5caa94dfb27 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f60617fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f60617fa700 1 -- 172.21.15.39:0/1090841735 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60680ba380 con 0x560539237270 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f60617fa700 1 -- 172.21.15.39:0/1090841735 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f60680afe40 con 0x560539237270 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f608be88b80 1 -- 172.21.15.39:0/1090841735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f607001c040 con 0x560539237270 2024-05-24T11:52:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.275+0000 7f60617fa700 1 -- 172.21.15.39:0/1090841735 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f606809c460 con 0x560539237270 2024-05-24T11:52:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.278+0000 7f608be88b80 10 client.26105 fetch_fsmap learned FSMap version 36 2024-05-24T11:52:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.278+0000 7f608be88b80 1 -- 172.21.15.39:0/1090841735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560539006c70 con 0x560539237270 2024-05-24T11:52:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.278+0000 7f60617fa700 1 -- 172.21.15.39:0/1090841735 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f607001c040 con 0x560539237270 2024-05-24T11:52:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.278+0000 7f60617fa700 1 -- 172.21.15.39:0/1090841735 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f606809b1a0 con 0x560539237270 2024-05-24T11:52:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.279+0000 7f608be88b80 10 client.26105 fetch_fsmap finished waiting for FSMap version 36 2024-05-24T11:52:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.279+0000 7f608be88b80 10 client.26105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:52:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.279+0000 7f608be88b80 20 client.26105 populate_metadata read hostname 'smithi039' 2024-05-24T11:52:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.279+0000 7f608be88b80 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5605392b1e80 0x5605392b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.279+0000 7f608be88b80 4 client.26105 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:52:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.279+0000 7f608be88b80 1 -- 172.21.15.39:0/1090841735 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560539179e60 con 0x5605392b1e80 2024-05-24T11:52:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.279+0000 7f607da69700 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5605392b1e80 0x5605392b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.280+0000 7f607da69700 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5605392b1e80 0x5605392b4270 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea0a7a5503cf711 server_cookie=cdebf0c7426cd23c in_seq=0 out_seq=0 2024-05-24T11:52:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.280+0000 7f60617fa700 10 client.26105 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.348+0000 7f60617fa700 1 -- 172.21.15.39:0/1090841735 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560539179e60 con 0x5605392b1e80 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.348+0000 7f60617fa700 10 client.26105 handle_command_reply: tid=0 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 client.26105 shutdown 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 -- 172.21.15.39:0/1090841735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 msgr2=0x560539266f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x560539266f90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60680057b0 tx=0x7f606809c830 comp rx=0 tx=0).stop 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 -- 172.21.15.39:0/1090841735 shutdown_connections 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560539231480 0x5605392687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5605392b1e80 0x5605392b4270 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560539237270 0x560539266f90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 --2- 172.21.15.39:0/1090841735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560539236950 0x560539250450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 -- 172.21.15.39:0/1090841735 >> 172.21.15.39:0/1090841735 conn(0x56053918f1b0 msgr2=0x560539230590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f60617fa700 0 client.26105 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 -- 172.21.15.39:0/1090841735 shutdown_connections 2024-05-24T11:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 1 -- 172.21.15.39:0/1090841735 wait complete. 2024-05-24T11:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.349+0000 7f608be88b80 20 client.26105 trim_cache size 0 max 16384 2024-05-24T11:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:52:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:52:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 -- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f60640a1370 msgr2=0x7f60640a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f60640a1370 0x7f60640a3830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f607000a400 tx=0x7f607000d040 comp rx=0 tx=0).stop 2024-05-24T11:52:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 -- 172.21.15.39:0/522400206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60781055e0 msgr2=0x7f60781a9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60781055e0 0x7f60781a9780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f606800c010 tx=0x7f6068005740 comp rx=0 tx=0).stop 2024-05-24T11:52:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 -- 172.21.15.39:0/522400206 shutdown_connections 2024-05-24T11:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60781055e0 0x7f60781a9780 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f60640a1370 0x7f60640a3830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60780ff520 0x7f607810a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 --2- 172.21.15.39:0/522400206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60780ffee0 0x7f607810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 -- 172.21.15.39:0/522400206 >> 172.21.15.39:0/522400206 conn(0x7f60780763a0 msgr2=0x7f60780fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 -- 172.21.15.39:0/522400206 shutdown_connections 2024-05-24T11:52:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:24.350+0000 7f607fccd700 1 -- 172.21.15.39:0/522400206 wait complete. 2024-05-24T11:52:26.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:25 smithi053 ceph-mon[47448]: pgmap v450: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 19 MiB/s wr, 1.45k op/s 2024-05-24T11:52:26.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:25 smithi137 ceph-mon[47907]: pgmap v450: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 19 MiB/s wr, 1.45k op/s 2024-05-24T11:52:26.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:25 smithi039 ceph-mon[42919]: pgmap v450: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 19 MiB/s wr, 1.45k op/s 2024-05-24T11:52:28.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:27 smithi053 ceph-mon[47448]: pgmap v451: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:52:28.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:27 smithi137 ceph-mon[47907]: pgmap v451: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:52:28.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:27 smithi039 ceph-mon[42919]: pgmap v451: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-05-24T11:52:29.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 785.0 s, 789.2 tps, lat 125.480 ms stddev 49.579 2024-05-24T11:52:30.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:29 smithi053 ceph-mon[47448]: pgmap v452: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 19 MiB/s wr, 1.51k op/s 2024-05-24T11:52:30.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:29 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:52:30.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:29 smithi137 ceph-mon[47907]: pgmap v452: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 19 MiB/s wr, 1.51k op/s 2024-05-24T11:52:30.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:29 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:52:30.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:29 smithi039 ceph-mon[42919]: pgmap v452: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 19 MiB/s wr, 1.51k op/s 2024-05-24T11:52:30.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:29 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:52:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:30 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:30 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:30 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:30 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:30 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:30 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:30 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:52:31.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:30 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:52:31.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:30 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:30 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:30 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:30 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:30 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:30 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:30 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:30 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:52:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:30 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:52:31.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:30 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:30 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:30 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:30 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:30 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:30 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:30 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:31.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:30 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:52:31.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:30 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:52:31.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:30 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:52:32.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:32 smithi053 ceph-mon[47448]: pgmap v453: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-05-24T11:52:32.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:32 smithi137 ceph-mon[47907]: pgmap v453: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-05-24T11:52:32.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:32 smithi039 ceph-mon[42919]: pgmap v453: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-05-24T11:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 790.0 s, 673.2 tps, lat 149.007 ms stddev 46.154 2024-05-24T11:52:34.493 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:34 smithi053 ceph-mon[47448]: pgmap v454: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-05-24T11:52:34.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:34 smithi137 ceph-mon[47907]: pgmap v454: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-05-24T11:52:34.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:34 smithi039 ceph-mon[42919]: pgmap v454: 97 pgs: 97 active+clean; 5.6 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-05-24T11:52:36.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:36 smithi053 ceph-mon[47448]: pgmap v455: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:52:36.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:36 smithi137 ceph-mon[47907]: pgmap v455: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:52:36.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:36 smithi039 ceph-mon[42919]: pgmap v455: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 21 MiB/s wr, 1.74k op/s 2024-05-24T11:52:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.996+0000 7f672c841700 1 -- 172.21.15.39:0/3589322416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67280ffda0 msgr2=0x7f6728100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.996+0000 7f672c841700 1 --2- 172.21.15.39:0/3589322416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67280ffda0 0x7f6728100220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f671c0098f0 tx=0x7f671c009c00 comp rx=0 tx=0).stop 2024-05-24T11:52:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.996+0000 7f672c841700 1 -- 172.21.15.39:0/3589322416 shutdown_connections 2024-05-24T11:52:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.996+0000 7f672c841700 1 --2- 172.21.15.39:0/3589322416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6728100760 0x7f672810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.996+0000 7f672c841700 1 --2- 172.21.15.39:0/3589322416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6728105e60 0x7f6728106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.996+0000 7f672c841700 1 --2- 172.21.15.39:0/3589322416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67280ffda0 0x7f6728100220 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.996+0000 7f672c841700 1 -- 172.21.15.39:0/3589322416 >> 172.21.15.39:0/3589322416 conn(0x7f6728076f10 msgr2=0x7f6728077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.996+0000 7f672c841700 1 -- 172.21.15.39:0/3589322416 shutdown_connections 2024-05-24T11:52:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.996+0000 7f672c841700 1 -- 172.21.15.39:0/3589322416 wait complete. 2024-05-24T11:52:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f672c841700 1 Processor -- start 2024-05-24T11:52:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f672c841700 1 -- start start 2024-05-24T11:52:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f672c841700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67280ffda0 0x7f672810cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f672c841700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6728100760 0x7f6728108cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f672c841700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6728105e60 0x7f6728072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f672c841700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6728109370 con 0x7f67280ffda0 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f672c841700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67281094b0 con 0x7f6728105e60 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f672c841700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6728109670 con 0x7f6728100760 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f6725d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6728100760 0x7f6728108cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.997+0000 7f6725d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6728100760 0x7f6728108cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38844/0 (socket says 172.21.15.39:38844) 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f6725d9b700 1 -- 172.21.15.39:0/150115883 learned_addr learned my addr 172.21.15.39:0/150115883 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f6726d9d700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6728105e60 0x7f6728072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f672659c700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67280ffda0 0x7f672810cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f6725d9b700 1 -- 172.21.15.39:0/150115883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6728105e60 msgr2=0x7f6728072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f6725d9b700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6728105e60 0x7f6728072a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f6725d9b700 1 -- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67280ffda0 msgr2=0x7f672810cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f6725d9b700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67280ffda0 0x7f672810cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f6725d9b700 1 -- 172.21.15.39:0/150115883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c009520 con 0x7f6728100760 2024-05-24T11:52:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f6726d9d700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6728105e60 0x7f6728072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:52:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f6725d9b700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6728100760 0x7f6728108cc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f671c00fa00 tx=0x7f671c00bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.998+0000 7f67177fe700 1 -- 172.21.15.39:0/150115883 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c01c070 con 0x7f6728100760 2024-05-24T11:52:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.999+0000 7f672c841700 1 -- 172.21.15.39:0/150115883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67281098f0 con 0x7f6728100760 2024-05-24T11:52:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.999+0000 7f67177fe700 1 -- 172.21.15.39:0/150115883 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f671c00bb70 con 0x7f6728100760 2024-05-24T11:52:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.999+0000 7f67177fe700 1 -- 172.21.15.39:0/150115883 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c004930 con 0x7f6728100760 2024-05-24T11:52:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:37.999+0000 7f672c841700 1 -- 172.21.15.39:0/150115883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67281147d0 con 0x7f6728100760 2024-05-24T11:52:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.002+0000 7f67177fe700 1 -- 172.21.15.39:0/150115883 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f671c020070 con 0x7f6728100760 2024-05-24T11:52:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.002+0000 7f67177fe700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f67100a1310 0x7f67100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.003+0000 7f67177fe700 1 -- 172.21.15.39:0/150115883 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f671c0cd960 con 0x7f6728100760 2024-05-24T11:52:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.003+0000 7f672659c700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f67100a1310 0x7f67100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.004+0000 7f672659c700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f67100a1310 0x7f67100a37d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6718005fd0 tx=0x7f6718009310 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.006+0000 7f67389fcb80 1 Processor -- start 2024-05-24T11:52:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.006+0000 7f67389fcb80 1 -- start start 2024-05-24T11:52:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.006+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.006+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc6270 0x558bfecbff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.006+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfecc08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.006+0000 7f67389fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfec170f0 con 0x558bfecc6270 2024-05-24T11:52:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.006+0000 7f67389fcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfec14ab0 con 0x558bfecc5950 2024-05-24T11:52:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.006+0000 7f67389fcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfec103a0 con 0x558bfecc0480 2024-05-24T11:52:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.007+0000 7f672659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.007+0000 7f672659c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:56842/0 (socket says 172.21.15.39:56842) 2024-05-24T11:52:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.007+0000 7f672659c700 1 -- 172.21.15.39:0/770494711 learned_addr learned my addr 172.21.15.39:0/770494711 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.007+0000 7f6726d9d700 1 --2- 172.21.15.39:0/770494711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfecc08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.008+0000 7f6725d9b700 1 --2- 172.21.15.39:0/770494711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc6270 0x558bfecbff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.008+0000 7f6726d9d700 1 -- 172.21.15.39:0/770494711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 msgr2=0x558bfecc5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.008+0000 7f6726d9d700 1 --2- 172.21.15.39:0/770494711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.008+0000 7f6726d9d700 1 -- 172.21.15.39:0/770494711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc6270 msgr2=0x558bfecbff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.008+0000 7f6726d9d700 1 --2- 172.21.15.39:0/770494711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc6270 0x558bfecbff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.008+0000 7f6726d9d700 1 -- 172.21.15.39:0/770494711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bfea91350 con 0x558bfecc0480 2024-05-24T11:52:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.008+0000 7f6726d9d700 1 --2- 172.21.15.39:0/770494711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfecc08a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f67200076b0 tx=0x7f67200079c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2c45aeb6772ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.008+0000 7f6714ff9700 1 -- 172.21.15.39:0/770494711 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720004bb0 con 0x558bfecc0480 2024-05-24T11:52:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.009+0000 7f6714ff9700 1 -- 172.21.15.39:0/770494711 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f672000f040 con 0x558bfecc0480 2024-05-24T11:52:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.009+0000 7f6714ff9700 1 -- 172.21.15.39:0/770494711 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720005260 con 0x558bfecc0480 2024-05-24T11:52:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 -- 172.21.15.39:0/770494711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 msgr2=0x558bfecc08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 --2- 172.21.15.39:0/770494711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfecc08a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f67200076b0 tx=0x7f67200079c0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 -- 172.21.15.39:0/770494711 shutdown_connections 2024-05-24T11:52:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 --2- 172.21.15.39:0/770494711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfecc08a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 --2- 172.21.15.39:0/770494711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 --2- 172.21.15.39:0/770494711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc6270 0x558bfecbff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 -- 172.21.15.39:0/770494711 >> 172.21.15.39:0/770494711 conn(0x558bfec1e1b0 msgr2=0x558bfecba840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 -- 172.21.15.39:0/770494711 shutdown_connections 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 -- 172.21.15.39:0/770494711 wait complete. 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.010+0000 7f67389fcb80 1 -- start start 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f67389fcb80 1 Processor -- start 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfeccc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc6270 0x558bfecc9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f67389fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfebac500 con 0x558bfecc6270 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f67389fcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfebac1a0 con 0x558bfecc5950 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f67389fcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfea93640 con 0x558bfecc0480 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f67389fcb80 10 client.0 _ll_register_callbacks cb 0x558bfebb5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:52:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f672659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfeccc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f6725d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f672659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfeccc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38864/0 (socket says 172.21.15.39:38864) 2024-05-24T11:52:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.011+0000 7f672659c700 1 -- 172.21.15.39:0/1715132464 learned_addr learned my addr 172.21.15.39:0/1715132464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f6725d9b700 1 -- 172.21.15.39:0/1715132464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 msgr2=0x558bfeccc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f6725d9b700 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfeccc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f6725d9b700 1 -- 172.21.15.39:0/1715132464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc6270 msgr2=0x558bfecc9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f6725d9b700 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc6270 0x558bfecc9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f6725d9b700 1 -- 172.21.15.39:0/1715132464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6720009d80 con 0x558bfecc5950 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f6725d9b700 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc94d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f671c0cb060 tx=0x7f671c09b230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4492e544fe00c2b server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f670dffb700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f670dffb700 1 -- 172.21.15.39:0/1715132464 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0a2ea0 con 0x558bfecc5950 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f670dffb700 1 -- 172.21.15.39:0/1715132464 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f671c005260 con 0x558bfecc5950 2024-05-24T11:52:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f67389fcb80 1 -- 172.21.15.39:0/1715132464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bfeb69210 con 0x558bfecc5950 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f670dffb700 1 -- 172.21.15.39:0/1715132464 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f671c005420 con 0x558bfecc5950 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f67389fcb80 10 client.26117 fetch_fsmap learned FSMap version 36 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.012+0000 7f67389fcb80 1 -- 172.21.15.39:0/1715132464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558bfea3db00 con 0x558bfecc5950 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.013+0000 7f670dffb700 1 -- 172.21.15.39:0/1715132464 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558bfeb69210 con 0x558bfecc5950 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.013+0000 7f670dffb700 1 -- 172.21.15.39:0/1715132464 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f671c099fa0 con 0x558bfecc5950 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.013+0000 7f67389fcb80 10 client.26117 fetch_fsmap finished waiting for FSMap version 36 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.013+0000 7f67389fcb80 10 client.26117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.013+0000 7f67389fcb80 20 client.26117 populate_metadata read hostname 'smithi039' 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.013+0000 7f67389fcb80 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558bfecd97b0 0x558bfecdbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.013+0000 7f67389fcb80 4 client.26117 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.013+0000 7f67389fcb80 1 -- 172.21.15.39:0/1715132464 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bfec8f7b0 con 0x558bfecd97b0 2024-05-24T11:52:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.013+0000 7f672659c700 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558bfecd97b0 0x558bfecdbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.014+0000 7f670dffb700 10 client.26117 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.014+0000 7f672659c700 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558bfecd97b0 0x558bfecdbbd0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418c7ba155e0695f server_cookie=efb36f089bb339f7 in_seq=0 out_seq=0 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f670dffb700 1 -- 172.21.15.39:0/1715132464 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558bfec8f7b0 con 0x558bfecd97b0 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f670dffb700 10 client.26117 handle_command_reply: tid=0 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f67389fcb80 1 client.26117 shutdown 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f67389fcb80 1 -- 172.21.15.39:0/1715132464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 msgr2=0x558bfecc94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f67389fcb80 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc94d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f671c0cb060 tx=0x7f671c09b230 comp rx=0 tx=0).stop 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f67389fcb80 1 -- 172.21.15.39:0/1715132464 shutdown_connections 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f67389fcb80 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc0480 0x558bfeccc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f67389fcb80 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558bfecd97b0 0x558bfecdbbd0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f67389fcb80 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc6270 0x558bfecc9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f67389fcb80 1 --2- 172.21.15.39:0/1715132464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc5950 0x558bfecc94d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f670dffb700 0 client.26117 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.015+0000 7f67389fcb80 1 -- 172.21.15.39:0/1715132464 >> 172.21.15.39:0/1715132464 conn(0x558bfec1e1b0 msgr2=0x558bfecbef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.016+0000 7f67389fcb80 1 -- 172.21.15.39:0/1715132464 shutdown_connections 2024-05-24T11:52:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.016+0000 7f67389fcb80 1 -- 172.21.15.39:0/1715132464 wait complete. 2024-05-24T11:52:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.016+0000 7f67389fcb80 20 client.26117 trim_cache size 0 max 16384 2024-05-24T11:52:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.044+0000 7f67389fcb80 1 Processor -- start 2024-05-24T11:52:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.044+0000 7f67389fcb80 1 -- start start 2024-05-24T11:52:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.044+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 0x558bfecba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.044+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.044+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc6270 0x558bfecd0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.044+0000 7f67389fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfecca100 con 0x558bfecc0480 2024-05-24T11:52:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.044+0000 7f67389fcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfeccaad0 con 0x558bfecc6270 2024-05-24T11:52:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.044+0000 7f67389fcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfecc02e0 con 0x558bfecc5950 2024-05-24T11:52:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6725d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6725d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38876/0 (socket says 172.21.15.39:38876) 2024-05-24T11:52:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6725d9b700 1 -- 172.21.15.39:0/3867758198 learned_addr learned my addr 172.21.15.39:0/3867758198 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6726d9d700 1 --2- 172.21.15.39:0/3867758198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc6270 0x558bfecd0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f672659c700 1 --2- 172.21.15.39:0/3867758198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 0x558bfecba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6725d9b700 1 -- 172.21.15.39:0/3867758198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc6270 msgr2=0x558bfecd0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6725d9b700 1 --2- 172.21.15.39:0/3867758198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc6270 0x558bfecd0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6725d9b700 1 -- 172.21.15.39:0/3867758198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 msgr2=0x558bfecba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6725d9b700 1 --2- 172.21.15.39:0/3867758198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 0x558bfecba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6725d9b700 1 -- 172.21.15.39:0/3867758198 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bfebd6500 con 0x558bfecc5950 2024-05-24T11:52:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f6725d9b700 1 --2- 172.21.15.39:0/3867758198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccbd10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f671c0058b0 tx=0x7f671c005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a69a8f113fe74ff server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f67157fa700 1 -- 172.21.15.39:0/3867758198 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c09aab0 con 0x558bfecc5950 2024-05-24T11:52:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f67157fa700 1 -- 172.21.15.39:0/3867758198 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f671c099fa0 con 0x558bfecc5950 2024-05-24T11:52:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.045+0000 7f67157fa700 1 -- 172.21.15.39:0/3867758198 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0b3150 con 0x558bfecc5950 2024-05-24T11:52:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.046+0000 7f67389fcb80 1 -- 172.21.15.39:0/3867758198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 msgr2=0x558bfeccbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.046+0000 7f67389fcb80 1 --2- 172.21.15.39:0/3867758198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccbd10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f671c0058b0 tx=0x7f671c005960 comp rx=0 tx=0).stop 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.046+0000 7f67389fcb80 1 -- 172.21.15.39:0/3867758198 shutdown_connections 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.046+0000 7f67389fcb80 1 --2- 172.21.15.39:0/3867758198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc6270 0x558bfecd0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.046+0000 7f67389fcb80 1 --2- 172.21.15.39:0/3867758198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 0x558bfecba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.046+0000 7f67389fcb80 1 --2- 172.21.15.39:0/3867758198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccbd10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.046+0000 7f67389fcb80 1 -- 172.21.15.39:0/3867758198 >> 172.21.15.39:0/3867758198 conn(0x558bfec1e1b0 msgr2=0x558bfecc8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 -- 172.21.15.39:0/3867758198 shutdown_connections 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 -- 172.21.15.39:0/3867758198 wait complete. 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 -- start start 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 Processor -- start 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 0x558bfeccec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc6270 0x558bfeccf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfed2ecb0 con 0x558bfecc0480 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfed2edf0 con 0x558bfecc6270 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bfed2ef30 con 0x558bfecc5950 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.047+0000 7f67389fcb80 10 client.0 _ll_register_callbacks cb 0x558bfebb5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f6725d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f6725d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:38890/0 (socket says 172.21.15.39:38890) 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f6725d9b700 1 -- 172.21.15.39:0/226463217 learned_addr learned my addr 172.21.15.39:0/226463217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f672659c700 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 0x558bfeccec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f672659c700 1 -- 172.21.15.39:0/226463217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 msgr2=0x558bfeccf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f672659c700 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f672659c700 1 -- 172.21.15.39:0/226463217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc6270 msgr2=0x558bfeccf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f672659c700 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc6270 0x558bfeccf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f672659c700 1 -- 172.21.15.39:0/226463217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bfea3db00 con 0x558bfecc0480 2024-05-24T11:52:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f672659c700 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 0x558bfeccec20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x558bfecd9b90 tx=0x7f67180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f35a30077ebc7e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.048+0000 7f670dffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:52:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.049+0000 7f670dffb700 1 -- 172.21.15.39:0/226463217 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718025040 con 0x558bfecc0480 2024-05-24T11:52:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.049+0000 7f670dffb700 1 -- 172.21.15.39:0/226463217 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6718018070 con 0x558bfecc0480 2024-05-24T11:52:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.049+0000 7f67389fcb80 1 -- 172.21.15.39:0/226463217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f672000f040 con 0x558bfecc0480 2024-05-24T11:52:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.049+0000 7f670dffb700 1 -- 172.21.15.39:0/226463217 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67180127f0 con 0x558bfecc0480 2024-05-24T11:52:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.049+0000 7f67389fcb80 10 client.16365 fetch_fsmap learned FSMap version 36 2024-05-24T11:52:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.049+0000 7f67389fcb80 1 -- 172.21.15.39:0/226463217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558bfea95c70 con 0x558bfecc0480 2024-05-24T11:52:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.050+0000 7f670dffb700 1 -- 172.21.15.39:0/226463217 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f672000f040 con 0x558bfecc0480 2024-05-24T11:52:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.050+0000 7f670dffb700 1 -- 172.21.15.39:0/226463217 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 36) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f671803ca40 con 0x558bfecc0480 2024-05-24T11:52:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.050+0000 7f67389fcb80 10 client.16365 fetch_fsmap finished waiting for FSMap version 36 2024-05-24T11:52:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.050+0000 7f67389fcb80 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:52:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.050+0000 7f67389fcb80 20 client.16365 populate_metadata read hostname 'smithi039' 2024-05-24T11:52:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.050+0000 7f67389fcb80 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558bfed30680 0x558bfed32b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.050+0000 7f67389fcb80 4 client.16365 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:52:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.050+0000 7f67389fcb80 1 -- 172.21.15.39:0/226463217 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558bfec08e60 con 0x558bfed30680 2024-05-24T11:52:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.052+0000 7f6725d9b700 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558bfed30680 0x558bfed32b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.053+0000 7f6725d9b700 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558bfed30680 0x558bfed32b40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d9326867941af05 server_cookie=95d7145d5f9a20c5 in_seq=0 out_seq=0 2024-05-24T11:52:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.053+0000 7f670dffb700 10 client.16365 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.091+0000 7f670dffb700 1 -- 172.21.15.39:0/226463217 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558bfec08e60 con 0x558bfed30680 2024-05-24T11:52:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.091+0000 7f670dffb700 10 client.16365 handle_command_reply: tid=0 2024-05-24T11:52:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.091+0000 7f67389fcb80 1 client.16365 shutdown 2024-05-24T11:52:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.094+0000 7f67389fcb80 1 -- 172.21.15.39:0/226463217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 msgr2=0x558bfeccec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.094+0000 7f67389fcb80 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 0x558bfeccec20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x558bfecd9b90 tx=0x7f67180047c0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.103+0000 7f67389fcb80 1 -- 172.21.15.39:0/226463217 shutdown_connections 2024-05-24T11:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.103+0000 7f67389fcb80 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x558bfed30680 0x558bfed32b40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.103+0000 7f67389fcb80 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558bfecc6270 0x558bfeccf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.103+0000 7f67389fcb80 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bfecc0480 0x558bfeccec20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.103+0000 7f67389fcb80 1 --2- 172.21.15.39:0/226463217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558bfecc5950 0x558bfeccf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.103+0000 7f67389fcb80 1 -- 172.21.15.39:0/226463217 >> 172.21.15.39:0/226463217 conn(0x558bfec1e1b0 msgr2=0x558bfecc3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.103+0000 7f670dffb700 0 client.16365 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.103+0000 7f67389fcb80 1 -- 172.21.15.39:0/226463217 shutdown_connections 2024-05-24T11:52:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.103+0000 7f67389fcb80 1 -- 172.21.15.39:0/226463217 wait complete. 2024-05-24T11:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.104+0000 7f67389fcb80 20 client.16365 trim_cache size 0 max 16384 2024-05-24T11:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:52:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:52:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 -- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f67100a1310 msgr2=0x7f67100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f67100a1310 0x7f67100a37d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6718005fd0 tx=0x7f6718009310 comp rx=0 tx=0).stop 2024-05-24T11:52:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 -- 172.21.15.39:0/150115883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6728100760 msgr2=0x7f6728108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6728100760 0x7f6728108cc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f671c00fa00 tx=0x7f671c00bec0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 -- 172.21.15.39:0/150115883 shutdown_connections 2024-05-24T11:52:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f67100a1310 0x7f67100a37d0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6718005fd0 tx=0x7f6718009310 comp rx=0 tx=0).stop 2024-05-24T11:52:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6728105e60 0x7f6728072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67280ffda0 0x7f672810cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 --2- 172.21.15.39:0/150115883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6728100760 0x7f6728108cc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.110+0000 7f672c841700 1 -- 172.21.15.39:0/150115883 >> 172.21.15.39:0/150115883 conn(0x7f6728076f10 msgr2=0x7f67280fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.115+0000 7f672c841700 1 -- 172.21.15.39:0/150115883 shutdown_connections 2024-05-24T11:52:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:38.115+0000 7f672c841700 1 -- 172.21.15.39:0/150115883 wait complete. 2024-05-24T11:52:38.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:38 smithi053 ceph-mon[47448]: pgmap v456: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:52:38.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:38 smithi137 ceph-mon[47907]: pgmap v456: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:52:38.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:38 smithi039 ceph-mon[42919]: pgmap v456: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-05-24T11:52:39.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 795.0 s, 750.8 tps, lat 131.920 ms stddev 46.233 2024-05-24T11:52:40.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:40 smithi053 ceph-mon[47448]: pgmap v457: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-05-24T11:52:40.497 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:40 smithi137 ceph-mon[47907]: pgmap v457: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-05-24T11:52:40.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:40 smithi039 ceph-mon[42919]: pgmap v457: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-05-24T11:52:42.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:42 smithi053 ceph-mon[47448]: pgmap v458: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2024-05-24T11:52:42.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:42 smithi137 ceph-mon[47907]: pgmap v458: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2024-05-24T11:52:42.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:42 smithi039 ceph-mon[42919]: pgmap v458: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2024-05-24T11:52:44.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 800.0 s, 965.2 tps, lat 104.786 ms stddev 38.679 2024-05-24T11:52:44.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:44 smithi053 ceph-mon[47448]: pgmap v459: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-05-24T11:52:44.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:44 smithi137 ceph-mon[47907]: pgmap v459: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-05-24T11:52:44.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:44 smithi039 ceph-mon[42919]: pgmap v459: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-05-24T11:52:46.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:46 smithi053 ceph-mon[47448]: pgmap v460: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:52:46.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:46 smithi137 ceph-mon[47907]: pgmap v460: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:52:46.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:46 smithi039 ceph-mon[42919]: pgmap v460: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-24T11:52:48.489 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:48 smithi053 ceph-mon[47448]: pgmap v461: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-05-24T11:52:48.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:48 smithi137 ceph-mon[47907]: pgmap v461: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-05-24T11:52:48.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:48 smithi039 ceph-mon[42919]: pgmap v461: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2024-05-24T11:52:49.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 805.0 s, 920.2 tps, lat 106.374 ms stddev 63.522 2024-05-24T11:52:50.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:50 smithi053 ceph-mon[47448]: pgmap v462: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2024-05-24T11:52:50.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:50 smithi137 ceph-mon[47907]: pgmap v462: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2024-05-24T11:52:50.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:50 smithi039 ceph-mon[42919]: pgmap v462: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2024-05-24T11:52:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.367+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2961253326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cd4107500 msgr2=0x7f5cd4107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.367+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2961253326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cd4107500 0x7f5cd4107980 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009920 tx=0x7f5cc8009c30 comp rx=0 tx=0).stop 2024-05-24T11:52:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.367+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2961253326 shutdown_connections 2024-05-24T11:52:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.367+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2961253326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cd410d5c0 0x7f5cd410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.367+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2961253326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cd4107ec0 0x7f5cd4114380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.367+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2961253326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cd4107500 0x7f5cd4107980 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.367+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2961253326 >> 172.21.15.39:0/2961253326 conn(0x7f5cd4076dc0 msgr2=0x7f5cd4105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.368+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2961253326 shutdown_connections 2024-05-24T11:52:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.370+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2961253326 wait complete. 2024-05-24T11:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.371+0000 7f5cdb67b700 1 Processor -- start 2024-05-24T11:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.371+0000 7f5cdb67b700 1 -- start start 2024-05-24T11:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cdb67b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cd4107500 0x7f5cd4072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cdb67b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cd4107ec0 0x7f5cd4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cdb67b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cd410d5c0 0x7f5cd406b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cdb67b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406c090 con 0x7f5cd410d5c0 2024-05-24T11:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cdb67b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406c200 con 0x7f5cd4107ec0 2024-05-24T11:52:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cdb67b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406c3c0 con 0x7f5cd4107500 2024-05-24T11:52:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cd9c18700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cd410d5c0 0x7f5cd406b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cd9c18700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cd410d5c0 0x7f5cd406b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51214/0 (socket says 172.21.15.39:51214) 2024-05-24T11:52:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cd9c18700 1 -- 172.21.15.39:0/2236061070 learned_addr learned my addr 172.21.15.39:0/2236061070 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cd9c18700 1 -- 172.21.15.39:0/2236061070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cd4107500 msgr2=0x7f5cd4072660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:52:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cd4107500 0x7f5cd4072660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cd9c18700 1 -- 172.21.15.39:0/2236061070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cd4107ec0 msgr2=0x7f5cd4072be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-24T11:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cd4107ec0 0x7f5cd4072be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.372+0000 7f5cd9c18700 1 -- 172.21.15.39:0/2236061070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc009c40 con 0x7f5cd410d5c0 2024-05-24T11:52:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.373+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cd410d5c0 0x7f5cd406b8c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e760 tx=0x7f5cd000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.373+0000 7f5cc67fc700 1 -- 172.21.15.39:0/2236061070 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0004220 con 0x7f5cd410d5c0 2024-05-24T11:52:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.373+0000 7f5cc67fc700 1 -- 172.21.15.39:0/2236061070 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cd0004380 con 0x7f5cd410d5c0 2024-05-24T11:52:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.373+0000 7f5cc67fc700 1 -- 172.21.15.39:0/2236061070 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0003b40 con 0x7f5cd410d5c0 2024-05-24T11:52:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.374+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2236061070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc8009520 con 0x7f5cd410d5c0 2024-05-24T11:52:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.374+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2236061070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd4070490 con 0x7f5cd410d5c0 2024-05-24T11:52:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.378+0000 7f5cc67fc700 1 -- 172.21.15.39:0/2236061070 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5cd0003ca0 con 0x7f5cd410d5c0 2024-05-24T11:52:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.379+0000 7f5cc67fc700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5cc00a1310 0x7f5cc00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.379+0000 7f5cc67fc700 1 -- 172.21.15.39:0/2236061070 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5cd00ccc60 con 0x7f5cd410d5c0 2024-05-24T11:52:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.379+0000 7f5cd9417700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5cc00a1310 0x7f5cc00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.380+0000 7f5cd9417700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5cc00a1310 0x7f5cc00a37d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00b480 tx=0x7f5ccc00d040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.381+0000 7f5ce7836b80 1 Processor -- start 2024-05-24T11:52:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.381+0000 7f5ce7836b80 1 -- start start 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06089d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de06083f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de060848a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de05fdb0f0 con 0x55de06089950 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de05fd8ab0 con 0x55de06084480 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de05fd43a0 con 0x55de0608a270 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd9417700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06089d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd9417700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06089d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51222/0 (socket says 172.21.15.39:51222) 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd9417700 1 -- 172.21.15.39:0/3807065569 learned_addr learned my addr 172.21.15.39:0/3807065569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/3807065569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de060848a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd8c16700 1 --2- 172.21.15.39:0/3807065569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de06083f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd9417700 1 -- 172.21.15.39:0/3807065569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 msgr2=0x55de06083f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd9417700 1 --2- 172.21.15.39:0/3807065569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de06083f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd9417700 1 -- 172.21.15.39:0/3807065569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 msgr2=0x55de060848a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd9417700 1 --2- 172.21.15.39:0/3807065569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de060848a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.382+0000 7f5cd9417700 1 -- 172.21.15.39:0/3807065569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de05e55350 con 0x55de06089950 2024-05-24T11:52:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.383+0000 7f5cd9417700 1 --2- 172.21.15.39:0/3807065569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06089d30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc016730 tx=0x7f5ccc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8190ffeb26e18b60 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.383+0000 7f5cbf7fe700 1 -- 172.21.15.39:0/3807065569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc024070 con 0x55de06089950 2024-05-24T11:52:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.383+0000 7f5cbf7fe700 1 -- 172.21.15.39:0/3807065569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ccc01c040 con 0x55de06089950 2024-05-24T11:52:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.383+0000 7f5cbf7fe700 1 -- 172.21.15.39:0/3807065569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc010040 con 0x55de06089950 2024-05-24T11:52:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.384+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3807065569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 msgr2=0x55de06089d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.384+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/3807065569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06089d30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc016730 tx=0x7f5ccc004ad0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.384+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3807065569 shutdown_connections 2024-05-24T11:52:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.384+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/3807065569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de060848a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.384+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/3807065569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06089d30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.384+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/3807065569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de06083f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.384+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3807065569 >> 172.21.15.39:0/3807065569 conn(0x55de05fe21b0 msgr2=0x55de0607e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.384+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3807065569 shutdown_connections 2024-05-24T11:52:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.384+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3807065569 wait complete. 2024-05-24T11:52:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.385+0000 7f5ce7836b80 1 -- start start 2024-05-24T11:52:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.385+0000 7f5ce7836b80 1 Processor -- start 2024-05-24T11:52:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.385+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de0608d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.385+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de0608db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.385+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de0608e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.385+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de05f70500 con 0x55de06089950 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.385+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de05f701a0 con 0x55de06084480 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.385+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de05e57640 con 0x55de0608a270 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5ce7836b80 10 client.0 _ll_register_callbacks cb 0x55de05f79ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9417700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de0608d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9c18700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de0608e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9417700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de0608d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:55140/0 (socket says 172.21.15.39:55140) 2024-05-24T11:52:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9417700 1 -- 172.21.15.39:0/1272653040 learned_addr learned my addr 172.21.15.39:0/1272653040 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd8c16700 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de0608db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9417700 1 -- 172.21.15.39:0/1272653040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 msgr2=0x55de0608e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9417700 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de0608e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9417700 1 -- 172.21.15.39:0/1272653040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 msgr2=0x55de0608db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9417700 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de0608db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9417700 1 -- 172.21.15.39:0/1272653040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc016410 con 0x55de06084480 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de0608e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cd9417700 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de0608d5f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc003c30 tx=0x7f5ccc003c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2b149aa4016188 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.386+0000 7f5cbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5cbcff9700 1 -- 172.21.15.39:0/1272653040 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc024070 con 0x55de06084480 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5cbcff9700 1 -- 172.21.15.39:0/1272653040 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5ccc004000 con 0x55de06084480 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5ce7836b80 1 -- 172.21.15.39:0/1272653040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de05f2d210 con 0x55de06084480 2024-05-24T11:52:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5cbcff9700 1 -- 172.21.15.39:0/1272653040 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ccc009560 con 0x55de06084480 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5ce7836b80 10 client.26129 fetch_fsmap learned FSMap version 36 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5ce7836b80 1 -- 172.21.15.39:0/1272653040 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55de05e01b00 con 0x55de06084480 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5cbcff9700 1 -- 172.21.15.39:0/1272653040 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de05f2d210 con 0x55de06084480 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5cbcff9700 1 -- 172.21.15.39:0/1272653040 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5ccc01f070 con 0x55de06084480 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5ce7836b80 10 client.26129 fetch_fsmap finished waiting for FSMap version 36 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5ce7836b80 10 client.26129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5ce7836b80 20 client.26129 populate_metadata read hostname 'smithi039' 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55de0609dab0 0x55de0609fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5ce7836b80 4 client.26129 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.387+0000 7f5ce7836b80 1 -- 172.21.15.39:0/1272653040 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de060537b0 con 0x55de0609dab0 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.388+0000 7f5cd8c16700 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55de0609dab0 0x55de0609fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.388+0000 7f5cbcff9700 10 client.26129 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.388+0000 7f5cd8c16700 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55de0609dab0 0x55de0609fed0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ce9ed561017e22 server_cookie=512f263b183297ed in_seq=0 out_seq=0 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.389+0000 7f5cbcff9700 1 -- 172.21.15.39:0/1272653040 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de060537b0 con 0x55de0609dab0 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.389+0000 7f5cbcff9700 10 client.26129 handle_command_reply: tid=0 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.389+0000 7f5ce7836b80 1 client.26129 shutdown 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 -- 172.21.15.39:0/1272653040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 msgr2=0x55de0608d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de0608d5f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc003c30 tx=0x7f5ccc003c60 comp rx=0 tx=0).stop 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 -- 172.21.15.39:0/1272653040 shutdown_connections 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de0608e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55de0609dab0 0x55de0609fed0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de0608d5f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/1272653040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de0608db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 -- 172.21.15.39:0/1272653040 >> 172.21.15.39:0/1272653040 conn(0x55de05fe21b0 msgr2=0x55de06082f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5cbcff9700 0 client.26129 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 -- 172.21.15.39:0/1272653040 shutdown_connections 2024-05-24T11:52:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.390+0000 7f5ce7836b80 1 -- 172.21.15.39:0/1272653040 wait complete. 2024-05-24T11:52:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.391+0000 7f5ce7836b80 20 client.26129 trim_cache size 0 max 16384 2024-05-24T11:52:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5ce7836b80 1 Processor -- start 2024-05-24T11:52:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5ce7836b80 1 -- start start 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de06084480 0x55de0607e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06094c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de0608a270 0x55de06095170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0608b660 con 0x55de06089950 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0607b9e0 con 0x55de0608a270 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0609a7a0 con 0x55de06084480 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5cd9c18700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de0608a270 0x55de06095170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5cd8c16700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06094c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5cd9c18700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de0608a270 0x55de06095170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:55150/0 (socket says 172.21.15.39:55150) 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5cd9c18700 1 -- 172.21.15.39:0/3764998407 learned_addr learned my addr 172.21.15.39:0/3764998407 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.411+0000 7f5cd8c16700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06094c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51242/0 (socket says 172.21.15.39:51242) 2024-05-24T11:52:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.412+0000 7f5cd9c18700 1 -- 172.21.15.39:0/3764998407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de06084480 msgr2=0x55de0607e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.412+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/3764998407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de06084480 0x55de0607e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.412+0000 7f5cd9c18700 1 -- 172.21.15.39:0/3764998407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 msgr2=0x55de06094c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.412+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/3764998407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06094c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.412+0000 7f5cd9c18700 1 -- 172.21.15.39:0/3764998407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de05f9a500 con 0x55de0608a270 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.412+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/3764998407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de0608a270 0x55de06095170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0096280 tx=0x7f5cd00999e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc407e0c0b5ed64 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.412+0000 7f5cbffff700 1 -- 172.21.15.39:0/3764998407 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0022350 con 0x55de0608a270 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.412+0000 7f5cbffff700 1 -- 172.21.15.39:0/3764998407 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cd00044f0 con 0x55de0608a270 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.412+0000 7f5cbffff700 1 -- 172.21.15.39:0/3764998407 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd009a240 con 0x55de0608a270 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.413+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3764998407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de0608a270 msgr2=0x55de06095170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.413+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/3764998407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de0608a270 0x55de06095170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0096280 tx=0x7f5cd00999e0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.413+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3764998407 shutdown_connections 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.413+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/3764998407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de06084480 0x55de0607e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.413+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/3764998407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de0608a270 0x55de06095170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.413+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/3764998407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06094c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.413+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3764998407 >> 172.21.15.39:0/3764998407 conn(0x55de05fe21b0 msgr2=0x55de0608bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.413+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3764998407 shutdown_connections 2024-05-24T11:52:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.413+0000 7f5ce7836b80 1 -- 172.21.15.39:0/3764998407 wait complete. 2024-05-24T11:52:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.414+0000 7f5ce7836b80 1 -- start start 2024-05-24T11:52:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.414+0000 7f5ce7836b80 1 Processor -- start 2024-05-24T11:52:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.414+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de06092c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.414+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06093160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.414+0000 7f5ce7836b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de060936a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.414+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de060f2fb0 con 0x55de06089950 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.414+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de060f30f0 con 0x55de06084480 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.414+0000 7f5ce7836b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de060f3230 con 0x55de0608a270 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.414+0000 7f5ce7836b80 10 client.0 _ll_register_callbacks cb 0x7f5cd00960c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9c18700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de060936a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9c18700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de060936a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:57474/0 (socket says 172.21.15.39:57474) 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9c18700 1 -- 172.21.15.39:0/2256880796 learned_addr learned my addr 172.21.15.39:0/2256880796 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9417700 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de06092c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9c18700 1 -- 172.21.15.39:0/2256880796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 msgr2=0x55de06092c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de06092c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9c18700 1 -- 172.21.15.39:0/2256880796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 msgr2=0x55de06093160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:52:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06093160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9c18700 1 -- 172.21.15.39:0/2256880796 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd4070490 con 0x55de0608a270 2024-05-24T11:52:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cd9c18700 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de060936a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00980d0 tx=0x7f5cd0099c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44404ab320d07d59 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:52:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:52:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.415+0000 7f5cbcff9700 1 -- 172.21.15.39:0/2256880796 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00b8170 con 0x55de0608a270 2024-05-24T11:52:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.416+0000 7f5cbcff9700 1 -- 172.21.15.39:0/2256880796 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5cd009a480 con 0x55de0608a270 2024-05-24T11:52:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.416+0000 7f5ce7836b80 1 -- 172.21.15.39:0/2256880796 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ccc01c040 con 0x55de0608a270 2024-05-24T11:52:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.416+0000 7f5cbcff9700 1 -- 172.21.15.39:0/2256880796 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cd0022350 con 0x55de0608a270 2024-05-24T11:52:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.419+0000 7f5ce7836b80 10 client.26050 fetch_fsmap learned FSMap version 36 2024-05-24T11:52:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.419+0000 7f5ce7836b80 1 -- 172.21.15.39:0/2256880796 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55de05e59c70 con 0x55de0608a270 2024-05-24T11:52:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.419+0000 7f5cbcff9700 1 -- 172.21.15.39:0/2256880796 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cd00a7070 con 0x55de0608a270 2024-05-24T11:52:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.419+0000 7f5cbcff9700 1 -- 172.21.15.39:0/2256880796 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 36) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5cd000c820 con 0x55de0608a270 2024-05-24T11:52:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.420+0000 7f5ce7836b80 10 client.26050 fetch_fsmap finished waiting for FSMap version 36 2024-05-24T11:52:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.420+0000 7f5ce7836b80 10 client.26050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:52:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.420+0000 7f5ce7836b80 20 client.26050 populate_metadata read hostname 'smithi039' 2024-05-24T11:52:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.420+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55de060f4650 0x55de060f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:52:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.420+0000 7f5ce7836b80 4 client.26050 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:52:51.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.420+0000 7f5ce7836b80 1 -- 172.21.15.39:0/2256880796 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de05fcce60 con 0x55de060f4650 2024-05-24T11:52:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.420+0000 7f5cd9417700 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55de060f4650 0x55de060f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:52:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.421+0000 7f5cbcff9700 10 client.26050 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.421+0000 7f5cd9417700 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55de060f4650 0x55de060f6a70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cbcf2f6fef2ff87 server_cookie=f19b4165495ba1ce in_seq=0 out_seq=0 2024-05-24T11:52:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.473+0000 7f5cbcff9700 1 -- 172.21.15.39:0/2256880796 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de05fcce60 con 0x55de060f4650 2024-05-24T11:52:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.473+0000 7f5cbcff9700 10 client.26050 handle_command_reply: tid=0 2024-05-24T11:52:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.473+0000 7f5ce7836b80 1 client.26050 shutdown 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 -- 172.21.15.39:0/2256880796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 msgr2=0x55de060936a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de060936a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00980d0 tx=0x7f5cd0099c00 comp rx=0 tx=0).stop 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 -- 172.21.15.39:0/2256880796 shutdown_connections 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55de0608a270 0x55de060936a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55de060f4650 0x55de060f6a70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55de06084480 0x55de06092c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5cbcff9700 0 client.26050 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 --2- 172.21.15.39:0/2256880796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de06089950 0x55de06093160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 -- 172.21.15.39:0/2256880796 >> 172.21.15.39:0/2256880796 conn(0x55de05fe21b0 msgr2=0x55de06087380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 -- 172.21.15.39:0/2256880796 shutdown_connections 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 1 -- 172.21.15.39:0/2256880796 wait complete. 2024-05-24T11:52:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.474+0000 7f5ce7836b80 20 client.26050 trim_cache size 0 max 16384 2024-05-24T11:52:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:52:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:52:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:52:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:52:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5cc00a1310 msgr2=0x7f5cc00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5cc00a1310 0x7f5cc00a37d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00b480 tx=0x7f5ccc00d040 comp rx=0 tx=0).stop 2024-05-24T11:52:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cd410d5c0 msgr2=0x7f5cd406b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:52:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cd410d5c0 0x7f5cd406b8c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e760 tx=0x7f5cd000eb20 comp rx=0 tx=0).stop 2024-05-24T11:52:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2236061070 shutdown_connections 2024-05-24T11:52:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cd4107500 0x7f5cd4072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f5cc00a1310 0x7f5cc00a37d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cd410d5c0 0x7f5cd406b8c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 --2- 172.21.15.39:0/2236061070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cd4107ec0 0x7f5cd4072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:52:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2236061070 >> 172.21.15.39:0/2236061070 conn(0x7f5cd4076dc0 msgr2=0x7f5cd4078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:52:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2236061070 shutdown_connections 2024-05-24T11:52:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:52:51.475+0000 7f5cdb67b700 1 -- 172.21.15.39:0/2236061070 wait complete. 2024-05-24T11:52:52.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:52 smithi053 ceph-mon[47448]: pgmap v463: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-05-24T11:52:52.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:52 smithi137 ceph-mon[47907]: pgmap v463: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-05-24T11:52:52.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:52 smithi039 ceph-mon[42919]: pgmap v463: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-05-24T11:52:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 810.0 s, 1104.6 tps, lat 92.599 ms stddev 61.639 2024-05-24T11:52:54.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:54 smithi053 ceph-mon[47448]: pgmap v464: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 14 MiB/s wr, 1.20k op/s 2024-05-24T11:52:54.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:54 smithi137 ceph-mon[47907]: pgmap v464: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 14 MiB/s wr, 1.20k op/s 2024-05-24T11:52:54.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:54 smithi039 ceph-mon[42919]: pgmap v464: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 14 MiB/s wr, 1.20k op/s 2024-05-24T11:52:56.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:56 smithi053 ceph-mon[47448]: pgmap v465: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2024-05-24T11:52:56.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:56 smithi137 ceph-mon[47907]: pgmap v465: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2024-05-24T11:52:56.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:56 smithi039 ceph-mon[42919]: pgmap v465: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2024-05-24T11:52:58.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:52:58 smithi053 ceph-mon[47448]: pgmap v466: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-05-24T11:52:58.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:52:58 smithi137 ceph-mon[47907]: pgmap v466: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-05-24T11:52:58.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:52:58 smithi039 ceph-mon[42919]: pgmap v466: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-05-24T11:52:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 815.0 s, 1077.9 tps, lat 92.876 ms stddev 37.827 2024-05-24T11:53:00.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:00 smithi053 ceph-mon[47448]: pgmap v467: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2024-05-24T11:53:00.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:00 smithi137 ceph-mon[47907]: pgmap v467: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2024-05-24T11:53:00.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:00 smithi039 ceph-mon[42919]: pgmap v467: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2024-05-24T11:53:02.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:02 smithi137 ceph-mon[47907]: pgmap v468: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-05-24T11:53:02.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:02 smithi039 ceph-mon[42919]: pgmap v468: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-05-24T11:53:02.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:02 smithi053 ceph-mon[47448]: pgmap v468: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-05-24T11:53:04.114 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 820.0 s, 1007.4 tps, lat 99.257 ms stddev 40.043 2024-05-24T11:53:04.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:04 smithi039 ceph-mon[42919]: pgmap v469: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 13 MiB/s wr, 964 op/s 2024-05-24T11:53:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.671+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/3745880281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf480ff410 msgr2=0x7fbf480ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.671+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/3745880281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf480ff410 0x7fbf480ff890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbf300098f0 tx=0x7fbf30009c00 comp rx=0 tx=0).stop 2024-05-24T11:53:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.677+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/3745880281 shutdown_connections 2024-05-24T11:53:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.677+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/3745880281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf480ffdd0 0x7fbf481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.677+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/3745880281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf48105dd0 0x7fbf481061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.677+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/3745880281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf480ff410 0x7fbf480ff890 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.677+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/3745880281 >> 172.21.15.39:0/3745880281 conn(0x7fbf480763a0 msgr2=0x7fbf480fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.677+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/3745880281 shutdown_connections 2024-05-24T11:53:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.677+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/3745880281 wait complete. 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.678+0000 7fbf4d2bf700 1 Processor -- start 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.678+0000 7fbf4d2bf700 1 -- start start 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.678+0000 7fbf4d2bf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf480ff410 0x7fbf48072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.678+0000 7fbf4d2bf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf480ffdd0 0x7fbf48072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.678+0000 7fbf4d2bf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf48105dd0 0x7fbf4806b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.678+0000 7fbf4d2bf700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4806c0f0 con 0x7fbf48105dd0 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.678+0000 7fbf4d2bf700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4806c260 con 0x7fbf480ffdd0 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.678+0000 7fbf4d2bf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4806c420 con 0x7fbf480ff410 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf477fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf48105dd0 0x7fbf4806b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf477fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf48105dd0 0x7fbf4806b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58468/0 (socket says 172.21.15.39:58468) 2024-05-24T11:53:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf477fe700 1 -- 172.21.15.39:0/2352342152 learned_addr learned my addr 172.21.15.39:0/2352342152 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf46ffd700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf480ff410 0x7fbf48072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf467fc700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf480ffdd0 0x7fbf48072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf467fc700 1 -- 172.21.15.39:0/2352342152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf480ff410 msgr2=0x7fbf48072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf467fc700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf480ff410 0x7fbf48072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf467fc700 1 -- 172.21.15.39:0/2352342152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf48105dd0 msgr2=0x7fbf4806b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf467fc700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf48105dd0 0x7fbf4806b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf467fc700 1 -- 172.21.15.39:0/2352342152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40009d00 con 0x7fbf480ffdd0 2024-05-24T11:53:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.679+0000 7fbf467fc700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf480ffdd0 0x7fbf48072bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009230 tx=0x7fbf3000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.680+0000 7fbf3ffff700 1 -- 172.21.15.39:0/2352342152 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3001c070 con 0x7fbf480ffdd0 2024-05-24T11:53:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.680+0000 7fbf3ffff700 1 -- 172.21.15.39:0/2352342152 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf3000b9f0 con 0x7fbf480ffdd0 2024-05-24T11:53:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.680+0000 7fbf3ffff700 1 -- 172.21.15.39:0/2352342152 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30003680 con 0x7fbf480ffdd0 2024-05-24T11:53:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.680+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/2352342152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf30009520 con 0x7fbf480ffdd0 2024-05-24T11:53:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.680+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/2352342152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf480703e0 con 0x7fbf480ffdd0 2024-05-24T11:53:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.684+0000 7fbf5947ab80 1 Processor -- start 2024-05-24T11:53:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.684+0000 7fbf3ffff700 1 -- 172.21.15.39:0/2352342152 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbf30020030 con 0x7fbf480ffdd0 2024-05-24T11:53:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.684+0000 7fbf5947ab80 1 -- start start 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b0ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b04f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a01a5c0f0 con 0x560a01b0a950 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a01a59ab0 con 0x560a01b0b270 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a01a553a0 con 0x560a01b05480 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf46ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b04f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf46ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b04f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49578/0 (socket says 172.21.15.39:49578) 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf46ffd700 1 -- 172.21.15.39:0/1475254070 learned_addr learned my addr 172.21.15.39:0/1475254070 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.685+0000 7fbf477fe700 1 --2- 172.21.15.39:0/1475254070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b0ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf467fc700 1 --2- 172.21.15.39:0/1475254070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf46ffd700 1 -- 172.21.15.39:0/1475254070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 msgr2=0x560a01b058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf46ffd700 1 --2- 172.21.15.39:0/1475254070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf46ffd700 1 -- 172.21.15.39:0/1475254070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 msgr2=0x560a01b0ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf46ffd700 1 --2- 172.21.15.39:0/1475254070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b0ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf46ffd700 1 -- 172.21.15.39:0/1475254070 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a018d6350 con 0x560a01b0b270 2024-05-24T11:53:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf3ffff700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbf280a13d0 0x7fbf280a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf46ffd700 1 --2- 172.21.15.39:0/1475254070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b04f40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009510 tx=0x7fbf400099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9933a9c5b8fc1df server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf477fe700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbf280a13d0 0x7fbf280a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf3ffff700 1 -- 172.21.15.39:0/2352342152 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbf300cdaa0 con 0x7fbf480ffdd0 2024-05-24T11:53:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf3d7fa700 1 -- 172.21.15.39:0/1475254070 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40007e50 con 0x560a01b0b270 2024-05-24T11:53:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf3d7fa700 1 -- 172.21.15.39:0/1475254070 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf4000f040 con 0x560a01b0b270 2024-05-24T11:53:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.686+0000 7fbf3d7fa700 1 -- 172.21.15.39:0/1475254070 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40005280 con 0x560a01b0b270 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.687+0000 7fbf477fe700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbf280a13d0 0x7fbf280a3890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800be40 tx=0x7fbf3800b400 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.687+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1475254070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 msgr2=0x560a01b04f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.687+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1475254070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b04f40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009510 tx=0x7fbf400099e0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.687+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1475254070 shutdown_connections 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.687+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1475254070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.687+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1475254070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b0ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.687+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1475254070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b04f40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.687+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1475254070 >> 172.21.15.39:0/1475254070 conn(0x560a01a631b0 msgr2=0x560a01aff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.687+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1475254070 shutdown_connections 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.688+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1475254070 wait complete. 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.688+0000 7fbf5947ab80 1 -- start start 2024-05-24T11:53:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.688+0000 7fbf5947ab80 1 Processor -- start 2024-05-24T11:53:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.688+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b11250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.688+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b0e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.688+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b0eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.688+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a019f1500 con 0x560a01b0a950 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.688+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a019f11a0 con 0x560a01b0b270 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.688+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a018d8640 con 0x560a01b05480 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf5947ab80 10 client.0 _ll_register_callbacks cb 0x560a019faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf477fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b0eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf477fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b0eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49588/0 (socket says 172.21.15.39:49588) 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf477fe700 1 -- 172.21.15.39:0/1176400712 learned_addr learned my addr 172.21.15.39:0/1176400712 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf477fe700 1 -- 172.21.15.39:0/1176400712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 msgr2=0x560a01b11250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf477fe700 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b11250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf477fe700 1 -- 172.21.15.39:0/1176400712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 msgr2=0x560a01b0e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf477fe700 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b0e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf477fe700 1 -- 172.21.15.39:0/1176400712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf400091c0 con 0x560a01b0b270 2024-05-24T11:53:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf477fe700 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b0eb00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf38011fb0 tx=0x7fbf380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d5efba7d856e2a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.689+0000 7fbf367fc700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf367fc700 1 -- 172.21.15.39:0/1176400712 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38016070 con 0x560a01b0b270 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf367fc700 1 -- 172.21.15.39:0/1176400712 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbf38027410 con 0x560a01b0b270 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1176400712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a019ae210 con 0x560a01b0b270 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf367fc700 1 -- 172.21.15.39:0/1176400712 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf380275d0 con 0x560a01b0b270 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf5947ab80 10 client.26147 fetch_fsmap learned FSMap version 36 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1176400712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560a01882b00 con 0x560a01b0b270 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf367fc700 1 -- 172.21.15.39:0/1176400712 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf38034020 con 0x560a01b0b270 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf367fc700 1 -- 172.21.15.39:0/1176400712 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbf3801c760 con 0x560a01b0b270 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf5947ab80 10 client.26147 fetch_fsmap finished waiting for FSMap version 36 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf5947ab80 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf5947ab80 20 client.26147 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560a01b1ec40 0x560a01b21080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf5947ab80 4 client.26147 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:53:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.690+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1176400712 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a01ad47b0 con 0x560a01b1ec40 2024-05-24T11:53:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.691+0000 7fbf46ffd700 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560a01b1ec40 0x560a01b21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.692+0000 7fbf367fc700 10 client.26147 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.692+0000 7fbf46ffd700 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560a01b1ec40 0x560a01b21080 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=820dc97ca619446f server_cookie=b69a7af8d597bf79 in_seq=0 out_seq=0 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.693+0000 7fbf367fc700 1 -- 172.21.15.39:0/1176400712 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a01ad47b0 con 0x560a01b1ec40 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.693+0000 7fbf367fc700 10 client.26147 handle_command_reply: tid=0 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.693+0000 7fbf5947ab80 1 client.26147 shutdown 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1176400712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 msgr2=0x560a01b0eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b0eb00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf38011fb0 tx=0x7fbf380047c0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1176400712 shutdown_connections 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b11250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560a01b1ec40 0x560a01b21080 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b0eb00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1176400712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b0e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf367fc700 0 client.26147 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1176400712 >> 172.21.15.39:0/1176400712 conn(0x560a01a631b0 msgr2=0x560a01b094a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1176400712 shutdown_connections 2024-05-24T11:53:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1176400712 wait complete. 2024-05-24T11:53:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.694+0000 7fbf5947ab80 20 client.26147 trim_cache size 0 max 16384 2024-05-24T11:53:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf5947ab80 1 Processor -- start 2024-05-24T11:53:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf5947ab80 1 -- start start 2024-05-24T11:53:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b05480 0x560a01aff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b0a950 0x560a01b05860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b10d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a01b05de0 con 0x560a01b05480 2024-05-24T11:53:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a01b262d0 con 0x560a01b0b270 2024-05-24T11:53:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a01b0a390 con 0x560a01b0a950 2024-05-24T11:53:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf477fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b10d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf477fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b10d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:49592/0 (socket says 172.21.15.39:49592) 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf477fe700 1 -- 172.21.15.39:0/765394169 learned_addr learned my addr 172.21.15.39:0/765394169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.719+0000 7fbf467fc700 1 --2- 172.21.15.39:0/765394169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b0a950 0x560a01b05860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.720+0000 7fbf477fe700 1 -- 172.21.15.39:0/765394169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b0a950 msgr2=0x560a01b05860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.720+0000 7fbf477fe700 1 --2- 172.21.15.39:0/765394169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b0a950 0x560a01b05860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.720+0000 7fbf477fe700 1 -- 172.21.15.39:0/765394169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b05480 msgr2=0x560a01aff910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.720+0000 7fbf477fe700 1 --2- 172.21.15.39:0/765394169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b05480 0x560a01aff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.720+0000 7fbf477fe700 1 -- 172.21.15.39:0/765394169 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a01a1b500 con 0x560a01b0b270 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.720+0000 7fbf477fe700 1 --2- 172.21.15.39:0/765394169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b10d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf3801ca90 tx=0x7fbf3801fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4400c6d00e2b84d server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.720+0000 7fbf3dffb700 1 -- 172.21.15.39:0/765394169 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800bba0 con 0x560a01b0b270 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.720+0000 7fbf3dffb700 1 -- 172.21.15.39:0/765394169 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf38034030 con 0x560a01b0b270 2024-05-24T11:53:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.720+0000 7fbf3dffb700 1 -- 172.21.15.39:0/765394169 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38027a80 con 0x560a01b0b270 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.721+0000 7fbf5947ab80 1 -- 172.21.15.39:0/765394169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 msgr2=0x560a01b10d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.721+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/765394169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b10d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf3801ca90 tx=0x7fbf3801fb50 comp rx=0 tx=0).stop 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.721+0000 7fbf5947ab80 1 -- 172.21.15.39:0/765394169 shutdown_connections 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.721+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/765394169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b10d80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.721+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/765394169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b05480 0x560a01aff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.721+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/765394169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b0a950 0x560a01b05860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.721+0000 7fbf5947ab80 1 -- 172.21.15.39:0/765394169 >> 172.21.15.39:0/765394169 conn(0x560a01a631b0 msgr2=0x560a01b0dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.721+0000 7fbf5947ab80 1 -- 172.21.15.39:0/765394169 shutdown_connections 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.721+0000 7fbf5947ab80 1 -- 172.21.15.39:0/765394169 wait complete. 2024-05-24T11:53:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf5947ab80 1 -- start start 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf5947ab80 1 Processor -- start 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b14130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf5947ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a01b741a0 con 0x560a01b0a950 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a01b742e0 con 0x560a01b0b270 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf5947ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a01b74420 con 0x560a01b05480 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf5947ab80 10 client.0 _ll_register_callbacks cb 0x560a019faad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf46ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf46ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b13bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:50580/0 (socket says 172.21.15.39:50580) 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.722+0000 7fbf46ffd700 1 -- 172.21.15.39:0/1971045091 learned_addr learned my addr 172.21.15.39:0/1971045091 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf477fe700 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b14670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf46ffd700 1 -- 172.21.15.39:0/1971045091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 msgr2=0x560a01b14670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf46ffd700 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b14670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf46ffd700 1 -- 172.21.15.39:0/1971045091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 msgr2=0x560a01b14130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf46ffd700 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b14130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf46ffd700 1 -- 172.21.15.39:0/1971045091 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a01882b00 con 0x560a01b05480 2024-05-24T11:53:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf46ffd700 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b13bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x560a01b00790 tx=0x7fbf40009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8d673166af8f5e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf367fc700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:53:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf367fc700 1 -- 172.21.15.39:0/1971045091 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40017670 con 0x560a01b05480 2024-05-24T11:53:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1971045091 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf4000f040 con 0x560a01b05480 2024-05-24T11:53:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf367fc700 1 -- 172.21.15.39:0/1971045091 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbf400177d0 con 0x560a01b05480 2024-05-24T11:53:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf367fc700 1 -- 172.21.15.39:0/1971045091 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf400179b0 con 0x560a01b05480 2024-05-24T11:53:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf5947ab80 10 client.26065 fetch_fsmap learned FSMap version 36 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.723+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1971045091 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560a018dac70 con 0x560a01b05480 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.724+0000 7fbf367fc700 1 -- 172.21.15.39:0/1971045091 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf4000f040 con 0x560a01b05480 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.724+0000 7fbf367fc700 1 -- 172.21.15.39:0/1971045091 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 36) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbf40031070 con 0x560a01b05480 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.724+0000 7fbf5947ab80 10 client.26065 fetch_fsmap finished waiting for FSMap version 36 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.724+0000 7fbf5947ab80 10 client.26065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.724+0000 7fbf5947ab80 20 client.26065 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.724+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560a01b75bd0 0x560a01b77ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.724+0000 7fbf5947ab80 4 client.26065 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.724+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1971045091 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a01a4de60 con 0x560a01b75bd0 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.724+0000 7fbf467fc700 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560a01b75bd0 0x560a01b77ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.725+0000 7fbf467fc700 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560a01b75bd0 0x560a01b77ff0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b79349016ef9184 server_cookie=a869201f79d6218b in_seq=0 out_seq=0 2024-05-24T11:53:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.725+0000 7fbf367fc700 10 client.26065 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:04.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:04 smithi053 ceph-mon[47448]: pgmap v469: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 13 MiB/s wr, 964 op/s 2024-05-24T11:53:04.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:04 smithi137 ceph-mon[47907]: pgmap v469: 97 pgs: 97 active+clean; 5.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 13 MiB/s wr, 964 op/s 2024-05-24T11:53:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.762+0000 7fbf367fc700 1 -- 172.21.15.39:0/1971045091 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a01a4de60 con 0x560a01b75bd0 2024-05-24T11:53:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.762+0000 7fbf367fc700 10 client.26065 handle_command_reply: tid=0 2024-05-24T11:53:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 client.26065 shutdown 2024-05-24T11:53:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1971045091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 msgr2=0x560a01b13bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b13bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x560a01b00790 tx=0x7fbf40009960 comp rx=0 tx=0).stop 2024-05-24T11:53:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1971045091 shutdown_connections 2024-05-24T11:53:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a01b05480 0x560a01b13bf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560a01b75bd0 0x560a01b77ff0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560a01b0b270 0x560a01b14670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 --2- 172.21.15.39:0/1971045091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a01b0a950 0x560a01b14130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1971045091 >> 172.21.15.39:0/1971045091 conn(0x560a01a631b0 msgr2=0x560a01b0cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf367fc700 0 client.26065 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1971045091 shutdown_connections 2024-05-24T11:53:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.764+0000 7fbf5947ab80 1 -- 172.21.15.39:0/1971045091 wait complete. 2024-05-24T11:53:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.766+0000 7fbf5947ab80 20 client.26065 trim_cache size 0 max 16384 2024-05-24T11:53:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:53:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:53:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:53:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:53:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/2352342152 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbf280a13d0 msgr2=0x7fbf280a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbf280a13d0 0x7fbf280a3890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800be40 tx=0x7fbf3800b400 comp rx=0 tx=0).stop 2024-05-24T11:53:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/2352342152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf480ffdd0 msgr2=0x7fbf48072bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf480ffdd0 0x7fbf48072bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009230 tx=0x7fbf3000fa00 comp rx=0 tx=0).stop 2024-05-24T11:53:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/2352342152 shutdown_connections 2024-05-24T11:53:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf480ff410 0x7fbf48072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fbf280a13d0 0x7fbf280a3890 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf48105dd0 0x7fbf4806b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 --2- 172.21.15.39:0/2352342152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf480ffdd0 0x7fbf48072bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.767+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/2352342152 >> 172.21.15.39:0/2352342152 conn(0x7fbf480763a0 msgr2=0x7fbf48101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.768+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/2352342152 shutdown_connections 2024-05-24T11:53:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:04.768+0000 7fbf4d2bf700 1 -- 172.21.15.39:0/2352342152 wait complete. 2024-05-24T11:53:05.744 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:53:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setuser ceph since I am not root 2024-05-24T11:53:05.744 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:53:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setgroup ceph since I am not root 2024-05-24T11:53:05.744 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:53:05 smithi137 ceph-mds[84908]: -- 172.21.15.137:0/854498163 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562caa8ae1a0 con 0x562caa884c00 2024-05-24T11:53:05.744 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:53:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: starting mds.f at 2024-05-24T11:53:06.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:06 smithi039 ceph-mon[42919]: pgmap v470: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-05-24T11:53:06.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:06 smithi039 ceph-mon[42919]: daemon mds.f restarted 2024-05-24T11:53:06.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:06 smithi039 ceph-mon[42919]: osdmap e102: 12 total, 12 up, 12 in 2024-05-24T11:53:06.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:06 smithi053 ceph-mon[47448]: pgmap v470: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-05-24T11:53:06.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:06 smithi053 ceph-mon[47448]: daemon mds.f restarted 2024-05-24T11:53:06.743 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:06 smithi053 ceph-mon[47448]: osdmap e102: 12 total, 12 up, 12 in 2024-05-24T11:53:06.748 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:06 smithi137 ceph-mon[47907]: pgmap v470: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 18 MiB/s wr, 1.47k op/s 2024-05-24T11:53:06.748 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:06 smithi137 ceph-mon[47907]: daemon mds.f restarted 2024-05-24T11:53:06.748 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:06 smithi137 ceph-mon[47907]: osdmap e102: 12 total, 12 up, 12 in 2024-05-24T11:53:07.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:07 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6834/465020726,v1:172.21.15.137:6835/465020726] up:boot 2024-05-24T11:53:07.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:07 smithi039 ceph-mon[42919]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:53:07.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:07 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:53:07.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:07 smithi039 ceph-mon[42919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:53:07.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:07 smithi039 ceph-mon[42919]: osdmap e103: 12 total, 12 up, 12 in 2024-05-24T11:53:07.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:07 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6834/465020726,v1:172.21.15.137:6835/465020726] up:boot 2024-05-24T11:53:07.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:07 smithi053 ceph-mon[47448]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:53:07.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:07 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:53:07.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:07 smithi053 ceph-mon[47448]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:53:07.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:07 smithi053 ceph-mon[47448]: osdmap e103: 12 total, 12 up, 12 in 2024-05-24T11:53:07.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:07 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6834/465020726,v1:172.21.15.137:6835/465020726] up:boot 2024-05-24T11:53:07.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:07 smithi137 ceph-mon[47907]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:53:07.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:07 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:53:07.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:07 smithi137 ceph-mon[47907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:53:07.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:07 smithi137 ceph-mon[47907]: osdmap e103: 12 total, 12 up, 12 in 2024-05-24T11:53:08.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:08 smithi039 ceph-mon[42919]: pgmap v472: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-05-24T11:53:08.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:08 smithi053 ceph-mon[47448]: pgmap v472: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-05-24T11:53:08.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:08 smithi137 ceph-mon[47907]: pgmap v472: 97 pgs: 97 active+clean; 6.0 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-05-24T11:53:09.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 825.0 s, 892.1 tps, lat 112.434 ms stddev 59.440 2024-05-24T11:53:10.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:10 smithi039 ceph-mon[42919]: pgmap v474: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2024-05-24T11:53:10.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:10 smithi053 ceph-mon[47448]: pgmap v474: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2024-05-24T11:53:10.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:10 smithi137 ceph-mon[47907]: pgmap v474: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2024-05-24T11:53:12.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:12 smithi039 ceph-mon[42919]: pgmap v475: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 17 MiB/s wr, 1.45k op/s 2024-05-24T11:53:12.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:12 smithi053 ceph-mon[47448]: pgmap v475: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 17 MiB/s wr, 1.45k op/s 2024-05-24T11:53:12.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:12 smithi137 ceph-mon[47907]: pgmap v475: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 17 MiB/s wr, 1.45k op/s 2024-05-24T11:53:14.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 830.0 s, 1107.2 tps, lat 90.040 ms stddev 47.041 2024-05-24T11:53:14.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:14 smithi039 ceph-mon[42919]: pgmap v476: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 9.1 MiB/s wr, 689 op/s 2024-05-24T11:53:14.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:14 smithi053 ceph-mon[47448]: pgmap v476: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 9.1 MiB/s wr, 689 op/s 2024-05-24T11:53:14.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:14 smithi137 ceph-mon[47907]: pgmap v476: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 9.1 MiB/s wr, 689 op/s 2024-05-24T11:53:16.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:15 smithi039 ceph-mon[42919]: pgmap v477: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-24T11:53:16.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:15 smithi039 ceph-mon[42919]: osdmap e104: 12 total, 12 up, 12 in 2024-05-24T11:53:16.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:15 smithi053 ceph-mon[47448]: pgmap v477: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-24T11:53:16.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:15 smithi053 ceph-mon[47448]: osdmap e104: 12 total, 12 up, 12 in 2024-05-24T11:53:16.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:15 smithi137 ceph-mon[47907]: pgmap v477: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-24T11:53:16.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:15 smithi137 ceph-mon[47907]: osdmap e104: 12 total, 12 up, 12 in 2024-05-24T11:53:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.969+0000 7f489db43700 1 -- 172.21.15.39:0/2947835488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48981091f0 msgr2=0x7f4898114c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.969+0000 7f489db43700 1 --2- 172.21.15.39:0/2947835488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48981091f0 0x7f4898114c70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4890009fc0 tx=0x7f489000b330 comp rx=0 tx=0).stop 2024-05-24T11:53:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.970+0000 7f489db43700 1 -- 172.21.15.39:0/2947835488 shutdown_connections 2024-05-24T11:53:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.970+0000 7f489db43700 1 --2- 172.21.15.39:0/2947835488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f489810deb0 0x7f489810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.970+0000 7f489db43700 1 --2- 172.21.15.39:0/2947835488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48981091f0 0x7f4898114c70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.970+0000 7f489db43700 1 --2- 172.21.15.39:0/2947835488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898108830 0x7f4898108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.970+0000 7f489db43700 1 -- 172.21.15.39:0/2947835488 >> 172.21.15.39:0/2947835488 conn(0x7f489806bf80 msgr2=0x7f489806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.975+0000 7f489db43700 1 -- 172.21.15.39:0/2947835488 shutdown_connections 2024-05-24T11:53:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.975+0000 7f489db43700 1 -- 172.21.15.39:0/2947835488 wait complete. 2024-05-24T11:53:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.975+0000 7f489db43700 1 Processor -- start 2024-05-24T11:53:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f489db43700 1 -- start start 2024-05-24T11:53:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f489db43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4898108830 0x7f4898114b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f489db43700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48981091f0 0x7f4898111c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f489db43700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898112150 0x7f48981125e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f489db43700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48981b2900 con 0x7f4898108830 2024-05-24T11:53:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f489db43700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489811c260 con 0x7f4898112150 2024-05-24T11:53:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f489db43700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489811c420 con 0x7f48981091f0 2024-05-24T11:53:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f488effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48981091f0 0x7f4898111c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f488effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48981091f0 0x7f4898111c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:43798/0 (socket says 172.21.15.39:43798) 2024-05-24T11:53:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.976+0000 7f488effd700 1 -- 172.21.15.39:0/3052383956 learned_addr learned my addr 172.21.15.39:0/3052383956 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.977+0000 7f4897fff700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898112150 0x7f48981125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.977+0000 7f4897fff700 1 -- 172.21.15.39:0/3052383956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48981091f0 msgr2=0x7f4898111c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.977+0000 7f4897fff700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48981091f0 0x7f4898111c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.977+0000 7f4897fff700 1 -- 172.21.15.39:0/3052383956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4898108830 msgr2=0x7f4898114b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.977+0000 7f4897fff700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4898108830 0x7f4898114b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.977+0000 7f4897fff700 1 -- 172.21.15.39:0/3052383956 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4890009c40 con 0x7f4898112150 2024-05-24T11:53:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.977+0000 7f4897fff700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898112150 0x7f48981125e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4890015000 tx=0x7f489000cd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.978+0000 7f48957fa700 1 -- 172.21.15.39:0/3052383956 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000e040 con 0x7f4898112150 2024-05-24T11:53:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.978+0000 7f48957fa700 1 -- 172.21.15.39:0/3052383956 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4890004c60 con 0x7f4898112150 2024-05-24T11:53:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.978+0000 7f48957fa700 1 -- 172.21.15.39:0/3052383956 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900052a0 con 0x7f4898112150 2024-05-24T11:53:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.978+0000 7f489db43700 1 -- 172.21.15.39:0/3052383956 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489811c6a0 con 0x7f4898112150 2024-05-24T11:53:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.978+0000 7f489db43700 1 -- 172.21.15.39:0/3052383956 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489811cb60 con 0x7f4898112150 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.983+0000 7f48a9cfeb80 1 Processor -- start 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.983+0000 7f48a9cfeb80 1 -- start start 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.983+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 0x56074161dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.983+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 0x560741617f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.983+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x5607416188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.983+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074156f0f0 con 0x56074161d950 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.983+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074156cab0 con 0x560741618480 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.983+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607415683a0 con 0x56074161e270 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.985+0000 7f48957fa700 1 -- 172.21.15.39:0/3052383956 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f489001d480 con 0x7f4898112150 2024-05-24T11:53:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.985+0000 7f48957fa700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48840a1350 0x7f48840a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.986+0000 7f4897fff700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48840a1350 0x7f48840a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.987+0000 7f48957fa700 1 -- 172.21.15.39:0/3052383956 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f48900ce5e0 con 0x7f4898112150 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.987+0000 7f4897fff700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48840a1350 0x7f48840a3810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f48900981a0 tx=0x7f4890009510 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.990+0000 7f48977fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 0x560741617f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x5607416188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 0x56074161dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 0x56074161dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43592/0 (socket says 172.21.15.39:43592) 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 -- 172.21.15.39:0/566573639 learned_addr learned my addr 172.21.15.39:0/566573639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 -- 172.21.15.39:0/566573639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 msgr2=0x560741617f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 --2- 172.21.15.39:0/566573639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 0x560741617f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 -- 172.21.15.39:0/566573639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 msgr2=0x56074161dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 --2- 172.21.15.39:0/566573639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 0x56074161dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 -- 172.21.15.39:0/566573639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607413e9350 con 0x560741618480 2024-05-24T11:53:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.991+0000 7f488effd700 1 --2- 172.21.15.39:0/566573639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x5607416188a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f488800ebc0 tx=0x7f488800c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31759321f647a3dc server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.993+0000 7f488dffb700 1 -- 172.21.15.39:0/566573639 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488800e240 con 0x560741618480 2024-05-24T11:53:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.994+0000 7f488dffb700 1 -- 172.21.15.39:0/566573639 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f488800e6a0 con 0x560741618480 2024-05-24T11:53:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:17.994+0000 7f488dffb700 1 -- 172.21.15.39:0/566573639 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880054e0 con 0x560741618480 2024-05-24T11:53:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.000+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/566573639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 msgr2=0x5607416188a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.000+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/566573639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x5607416188a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f488800ebc0 tx=0x7f488800c410 comp rx=0 tx=0).stop 2024-05-24T11:53:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.001+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/566573639 shutdown_connections 2024-05-24T11:53:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.001+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/566573639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x5607416188a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.001+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/566573639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 0x56074161dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.001+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/566573639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 0x560741617f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.001+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/566573639 >> 172.21.15.39:0/566573639 conn(0x5607415761b0 msgr2=0x560741612840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.005+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/566573639 shutdown_connections 2024-05-24T11:53:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.007+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/566573639 wait complete. 2024-05-24T11:53:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.007+0000 7f48a9cfeb80 1 -- start start 2024-05-24T11:53:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.007+0000 7f48a9cfeb80 1 Processor -- start 2024-05-24T11:53:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.008+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x560741624210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.008+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 0x560741621500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.008+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 0x560741621c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.008+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560741504500 con 0x56074161d950 2024-05-24T11:53:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.008+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607415041a0 con 0x560741618480 2024-05-24T11:53:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.008+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607413eb640 con 0x56074161e270 2024-05-24T11:53:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.008+0000 7f48a9cfeb80 10 client.0 _ll_register_callbacks cb 0x56074150dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x560741624210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 0x560741621c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x560741624210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:38404/0 (socket says 172.21.15.39:38404) 2024-05-24T11:53:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 -- 172.21.15.39:0/2718243642 learned_addr learned my addr 172.21.15.39:0/2718243642 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f48977fe700 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 0x560741621500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 -- 172.21.15.39:0/2718243642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 msgr2=0x560741621c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 0x560741621c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 -- 172.21.15.39:0/2718243642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 msgr2=0x560741621500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 0x560741621500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 -- 172.21.15.39:0/2718243642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488800e870 con 0x560741618480 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f488effd700 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x560741624210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f488800eb90 tx=0x7f48880050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d825e2405dbdd659 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.010+0000 7f4876ffd700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f4876ffd700 1 -- 172.21.15.39:0/2718243642 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888005720 con 0x560741618480 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f4876ffd700 1 -- 172.21.15.39:0/2718243642 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f488800e240 con 0x560741618480 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2718243642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607414c1210 con 0x560741618480 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f4876ffd700 1 -- 172.21.15.39:0/2718243642 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4888004d90 con 0x560741618480 2024-05-24T11:53:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f48a9cfeb80 10 client.26171 fetch_fsmap learned FSMap version 38 2024-05-24T11:53:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2718243642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560741395b00 con 0x560741618480 2024-05-24T11:53:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f4876ffd700 1 -- 172.21.15.39:0/2718243642 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5607414c1210 con 0x560741618480 2024-05-24T11:53:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f4876ffd700 1 -- 172.21.15.39:0/2718243642 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f488801d4e0 con 0x560741618480 2024-05-24T11:53:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f48a9cfeb80 10 client.26171 fetch_fsmap finished waiting for FSMap version 38 2024-05-24T11:53:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f48a9cfeb80 10 client.26171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f48a9cfeb80 20 client.26171 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560741631be0 0x560741634000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f48a9cfeb80 4 client.26171 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:53:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.011+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2718243642 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607415e77b0 con 0x560741631be0 2024-05-24T11:53:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.012+0000 7f48977fe700 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560741631be0 0x560741634000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.013+0000 7f48977fe700 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560741631be0 0x560741634000 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0604504bf28448e server_cookie=2b5519e892dc2674 in_seq=0 out_seq=0 2024-05-24T11:53:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.013+0000 7f4876ffd700 10 client.26171 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.014+0000 7f4876ffd700 1 -- 172.21.15.39:0/2718243642 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607415e77b0 con 0x560741631be0 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.014+0000 7f4876ffd700 10 client.26171 handle_command_reply: tid=0 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.014+0000 7f48a9cfeb80 1 client.26171 shutdown 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.014+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2718243642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 msgr2=0x560741624210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.014+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x560741624210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f488800eb90 tx=0x7f48880050c0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.015+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2718243642 shutdown_connections 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.015+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161e270 0x560741621c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.015+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x560741631be0 0x560741634000 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.015+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560741618480 0x560741624210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.015+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2718243642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161d950 0x560741621500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.015+0000 7f4876ffd700 0 client.26171 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.015+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2718243642 >> 172.21.15.39:0/2718243642 conn(0x5607415761b0 msgr2=0x56074161c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.015+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2718243642 shutdown_connections 2024-05-24T11:53:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.015+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2718243642 wait complete. 2024-05-24T11:53:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.017+0000 7f48a9cfeb80 20 client.26171 trim_cache size 0 max 16384 2024-05-24T11:53:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48a9cfeb80 1 Processor -- start 2024-05-24T11:53:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48a9cfeb80 1 -- start start 2024-05-24T11:53:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560741618480 0x560741633fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161d950 0x560741618860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161e270 0x56074162d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x560741614130 con 0x56074161e270 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560741639240 con 0x56074161d950 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56074161fa80 con 0x560741618480 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161e270 0x56074162d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161e270 0x56074162d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43600/0 (socket says 172.21.15.39:43600) 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 -- 172.21.15.39:0/1923263303 learned_addr learned my addr 172.21.15.39:0/1923263303 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48977fe700 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161d950 0x560741618860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560741618480 0x560741633fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 -- 172.21.15.39:0/1923263303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560741618480 msgr2=0x560741633fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560741618480 0x560741633fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 -- 172.21.15.39:0/1923263303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161d950 msgr2=0x560741618860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161d950 0x560741618860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 -- 172.21.15.39:0/1923263303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56074152e500 con 0x56074161e270 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f48977fe700 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161d950 0x560741618860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488effd700 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161e270 0x56074162d720 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f488800c9b0 tx=0x7f488800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba41966d66f25065 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488e7fc700 1 -- 172.21.15.39:0/1923263303 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488800cd40 con 0x56074161e270 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488e7fc700 1 -- 172.21.15.39:0/1923263303 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4888012030 con 0x56074161e270 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.034+0000 7f488e7fc700 1 -- 172.21.15.39:0/1923263303 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488801cee0 con 0x56074161e270 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/1923263303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161e270 msgr2=0x56074162d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161e270 0x56074162d720 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f488800c9b0 tx=0x7f488800c8e0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/1923263303 shutdown_connections 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560741618480 0x560741633fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56074161e270 0x56074162d720 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/1923263303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161d950 0x560741618860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/1923263303 >> 172.21.15.39:0/1923263303 conn(0x5607415761b0 msgr2=0x560741617520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/1923263303 shutdown_connections 2024-05-24T11:53:18.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/1923263303 wait complete. 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 -- start start 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.035+0000 7f48a9cfeb80 1 Processor -- start 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560741618480 0x560741629090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161d950 0x560741624c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48a9cfeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161e270 0x5607416251a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607416258b0 con 0x560741618480 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560741625a20 con 0x56074161e270 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48a9cfeb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607416870d0 con 0x56074161d950 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48a9cfeb80 10 client.0 _ll_register_callbacks cb 0x56074150dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f488effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560741618480 0x560741629090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f488effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560741618480 0x560741629090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43602/0 (socket says 172.21.15.39:43602) 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f488effd700 1 -- 172.21.15.39:0/2615609684 learned_addr learned my addr 172.21.15.39:0/2615609684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48977fe700 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161d950 0x560741624c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f488effd700 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161e270 0x5607416251a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48977fe700 1 -- 172.21.15.39:0/2615609684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161e270 msgr2=0x5607416251a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48977fe700 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161e270 0x5607416251a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48977fe700 1 -- 172.21.15.39:0/2615609684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560741618480 msgr2=0x560741629090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48977fe700 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560741618480 0x560741629090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48977fe700 1 -- 172.21.15.39:0/2615609684 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488000e040 con 0x56074161d950 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f48977fe700 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161d950 0x560741624c40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x560741634910 tx=0x7f488000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d90529145fa6daa server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.036+0000 7f4876ffd700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f4876ffd700 1 -- 172.21.15.39:0/2615609684 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488001e070 con 0x56074161d950 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2615609684 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f488800e6a0 con 0x56074161d950 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f4876ffd700 1 -- 172.21.15.39:0/2615609684 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4880014b00 con 0x56074161d950 2024-05-24T11:53:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f4876ffd700 1 -- 172.21.15.39:0/2615609684 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4880014cc0 con 0x56074161d950 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f48a9cfeb80 10 client.26086 fetch_fsmap learned FSMap version 38 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2615609684 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560741395b00 con 0x56074161d950 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f4876ffd700 1 -- 172.21.15.39:0/2615609684 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f488002e080 con 0x56074161d950 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f4876ffd700 1 -- 172.21.15.39:0/2615609684 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 38) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f488002a3f0 con 0x56074161d950 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f48a9cfeb80 10 client.26086 fetch_fsmap finished waiting for FSMap version 38 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f48a9cfeb80 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f48a9cfeb80 20 client.26086 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5607416888e0 0x56074168ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f48a9cfeb80 4 client.26086 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2615609684 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560741560e60 con 0x5607416888e0 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.037+0000 7f488effd700 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5607416888e0 0x56074168ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.038+0000 7f488effd700 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5607416888e0 0x56074168ad00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=770ba48f5e94c256 server_cookie=9f345a86cf77aedf in_seq=0 out_seq=0 2024-05-24T11:53:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.038+0000 7f4876ffd700 10 client.26086 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.065+0000 7f4876ffd700 1 -- 172.21.15.39:0/2615609684 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560741560e60 con 0x5607416888e0 2024-05-24T11:53:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.065+0000 7f4876ffd700 10 client.26086 handle_command_reply: tid=0 2024-05-24T11:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.065+0000 7f48a9cfeb80 1 client.26086 shutdown 2024-05-24T11:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.065+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2615609684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161d950 msgr2=0x560741624c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.065+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161d950 0x560741624c40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x560741634910 tx=0x7f488000bea0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.066+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2615609684 shutdown_connections 2024-05-24T11:53:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.067+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5607416888e0 0x56074168ad00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.067+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56074161e270 0x5607416251a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.067+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560741618480 0x560741629090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.067+0000 7f48a9cfeb80 1 --2- 172.21.15.39:0/2615609684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56074161d950 0x560741624c40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.067+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2615609684 >> 172.21.15.39:0/2615609684 conn(0x5607415761b0 msgr2=0x56074161e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.067+0000 7f4876ffd700 0 client.26086 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.067+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2615609684 shutdown_connections 2024-05-24T11:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.067+0000 7f48a9cfeb80 1 -- 172.21.15.39:0/2615609684 wait complete. 2024-05-24T11:53:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.067+0000 7f48a9cfeb80 20 client.26086 trim_cache size 0 max 16384 2024-05-24T11:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:53:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 -- 172.21.15.39:0/3052383956 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48840a1350 msgr2=0x7f48840a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48840a1350 0x7f48840a3810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f48900981a0 tx=0x7f4890009510 comp rx=0 tx=0).stop 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 -- 172.21.15.39:0/3052383956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898112150 msgr2=0x7f48981125e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898112150 0x7f48981125e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4890015000 tx=0x7f489000cd90 comp rx=0 tx=0).stop 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 -- 172.21.15.39:0/3052383956 shutdown_connections 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48840a1350 0x7f48840a3810 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4898112150 0x7f48981125e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4898108830 0x7f4898114b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 --2- 172.21.15.39:0/3052383956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48981091f0 0x7f4898111c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 -- 172.21.15.39:0/3052383956 >> 172.21.15.39:0/3052383956 conn(0x7f489806bf80 msgr2=0x7f489810a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 -- 172.21.15.39:0/3052383956 shutdown_connections 2024-05-24T11:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:18.068+0000 7f489db43700 1 -- 172.21.15.39:0/3052383956 wait complete. 2024-05-24T11:53:18.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:17 smithi039 ceph-mon[42919]: pgmap v479: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2024-05-24T11:53:18.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:17 smithi053 ceph-mon[47448]: pgmap v479: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2024-05-24T11:53:18.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:17 smithi137 ceph-mon[47907]: pgmap v479: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2024-05-24T11:53:18.743 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:53:18 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setuser ceph since I am not root 2024-05-24T11:53:18.744 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:53:18 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: ignoring --setgroup ceph since I am not root 2024-05-24T11:53:18.744 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:53:18 smithi137 ceph-mds[84908]: -- 172.21.15.137:0/2775171086 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56139ea841a0 con 0x56139ea5b000 2024-05-24T11:53:18.744 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:53:18 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: starting mds.f at 2024-05-24T11:53:19.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:18 smithi039 ceph-mon[42919]: daemon mds.f restarted 2024-05-24T11:53:19.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:18 smithi039 ceph-mon[42919]: osdmap e105: 12 total, 12 up, 12 in 2024-05-24T11:53:19.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 835.0 s, 1154.6 tps, lat 86.633 ms stddev 45.016 2024-05-24T11:53:19.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:18 smithi053 ceph-mon[47448]: daemon mds.f restarted 2024-05-24T11:53:19.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:18 smithi053 ceph-mon[47448]: osdmap e105: 12 total, 12 up, 12 in 2024-05-24T11:53:19.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:18 smithi137 ceph-mon[47907]: daemon mds.f restarted 2024-05-24T11:53:19.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:18 smithi137 ceph-mon[47907]: osdmap e105: 12 total, 12 up, 12 in 2024-05-24T11:53:20.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:19 smithi053 ceph-mon[47448]: pgmap v481: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 17 MiB/s wr, 1.21k op/s 2024-05-24T11:53:20.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:19 smithi053 ceph-mon[47448]: mds.? [v2:172.21.15.137:6834/535347136,v1:172.21.15.137:6835/535347136] up:boot 2024-05-24T11:53:20.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:19 smithi053 ceph-mon[47448]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:53:20.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:19 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:53:20.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:19 smithi053 ceph-mon[47448]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:53:20.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:19 smithi137 ceph-mon[47907]: pgmap v481: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 17 MiB/s wr, 1.21k op/s 2024-05-24T11:53:20.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:19 smithi137 ceph-mon[47907]: mds.? [v2:172.21.15.137:6834/535347136,v1:172.21.15.137:6835/535347136] up:boot 2024-05-24T11:53:20.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:19 smithi137 ceph-mon[47907]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:53:20.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:19 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:53:20.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:19 smithi137 ceph-mon[47907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:53:20.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:19 smithi039 ceph-mon[42919]: pgmap v481: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 17 MiB/s wr, 1.21k op/s 2024-05-24T11:53:20.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:19 smithi039 ceph-mon[42919]: mds.? [v2:172.21.15.137:6834/535347136,v1:172.21.15.137:6835/535347136] up:boot 2024-05-24T11:53:20.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:19 smithi039 ceph-mon[42919]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-24T11:53:20.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:19 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-24T11:53:20.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:19 smithi039 ceph-mon[42919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-24T11:53:22.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:21 smithi053 ceph-mon[47448]: pgmap v482: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2024-05-24T11:53:22.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:21 smithi137 ceph-mon[47907]: pgmap v482: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2024-05-24T11:53:22.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:21 smithi039 ceph-mon[42919]: pgmap v482: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 23 MiB/s wr, 1.64k op/s 2024-05-24T11:53:24.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 840.0 s, 1118.8 tps, lat 89.391 ms stddev 40.083 2024-05-24T11:53:24.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:23 smithi053 ceph-mon[47448]: pgmap v483: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 825 op/s 2024-05-24T11:53:24.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:23 smithi137 ceph-mon[47907]: pgmap v483: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 825 op/s 2024-05-24T11:53:24.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:23 smithi039 ceph-mon[42919]: pgmap v483: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 12 MiB/s wr, 825 op/s 2024-05-24T11:53:26.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:25 smithi053 ceph-mon[47448]: pgmap v484: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 18 MiB/s wr, 1.32k op/s 2024-05-24T11:53:26.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:25 smithi137 ceph-mon[47907]: pgmap v484: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 18 MiB/s wr, 1.32k op/s 2024-05-24T11:53:26.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:25 smithi039 ceph-mon[42919]: pgmap v484: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 18 MiB/s wr, 1.32k op/s 2024-05-24T11:53:28.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:27 smithi053 ceph-mon[47448]: pgmap v485: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2024-05-24T11:53:28.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:27 smithi137 ceph-mon[47907]: pgmap v485: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2024-05-24T11:53:28.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:27 smithi039 ceph-mon[42919]: pgmap v485: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2024-05-24T11:53:29.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 845.0 s, 1128.2 tps, lat 88.689 ms stddev 36.723 2024-05-24T11:53:30.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:29 smithi053 ceph-mon[47448]: pgmap v486: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2024-05-24T11:53:30.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:29 smithi137 ceph-mon[47907]: pgmap v486: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2024-05-24T11:53:30.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:29 smithi039 ceph-mon[42919]: pgmap v486: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2024-05-24T11:53:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.261+0000 7fb9bf075700 1 -- 172.21.15.39:0/3380273136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8107000 msgr2=0x7fb9b81073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.261+0000 7fb9bf075700 1 --2- 172.21.15.39:0/3380273136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8107000 0x7fb9b81073e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009990 tx=0x7fb9a8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.261+0000 7fb9bf075700 1 -- 172.21.15.39:0/3380273136 shutdown_connections 2024-05-24T11:53:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.261+0000 7fb9bf075700 1 --2- 172.21.15.39:0/3380273136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9b8068ca0 0x7fb9b810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.261+0000 7fb9bf075700 1 --2- 172.21.15.39:0/3380273136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8107000 0x7fb9b81073e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.261+0000 7fb9bf075700 1 --2- 172.21.15.39:0/3380273136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b80682e0 0x7fb9b8068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.261+0000 7fb9bf075700 1 -- 172.21.15.39:0/3380273136 >> 172.21.15.39:0/3380273136 conn(0x7fb9b8076f10 msgr2=0x7fb9b8077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.262+0000 7fb9bf075700 1 -- 172.21.15.39:0/3380273136 shutdown_connections 2024-05-24T11:53:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.262+0000 7fb9bf075700 1 -- 172.21.15.39:0/3380273136 wait complete. 2024-05-24T11:53:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.262+0000 7fb9bf075700 1 Processor -- start 2024-05-24T11:53:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.262+0000 7fb9bf075700 1 -- start start 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bf075700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b80682e0 0x7fb9b819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bf075700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9b8068ca0 0x7fb9b819e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bf075700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b8107000 0x7fb9b810cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bf075700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b810d560 con 0x7fb9b8068ca0 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bf075700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b810d6a0 con 0x7fb9b80682e0 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bf075700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b810d860 con 0x7fb9b8107000 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bce11700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b80682e0 0x7fb9b819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b8107000 0x7fb9b810cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b8107000 0x7fb9b810cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:59790/0 (socket says 172.21.15.39:59790) 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.263+0000 7fb9bd612700 1 -- 172.21.15.39:0/1075596055 learned_addr learned my addr 172.21.15.39:0/1075596055 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.264+0000 7fb9bce11700 1 -- 172.21.15.39:0/1075596055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b8107000 msgr2=0x7fb9b810cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.264+0000 7fb9bce11700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b8107000 0x7fb9b810cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.264+0000 7fb9bce11700 1 -- 172.21.15.39:0/1075596055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9b8068ca0 msgr2=0x7fb9b819e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.264+0000 7fb9bce11700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9b8068ca0 0x7fb9b819e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.264+0000 7fb9bce11700 1 -- 172.21.15.39:0/1075596055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a80095f0 con 0x7fb9b80682e0 2024-05-24T11:53:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.264+0000 7fb9bd612700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b8107000 0x7fb9b810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:53:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.264+0000 7fb9bce11700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b80682e0 0x7fb9b819df70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8000c00 tx=0x7fb9a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.264+0000 7fb9b5ffb700 1 -- 172.21.15.39:0/1075596055 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a801d070 con 0x7fb9b80682e0 2024-05-24T11:53:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.265+0000 7fb9b5ffb700 1 -- 172.21.15.39:0/1075596055 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9a800fcb0 con 0x7fb9b80682e0 2024-05-24T11:53:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.265+0000 7fb9b5ffb700 1 -- 172.21.15.39:0/1075596055 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8022c90 con 0x7fb9b80682e0 2024-05-24T11:53:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.265+0000 7fb9bf075700 1 -- 172.21.15.39:0/1075596055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b810dae0 con 0x7fb9b80682e0 2024-05-24T11:53:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.265+0000 7fb9bf075700 1 -- 172.21.15.39:0/1075596055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b8199c20 con 0x7fb9b80682e0 2024-05-24T11:53:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.270+0000 7fb9cb230b80 1 Processor -- start 2024-05-24T11:53:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.270+0000 7fb9cb230b80 1 -- start start 2024-05-24T11:53:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.271+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b16950 0x562ce7b16d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.271+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b17270 0x562ce7b10f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.271+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b11480 0x562ce7b118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.271+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7a680f0 con 0x562ce7b16950 2024-05-24T11:53:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.271+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7a65ab0 con 0x562ce7b17270 2024-05-24T11:53:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.271+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7a613a0 con 0x562ce7b11480 2024-05-24T11:53:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.271+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b16950 0x562ce7b16d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.271+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b16950 0x562ce7b16d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51140/0 (socket says 172.21.15.39:51140) 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.271+0000 7fb9bd612700 1 -- 172.21.15.39:0/1302931527 learned_addr learned my addr 172.21.15.39:0/1302931527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.272+0000 7fb9bd612700 1 -- 172.21.15.39:0/1302931527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b11480 msgr2=0x562ce7b118a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.272+0000 7fb9bd612700 1 --2- 172.21.15.39:0/1302931527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b11480 0x562ce7b118a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.272+0000 7fb9bd612700 1 -- 172.21.15.39:0/1302931527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b17270 msgr2=0x562ce7b10f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.272+0000 7fb9bd612700 1 --2- 172.21.15.39:0/1302931527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b17270 0x562ce7b10f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.272+0000 7fb9bd612700 1 -- 172.21.15.39:0/1302931527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce78e2350 con 0x562ce7b16950 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.272+0000 7fb9bd612700 1 --2- 172.21.15.39:0/1302931527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b16950 0x562ce7b16d30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000e5b0 tx=0x7fb9b0009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e13012d8b50b65 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.272+0000 7fb9b5ffb700 1 -- 172.21.15.39:0/1075596055 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb9a8022470 con 0x7fb9b80682e0 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.273+0000 7fb99effd700 1 -- 172.21.15.39:0/1302931527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0014890 con 0x562ce7b16950 2024-05-24T11:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.273+0000 7fb99effd700 1 -- 172.21.15.39:0/1302931527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9b000f040 con 0x562ce7b16950 2024-05-24T11:53:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.273+0000 7fb99effd700 1 -- 172.21.15.39:0/1302931527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0013cb0 con 0x562ce7b16950 2024-05-24T11:53:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.273+0000 7fb9b5ffb700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb9a00a1310 0x7fb9a00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9b5ffb700 1 -- 172.21.15.39:0/1075596055 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb9a80ce280 con 0x7fb9b80682e0 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9bce11700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb9a00a1310 0x7fb9a00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9cb230b80 1 -- 172.21.15.39:0/1302931527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b16950 msgr2=0x562ce7b16d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/1302931527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b16950 0x562ce7b16d30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000e5b0 tx=0x7fb9b0009710 comp rx=0 tx=0).stop 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9cb230b80 1 -- 172.21.15.39:0/1302931527 shutdown_connections 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/1302931527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b11480 0x562ce7b118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/1302931527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b16950 0x562ce7b16d30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/1302931527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b17270 0x562ce7b10f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9cb230b80 1 -- 172.21.15.39:0/1302931527 >> 172.21.15.39:0/1302931527 conn(0x562ce7a6f1b0 msgr2=0x562ce7b0b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9cb230b80 1 -- 172.21.15.39:0/1302931527 shutdown_connections 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.274+0000 7fb9cb230b80 1 -- 172.21.15.39:0/1302931527 wait complete. 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.275+0000 7fb9bce11700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb9a00a1310 0x7fb9a00a37d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x562ce7b0c790 tx=0x7fb9a8005ee0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.275+0000 7fb9cb230b80 1 -- start start 2024-05-24T11:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.275+0000 7fb9cb230b80 1 Processor -- start 2024-05-24T11:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b11480 0x562ce7b1a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b16950 0x562ce7b1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b1b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce79fd500 con 0x562ce7b17270 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce79fd1a0 con 0x562ce7b11480 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce78e4640 con 0x562ce7b16950 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9cb230b80 10 client.0 _ll_register_callbacks cb 0x562ce7a06ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b1b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b1b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51152/0 (socket says 172.21.15.39:51152) 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9bd612700 1 -- 172.21.15.39:0/3390248457 learned_addr learned my addr 172.21.15.39:0/3390248457 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.276+0000 7fb9bd612700 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b11480 0x562ce7b1a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb9bd612700 1 -- 172.21.15.39:0/3390248457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b16950 msgr2=0x562ce7b1ab60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb9bd612700 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b16950 0x562ce7b1ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb9bd612700 1 -- 172.21.15.39:0/3390248457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b11480 msgr2=0x562ce7b1a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb9bd612700 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b11480 0x562ce7b1a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb9bd612700 1 -- 172.21.15.39:0/3390248457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b000e260 con 0x562ce7b17270 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb9bd612700 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b1b0a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0013620 tx=0x7fb9b0013850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d838970d75aed97 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb997fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb997fff700 1 -- 172.21.15.39:0/3390248457 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0025070 con 0x562ce7b17270 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb997fff700 1 -- 172.21.15.39:0/3390248457 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb9b0006e40 con 0x562ce7b17270 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.277+0000 7fb9cb230b80 1 -- 172.21.15.39:0/3390248457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ce79ba210 con 0x562ce7b17270 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb997fff700 1 -- 172.21.15.39:0/3390248457 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9b0014cf0 con 0x562ce7b17270 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb9cb230b80 10 client.16422 fetch_fsmap learned FSMap version 40 2024-05-24T11:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb9cb230b80 1 -- 172.21.15.39:0/3390248457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562ce788eb00 con 0x562ce7b17270 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb997fff700 1 -- 172.21.15.39:0/3390248457 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562ce79ba210 con 0x562ce7b17270 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb997fff700 1 -- 172.21.15.39:0/3390248457 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb9b002b030 con 0x562ce7b17270 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb9cb230b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb9cb230b80 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb9cb230b80 20 client.16422 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562ce7b2ac80 0x562ce7b2d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb9cb230b80 4 client.16422 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.278+0000 7fb9cb230b80 1 -- 172.21.15.39:0/3390248457 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ce7ae07b0 con 0x562ce7b2ac80 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.279+0000 7fb9b7fff700 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562ce7b2ac80 0x562ce7b2d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.280+0000 7fb9b7fff700 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562ce7b2ac80 0x562ce7b2d0a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da537157cf23c5d server_cookie=c183051e7b816d4d in_seq=0 out_seq=0 2024-05-24T11:53:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.280+0000 7fb997fff700 10 client.16422 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.280+0000 7fb997fff700 1 -- 172.21.15.39:0/3390248457 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ce7ae07b0 con 0x562ce7b2ac80 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.280+0000 7fb997fff700 10 client.16422 handle_command_reply: tid=0 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 client.16422 shutdown 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 -- 172.21.15.39:0/3390248457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 msgr2=0x562ce7b1b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b1b0a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0013620 tx=0x7fb9b0013850 comp rx=0 tx=0).stop 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 -- 172.21.15.39:0/3390248457 shutdown_connections 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562ce7b2ac80 0x562ce7b2d0a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b1b0a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb997fff700 0 client.16422 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b11480 0x562ce7b1a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/3390248457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b16950 0x562ce7b1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 -- 172.21.15.39:0/3390248457 >> 172.21.15.39:0/3390248457 conn(0x562ce7a6f1b0 msgr2=0x562ce7b154c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 -- 172.21.15.39:0/3390248457 shutdown_connections 2024-05-24T11:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 1 -- 172.21.15.39:0/3390248457 wait complete. 2024-05-24T11:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.281+0000 7fb9cb230b80 20 client.16422 trim_cache size 0 max 16384 2024-05-24T11:53:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.307+0000 7fb9cb230b80 1 Processor -- start 2024-05-24T11:53:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9cb230b80 1 -- start start 2024-05-24T11:53:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b11480 0x562ce7b0b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b16950 0x562ce7b15390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b17270 0x562ce7b21c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7b1d710 con 0x562ce7b11480 2024-05-24T11:53:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7b32270 con 0x562ce7b16950 2024-05-24T11:53:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7b2d700 con 0x562ce7b17270 2024-05-24T11:53:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b11480 0x562ce7b0b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b11480 0x562ce7b0b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51166/0 (socket says 172.21.15.39:51166) 2024-05-24T11:53:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9bd612700 1 -- 172.21.15.39:0/298644806 learned_addr learned my addr 172.21.15.39:0/298644806 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.311+0000 7fb9bd612700 1 --2- 172.21.15.39:0/298644806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b17270 0x562ce7b21c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.312+0000 7fb9bd612700 1 -- 172.21.15.39:0/298644806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b17270 msgr2=0x562ce7b21c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.312+0000 7fb9bd612700 1 --2- 172.21.15.39:0/298644806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b17270 0x562ce7b21c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.312+0000 7fb9bd612700 1 -- 172.21.15.39:0/298644806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b16950 msgr2=0x562ce7b15390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.312+0000 7fb9bd612700 1 --2- 172.21.15.39:0/298644806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b16950 0x562ce7b15390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.312+0000 7fb9bd612700 1 -- 172.21.15.39:0/298644806 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce7a27500 con 0x562ce7b11480 2024-05-24T11:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.312+0000 7fb9bd612700 1 --2- 172.21.15.39:0/298644806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b11480 0x562ce7b0b910 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009580 tx=0x7fb9b0007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b71c227053d6842 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.313+0000 7fb99f7fe700 1 -- 172.21.15.39:0/298644806 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0025040 con 0x562ce7b11480 2024-05-24T11:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.313+0000 7fb99f7fe700 1 -- 172.21.15.39:0/298644806 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9b000e050 con 0x562ce7b11480 2024-05-24T11:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.313+0000 7fb99f7fe700 1 -- 172.21.15.39:0/298644806 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00076b0 con 0x562ce7b11480 2024-05-24T11:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.315+0000 7fb9cb230b80 1 -- 172.21.15.39:0/298644806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b11480 msgr2=0x562ce7b0b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.315+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/298644806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b11480 0x562ce7b0b910 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009580 tx=0x7fb9b0007040 comp rx=0 tx=0).stop 2024-05-24T11:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.319+0000 7fb9cb230b80 1 -- 172.21.15.39:0/298644806 shutdown_connections 2024-05-24T11:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.319+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/298644806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b17270 0x562ce7b21c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.319+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/298644806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b11480 0x562ce7b0b910 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.319+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/298644806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b16950 0x562ce7b15390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.319+0000 7fb9cb230b80 1 -- 172.21.15.39:0/298644806 >> 172.21.15.39:0/298644806 conn(0x562ce7a6f1b0 msgr2=0x562ce7b0f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.326+0000 7fb9cb230b80 1 -- 172.21.15.39:0/298644806 shutdown_connections 2024-05-24T11:53:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.326+0000 7fb9cb230b80 1 -- 172.21.15.39:0/298644806 wait complete. 2024-05-24T11:53:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.326+0000 7fb9cb230b80 1 -- start start 2024-05-24T11:53:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.326+0000 7fb9cb230b80 1 Processor -- start 2024-05-24T11:53:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b11480 0x562ce7b21c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b16950 0x562ce7b22180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9cb230b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b14380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7b148c0 con 0x562ce7b17270 2024-05-24T11:53:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7b14a30 con 0x562ce7b16950 2024-05-24T11:53:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9cb230b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7b14bf0 con 0x562ce7b11480 2024-05-24T11:53:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9cb230b80 10 client.0 _ll_register_callbacks cb 0x562ce7a06ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b14380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9b7fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b16950 0x562ce7b22180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9bd612700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b14380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51168/0 (socket says 172.21.15.39:51168) 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.327+0000 7fb9bd612700 1 -- 172.21.15.39:0/603504508 learned_addr learned my addr 172.21.15.39:0/603504508 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb9bd612700 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b11480 0x562ce7b21c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb9bd612700 1 -- 172.21.15.39:0/603504508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b11480 msgr2=0x562ce7b21c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb9bd612700 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b11480 0x562ce7b21c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb9bd612700 1 -- 172.21.15.39:0/603504508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b16950 msgr2=0x562ce7b22180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb9bd612700 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b16950 0x562ce7b22180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb9bd612700 1 -- 172.21.15.39:0/603504508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ce788eb00 con 0x562ce7b17270 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb9bd612700 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b14380 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000e5b0 tx=0x7fb9b000e1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc487e228583f263 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb997fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb997fff700 1 -- 172.21.15.39:0/603504508 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0013690 con 0x562ce7b17270 2024-05-24T11:53:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb997fff700 1 -- 172.21.15.39:0/603504508 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8454+0+0 (secure 0 0 0) 0x7fb9b0007370 con 0x562ce7b17270 2024-05-24T11:53:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.328+0000 7fb9cb230b80 1 -- 172.21.15.39:0/603504508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9b000f040 con 0x562ce7b17270 2024-05-24T11:53:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.329+0000 7fb997fff700 1 -- 172.21.15.39:0/603504508 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9b0007680 con 0x562ce7b17270 2024-05-24T11:53:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.329+0000 7fb9cb230b80 10 client.16428 fetch_fsmap learned FSMap version 40 2024-05-24T11:53:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.329+0000 7fb9cb230b80 1 -- 172.21.15.39:0/603504508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562ce78e6c70 con 0x562ce7b17270 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.329+0000 7fb997fff700 1 -- 172.21.15.39:0/603504508 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9b000f040 con 0x562ce7b17270 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.329+0000 7fb997fff700 1 -- 172.21.15.39:0/603504508 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb9b000e970 con 0x562ce7b17270 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.330+0000 7fb9cb230b80 10 client.16428 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.330+0000 7fb9cb230b80 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.330+0000 7fb9cb230b80 20 client.16428 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.331+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562ce7b81a70 0x562ce7b83e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.331+0000 7fb9cb230b80 4 client.16428 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.331+0000 7fb9cb230b80 1 -- 172.21.15.39:0/603504508 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ce7a59e60 con 0x562ce7b81a70 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.331+0000 7fb9b7fff700 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562ce7b81a70 0x562ce7b83e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.331+0000 7fb997fff700 10 client.16428 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:31.331+0000 7fb9b7fff700 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562ce7b81a70 0x562ce7b83e60 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccdd97b89cb5efc server_cookie=fe1940d0df7b8631 in_seq=0 out_seq=0 2024-05-24T11:53:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:31 smithi053 ceph-mon[47448]: pgmap v487: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 18 MiB/s wr, 1.35k op/s 2024-05-24T11:53:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:31 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:53:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:31 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:53:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:31 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:53:32.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:31 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:53:32.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:31 smithi137 ceph-mon[47907]: pgmap v487: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 18 MiB/s wr, 1.35k op/s 2024-05-24T11:53:32.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:31 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:53:32.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:31 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:53:32.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:31 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:53:32.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:31 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:53:32.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:31 smithi039 ceph-mon[42919]: pgmap v487: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 18 MiB/s wr, 1.35k op/s 2024-05-24T11:53:32.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:31 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:53:32.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:31 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:53:32.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:31 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:53:32.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:31 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:53:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.924+0000 7fb997fff700 1 -- 172.21.15.39:0/603504508 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ce7a59e60 con 0x562ce7b81a70 2024-05-24T11:53:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.924+0000 7fb997fff700 10 client.16428 handle_command_reply: tid=0 2024-05-24T11:53:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.924+0000 7fb9cb230b80 1 client.16428 shutdown 2024-05-24T11:53:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb9cb230b80 1 -- 172.21.15.39:0/603504508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 msgr2=0x562ce7b14380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b14380 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000e5b0 tx=0x7fb9b000e1c0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb9cb230b80 1 -- 172.21.15.39:0/603504508 shutdown_connections 2024-05-24T11:53:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ce7b11480 0x562ce7b21c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x562ce7b81a70 0x562ce7b83e60 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce7b17270 0x562ce7b14380 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb9cb230b80 1 --2- 172.21.15.39:0/603504508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562ce7b16950 0x562ce7b22180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb997fff700 0 client.16428 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb9cb230b80 1 -- 172.21.15.39:0/603504508 >> 172.21.15.39:0/603504508 conn(0x562ce7a6f1b0 msgr2=0x562ce7b10610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.926+0000 7fb9cb230b80 1 -- 172.21.15.39:0/603504508 shutdown_connections 2024-05-24T11:53:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.927+0000 7fb9cb230b80 1 -- 172.21.15.39:0/603504508 wait complete. 2024-05-24T11:53:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.927+0000 7fb9cb230b80 20 client.16428 trim_cache size 0 max 16384 2024-05-24T11:53:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:53:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:53:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:53:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:53:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 -- 172.21.15.39:0/1075596055 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb9a00a1310 msgr2=0x7fb9a00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb9a00a1310 0x7fb9a00a37d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x562ce7b0c790 tx=0x7fb9a8005ee0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 -- 172.21.15.39:0/1075596055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b80682e0 msgr2=0x7fb9b819df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b80682e0 0x7fb9b819df70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8000c00 tx=0x7fb9a8004730 comp rx=0 tx=0).stop 2024-05-24T11:53:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 -- 172.21.15.39:0/1075596055 shutdown_connections 2024-05-24T11:53:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b8107000 0x7fb9b810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fb9a00a1310 0x7fb9a00a37d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b80682e0 0x7fb9b819df70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 --2- 172.21.15.39:0/1075596055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9b8068ca0 0x7fb9b819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.928+0000 7fb9bf075700 1 -- 172.21.15.39:0/1075596055 >> 172.21.15.39:0/1075596055 conn(0x7fb9b8076f10 msgr2=0x7fb9b80fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.929+0000 7fb9bf075700 1 -- 172.21.15.39:0/1075596055 shutdown_connections 2024-05-24T11:53:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:32.929+0000 7fb9bf075700 1 -- 172.21.15.39:0/1075596055 wait complete. 2024-05-24T11:53:34.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 850.0 s, 1030.0 tps, lat 96.934 ms stddev 39.747 2024-05-24T11:53:34.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:33 smithi053 ceph-mon[47448]: pgmap v488: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-05-24T11:53:34.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:33 smithi137 ceph-mon[47907]: pgmap v488: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-05-24T11:53:34.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:33 smithi039 ceph-mon[42919]: pgmap v488: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-05-24T11:53:36.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:35 smithi053 ceph-mon[47448]: pgmap v489: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 20 MiB/s wr, 1.57k op/s 2024-05-24T11:53:36.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:35 smithi137 ceph-mon[47907]: pgmap v489: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 20 MiB/s wr, 1.57k op/s 2024-05-24T11:53:36.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:35 smithi039 ceph-mon[42919]: pgmap v489: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 20 MiB/s wr, 1.57k op/s 2024-05-24T11:53:38.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:37 smithi053 ceph-mon[47448]: pgmap v490: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-24T11:53:38.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:37 smithi137 ceph-mon[47907]: pgmap v490: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-24T11:53:38.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:37 smithi039 ceph-mon[42919]: pgmap v490: 97 pgs: 97 active+clean; 5.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-24T11:53:39.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 855.0 s, 993.1 tps, lat 100.692 ms stddev 45.328 2024-05-24T11:53:40.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:39 smithi053 ceph-mon[47448]: pgmap v491: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-05-24T11:53:40.248 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:39 smithi137 ceph-mon[47907]: pgmap v491: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-05-24T11:53:40.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:39 smithi039 ceph-mon[42919]: pgmap v491: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-05-24T11:53:42.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:41 smithi053 ceph-mon[47448]: pgmap v492: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2024-05-24T11:53:42.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:41 smithi137 ceph-mon[47907]: pgmap v492: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2024-05-24T11:53:42.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:41 smithi039 ceph-mon[42919]: pgmap v492: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2024-05-24T11:53:44.115 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 860.0 s, 1033.2 tps, lat 96.784 ms stddev 38.068 2024-05-24T11:53:44.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:43 smithi053 ceph-mon[47448]: pgmap v493: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-05-24T11:53:44.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:43 smithi137 ceph-mon[47907]: pgmap v493: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-05-24T11:53:44.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:43 smithi039 ceph-mon[42919]: pgmap v493: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-05-24T11:53:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.128+0000 7f11b2c9e700 1 -- 172.21.15.39:0/2722105612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11ac105dd0 msgr2=0x7f11ac1061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.128+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/2722105612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11ac105dd0 0x7f11ac1061b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f11a00098f0 tx=0x7f11a0009c00 comp rx=0 tx=0).stop 2024-05-24T11:53:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.130+0000 7f11b2c9e700 1 -- 172.21.15.39:0/2722105612 shutdown_connections 2024-05-24T11:53:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.130+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/2722105612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11ac0ffdd0 0x7f11ac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.130+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/2722105612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11ac105dd0 0x7f11ac1061b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.130+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/2722105612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11ac0ff410 0x7f11ac0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.130+0000 7f11b2c9e700 1 -- 172.21.15.39:0/2722105612 >> 172.21.15.39:0/2722105612 conn(0x7f11ac0763a0 msgr2=0x7f11ac0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.131+0000 7f11b2c9e700 1 -- 172.21.15.39:0/2722105612 shutdown_connections 2024-05-24T11:53:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.131+0000 7f11b2c9e700 1 -- 172.21.15.39:0/2722105612 wait complete. 2024-05-24T11:53:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.132+0000 7f11b2c9e700 1 Processor -- start 2024-05-24T11:53:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.132+0000 7f11b2c9e700 1 -- start start 2024-05-24T11:53:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.132+0000 7f11b2c9e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11ac0ff410 0x7f11ac1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.132+0000 7f11b2c9e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11ac0ffdd0 0x7f11ac1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.132+0000 7f11b2c9e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11ac105dd0 0x7f11ac19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.132+0000 7f11b2c9e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac19dde0 con 0x7f11ac0ffdd0 2024-05-24T11:53:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.132+0000 7f11b2c9e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac19df50 con 0x7f11ac0ff410 2024-05-24T11:53:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.132+0000 7f11b2c9e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac19e110 con 0x7f11ac105dd0 2024-05-24T11:53:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.133+0000 7f11abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11ac0ffdd0 0x7f11ac1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.133+0000 7f11abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11ac0ffdd0 0x7f11ac1a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51250/0 (socket says 172.21.15.39:51250) 2024-05-24T11:53:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.133+0000 7f11abfff700 1 -- 172.21.15.39:0/1583451664 learned_addr learned my addr 172.21.15.39:0/1583451664 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.133+0000 7f11b123b700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11ac105dd0 0x7f11ac19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.134+0000 7f11b123b700 1 -- 172.21.15.39:0/1583451664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11ac0ff410 msgr2=0x7f11ac1a3640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-24T11:53:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.134+0000 7f11b123b700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11ac0ff410 0x7f11ac1a3640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.134+0000 7f11b123b700 1 -- 172.21.15.39:0/1583451664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11ac0ffdd0 msgr2=0x7f11ac1a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.134+0000 7f11b123b700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11ac0ffdd0 0x7f11ac1a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.134+0000 7f11b123b700 1 -- 172.21.15.39:0/1583451664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0009520 con 0x7f11ac105dd0 2024-05-24T11:53:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.134+0000 7f11b123b700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11ac105dd0 0x7f11ac19d770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f11a400efc0 tx=0x7f11a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.135+0000 7f11a9ffb700 1 -- 172.21.15.39:0/1583451664 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4018070 con 0x7f11ac105dd0 2024-05-24T11:53:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.135+0000 7f11a9ffb700 1 -- 172.21.15.39:0/1583451664 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11a400f040 con 0x7f11ac105dd0 2024-05-24T11:53:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.135+0000 7f11a9ffb700 1 -- 172.21.15.39:0/1583451664 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a400cc70 con 0x7f11ac105dd0 2024-05-24T11:53:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.136+0000 7f11b2c9e700 1 -- 172.21.15.39:0/1583451664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac19e390 con 0x7f11ac105dd0 2024-05-24T11:53:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.136+0000 7f11b2c9e700 1 -- 172.21.15.39:0/1583451664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac106d20 con 0x7f11ac105dd0 2024-05-24T11:53:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.141+0000 7f11bee59b80 1 Processor -- start 2024-05-24T11:53:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.142+0000 7f11a9ffb700 1 -- 172.21.15.39:0/1583451664 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f11a4007500 con 0x7f11ac105dd0 2024-05-24T11:53:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11bee59b80 1 -- start start 2024-05-24T11:53:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb755f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb7568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11bee59b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb6ad0f0 con 0x55ecdb75b950 2024-05-24T11:53:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11bee59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb6aaab0 con 0x55ecdb756480 2024-05-24T11:53:46.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11bee59b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb6a63a0 con 0x55ecdb75c270 2024-05-24T11:53:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11b0a3a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb755f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51252/0 (socket says 172.21.15.39:51252) 2024-05-24T11:53:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11abfff700 1 -- 172.21.15.39:0/2798457686 learned_addr learned my addr 172.21.15.39:0/2798457686 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11b0a3a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb755f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:33522/0 (socket says 172.21.15.39:33522) 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11abfff700 1 --2- 172.21.15.39:0/2798457686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb7568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11a9ffb700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f119c0a1420 0x7f119c0a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11b0a3a700 1 -- 172.21.15.39:0/2798457686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 msgr2=0x55ecdb7568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/2798457686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb7568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11b0a3a700 1 -- 172.21.15.39:0/2798457686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 msgr2=0x55ecdb75bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/2798457686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.143+0000 7f11b0a3a700 1 -- 172.21.15.39:0/2798457686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecdb527350 con 0x55ecdb75c270 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.144+0000 7f11b123b700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f119c0a1420 0x7f119c0a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.144+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/2798457686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb755f40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009230 tx=0x7f11a0009440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df661241ceed6f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.144+0000 7f11a9ffb700 1 -- 172.21.15.39:0/1583451664 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f11a40cbee0 con 0x7f11ac105dd0 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.144+0000 7f1196ffd700 1 -- 172.21.15.39:0/2798457686 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a001c070 con 0x55ecdb75c270 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.144+0000 7f1196ffd700 1 -- 172.21.15.39:0/2798457686 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11a000f460 con 0x55ecdb75c270 2024-05-24T11:53:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.144+0000 7f1196ffd700 1 -- 172.21.15.39:0/2798457686 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a000b8c0 con 0x55ecdb75c270 2024-05-24T11:53:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11bee59b80 1 -- 172.21.15.39:0/2798457686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 msgr2=0x55ecdb755f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11b123b700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f119c0a1420 0x7f119c0a38e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11a40c9720 tx=0x7f11a4095450 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2798457686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb755f40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009230 tx=0x7f11a0009440 comp rx=0 tx=0).stop 2024-05-24T11:53:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11bee59b80 1 -- 172.21.15.39:0/2798457686 shutdown_connections 2024-05-24T11:53:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2798457686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb7568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2798457686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2798457686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb755f40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11bee59b80 1 -- 172.21.15.39:0/2798457686 >> 172.21.15.39:0/2798457686 conn(0x55ecdb6b41b0 msgr2=0x55ecdb750840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11bee59b80 1 -- 172.21.15.39:0/2798457686 shutdown_connections 2024-05-24T11:53:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.145+0000 7f11bee59b80 1 -- 172.21.15.39:0/2798457686 wait complete. 2024-05-24T11:53:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11bee59b80 1 -- start start 2024-05-24T11:53:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11bee59b80 1 Processor -- start 2024-05-24T11:53:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb762220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb75fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11bee59b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb642500 con 0x55ecdb75b950 2024-05-24T11:53:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11bee59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb6421a0 con 0x55ecdb756480 2024-05-24T11:53:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11bee59b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb529640 con 0x55ecdb75c270 2024-05-24T11:53:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11bee59b80 10 client.0 _ll_register_callbacks cb 0x55ecdb64bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.146+0000 7f11abfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb75fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11abfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb75fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:33526/0 (socket says 172.21.15.39:33526) 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11abfff700 1 -- 172.21.15.39:0/2992654065 learned_addr learned my addr 172.21.15.39:0/2992654065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11abfff700 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb762220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11abfff700 1 -- 172.21.15.39:0/2992654065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 msgr2=0x55ecdb762220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11abfff700 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb762220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11abfff700 1 -- 172.21.15.39:0/2992654065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 msgr2=0x55ecdb75f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11abfff700 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11abfff700 1 -- 172.21.15.39:0/2992654065 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac104190 con 0x55ecdb75c270 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.147+0000 7f11abfff700 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb75fb40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f119800f5f0 tx=0x7f1198009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204ea85f16e5ece6 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.148+0000 7f118bfff700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:53:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.148+0000 7f118bfff700 1 -- 172.21.15.39:0/2992654065 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198004030 con 0x55ecdb75c270 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.148+0000 7f11bee59b80 1 -- 172.21.15.39:0/2992654065 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecdb5ff210 con 0x55ecdb75c270 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.148+0000 7f118bfff700 1 -- 172.21.15.39:0/2992654065 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1198004190 con 0x55ecdb75c270 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.148+0000 7f118bfff700 1 -- 172.21.15.39:0/2992654065 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1198004350 con 0x55ecdb75c270 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.148+0000 7f11bee59b80 10 client.26113 fetch_fsmap learned FSMap version 40 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.148+0000 7f11bee59b80 1 -- 172.21.15.39:0/2992654065 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ecdb4d3b00 con 0x55ecdb75c270 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.149+0000 7f118bfff700 1 -- 172.21.15.39:0/2992654065 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ecdb5ff210 con 0x55ecdb75c270 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.149+0000 7f118bfff700 1 -- 172.21.15.39:0/2992654065 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1198022460 con 0x55ecdb75c270 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.149+0000 7f11bee59b80 10 client.26113 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.149+0000 7f11bee59b80 10 client.26113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.149+0000 7f11bee59b80 20 client.26113 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.149+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ecdb76fab0 0x55ecdb771ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.149+0000 7f11bee59b80 4 client.26113 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.149+0000 7f11bee59b80 1 -- 172.21.15.39:0/2992654065 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecdb7257b0 con 0x55ecdb76fab0 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.149+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ecdb76fab0 0x55ecdb771ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.150+0000 7f118bfff700 10 client.26113 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.150+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ecdb76fab0 0x55ecdb771ed0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c3089965c711dc1 server_cookie=e44d23a8f78f862b in_seq=0 out_seq=0 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.150+0000 7f118bfff700 1 -- 172.21.15.39:0/2992654065 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ecdb7257b0 con 0x55ecdb76fab0 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.150+0000 7f118bfff700 10 client.26113 handle_command_reply: tid=0 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 client.26113 shutdown 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 -- 172.21.15.39:0/2992654065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 msgr2=0x55ecdb75fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb75fb40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f119800f5f0 tx=0x7f1198009c90 comp rx=0 tx=0).stop 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 -- 172.21.15.39:0/2992654065 shutdown_connections 2024-05-24T11:53:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb75fb40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ecdb76fab0 0x55ecdb771ed0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb762220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 --2- 172.21.15.39:0/2992654065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb75f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f118bfff700 0 client.26113 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 -- 172.21.15.39:0/2992654065 >> 172.21.15.39:0/2992654065 conn(0x55ecdb6b41b0 msgr2=0x55ecdb75a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 -- 172.21.15.39:0/2992654065 shutdown_connections 2024-05-24T11:53:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 1 -- 172.21.15.39:0/2992654065 wait complete. 2024-05-24T11:53:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.151+0000 7f11bee59b80 20 client.26113 trim_cache size 0 max 16384 2024-05-24T11:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.173+0000 7f11bee59b80 1 Processor -- start 2024-05-24T11:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.173+0000 7f11bee59b80 1 -- start start 2024-05-24T11:53:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.173+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb750910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.173+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb76b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.173+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb76bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.173+0000 7f11bee59b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb752380 con 0x55ecdb75b950 2024-05-24T11:53:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.173+0000 7f11bee59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb76d260 con 0x55ecdb756480 2024-05-24T11:53:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.173+0000 7f11bee59b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb75f360 con 0x55ecdb75c270 2024-05-24T11:53:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11abfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb750910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11abfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb76bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11abfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb750910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:35210/0 (socket says 172.21.15.39:35210) 2024-05-24T11:53:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11abfff700 1 -- 172.21.15.39:0/1055252567 learned_addr learned my addr 172.21.15.39:0/1055252567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/1055252567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb76b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11abfff700 1 -- 172.21.15.39:0/1055252567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 msgr2=0x55ecdb76bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11abfff700 1 --2- 172.21.15.39:0/1055252567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb76bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11abfff700 1 -- 172.21.15.39:0/1055252567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 msgr2=0x55ecdb76b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11abfff700 1 --2- 172.21.15.39:0/1055252567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb76b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.174+0000 7f11abfff700 1 -- 172.21.15.39:0/1055252567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac19e5f0 con 0x55ecdb756480 2024-05-24T11:53:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.175+0000 7f11abfff700 1 --2- 172.21.15.39:0/1055252567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb750910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f119800fc90 tx=0x7f1198010720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e91eb8161b1a888 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.175+0000 7f11977fe700 1 -- 172.21.15.39:0/1055252567 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198022a90 con 0x55ecdb756480 2024-05-24T11:53:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.175+0000 7f11977fe700 1 -- 172.21.15.39:0/1055252567 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1198009310 con 0x55ecdb756480 2024-05-24T11:53:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.175+0000 7f11977fe700 1 -- 172.21.15.39:0/1055252567 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198022650 con 0x55ecdb756480 2024-05-24T11:53:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.176+0000 7f11bee59b80 1 -- 172.21.15.39:0/1055252567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 msgr2=0x55ecdb750910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.176+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1055252567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb750910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f119800fc90 tx=0x7f1198010720 comp rx=0 tx=0).stop 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.176+0000 7f11bee59b80 1 -- 172.21.15.39:0/1055252567 shutdown_connections 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.176+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1055252567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75c270 0x55ecdb76bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.176+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1055252567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb750910 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.176+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1055252567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75b950 0x55ecdb76b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.176+0000 7f11bee59b80 1 -- 172.21.15.39:0/1055252567 >> 172.21.15.39:0/1055252567 conn(0x55ecdb6b41b0 msgr2=0x55ecdb776830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.176+0000 7f11bee59b80 1 -- 172.21.15.39:0/1055252567 shutdown_connections 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.176+0000 7f11bee59b80 1 -- 172.21.15.39:0/1055252567 wait complete. 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11bee59b80 1 -- start start 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11bee59b80 1 Processor -- start 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb7670c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75b950 0x55ecdb75a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11bee59b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75c270 0x55ecdb7593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11bee59b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb75af10 con 0x55ecdb75c270 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11bee59b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb75b080 con 0x55ecdb756480 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11bee59b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecdb759930 con 0x55ecdb75b950 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11bee59b80 10 client.0 _ll_register_callbacks cb 0x55ecdb64bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75c270 0x55ecdb7593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75c270 0x55ecdb7593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51278/0 (socket says 172.21.15.39:51278) 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11abfff700 1 -- 172.21.15.39:0/1279757733 learned_addr learned my addr 172.21.15.39:0/1279757733 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11abfff700 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb7670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.177+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75b950 0x55ecdb75a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f11abfff700 1 -- 172.21.15.39:0/1279757733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75b950 msgr2=0x55ecdb75a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f11abfff700 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75b950 0x55ecdb75a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f11abfff700 1 -- 172.21.15.39:0/1279757733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75c270 msgr2=0x55ecdb7593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f11abfff700 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75c270 0x55ecdb7593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f11abfff700 1 -- 172.21.15.39:0/1279757733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecdb4d3b00 con 0x55ecdb756480 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f11abfff700 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb7670c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f11980095c0 tx=0x7f11980104e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f3b73c99370487e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f118bfff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f118bfff700 1 -- 172.21.15.39:0/1279757733 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198022b00 con 0x55ecdb756480 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f118bfff700 1 -- 172.21.15.39:0/1279757733 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8454+0+0 (secure 0 0 0) 0x7f1198030020 con 0x55ecdb756480 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f11bee59b80 1 -- 172.21.15.39:0/1279757733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11a000f460 con 0x55ecdb756480 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f118bfff700 1 -- 172.21.15.39:0/1279757733 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1198005070 con 0x55ecdb756480 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f11bee59b80 10 client.26210 fetch_fsmap learned FSMap version 40 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.178+0000 7f11bee59b80 1 -- 172.21.15.39:0/1279757733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ecdb66c500 con 0x55ecdb756480 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.179+0000 7f118bfff700 1 -- 172.21.15.39:0/1279757733 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11a000f460 con 0x55ecdb756480 2024-05-24T11:53:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.179+0000 7f118bfff700 1 -- 172.21.15.39:0/1279757733 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1198045020 con 0x55ecdb756480 2024-05-24T11:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.179+0000 7f11bee59b80 10 client.26210 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.179+0000 7f11bee59b80 10 client.26210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.179+0000 7f11bee59b80 20 client.26210 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.179+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ecdb7c6830 0x55ecdb7c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.179+0000 7f11bee59b80 4 client.26210 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.179+0000 7f11bee59b80 1 -- 172.21.15.39:0/1279757733 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecdb69ee60 con 0x55ecdb7c6830 2024-05-24T11:53:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.179+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ecdb7c6830 0x55ecdb7c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.180+0000 7f118bfff700 10 client.26210 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.180+0000 7f11b0a3a700 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ecdb7c6830 0x55ecdb7c8c50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6479df2068698e8e server_cookie=379b72411e670620 in_seq=0 out_seq=0 2024-05-24T11:53:46.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:45 smithi053 ceph-mon[47448]: pgmap v494: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-05-24T11:53:46.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:45 smithi137 ceph-mon[47907]: pgmap v494: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-05-24T11:53:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f118bfff700 1 -- 172.21.15.39:0/1279757733 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecdb69ee60 con 0x55ecdb7c6830 2024-05-24T11:53:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f118bfff700 10 client.26210 handle_command_reply: tid=0 2024-05-24T11:53:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f11bee59b80 1 client.26210 shutdown 2024-05-24T11:53:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f11bee59b80 1 -- 172.21.15.39:0/1279757733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 msgr2=0x55ecdb7670c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb7670c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f11980095c0 tx=0x7f11980104e0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f11bee59b80 1 -- 172.21.15.39:0/1279757733 shutdown_connections 2024-05-24T11:53:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x55ecdb7c6830 0x55ecdb7c8c50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecdb75c270 0x55ecdb7593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ecdb756480 0x55ecdb7670c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.269+0000 7f11bee59b80 1 --2- 172.21.15.39:0/1279757733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ecdb75b950 0x55ecdb75a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.270+0000 7f11bee59b80 1 -- 172.21.15.39:0/1279757733 >> 172.21.15.39:0/1279757733 conn(0x55ecdb6b41b0 msgr2=0x55ecdb7545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.270+0000 7f118bfff700 0 client.26210 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.270+0000 7f11bee59b80 1 -- 172.21.15.39:0/1279757733 shutdown_connections 2024-05-24T11:53:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.270+0000 7f11bee59b80 1 -- 172.21.15.39:0/1279757733 wait complete. 2024-05-24T11:53:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.270+0000 7f11bee59b80 20 client.26210 trim_cache size 0 max 16384 2024-05-24T11:53:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:53:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:53:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:53:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:53:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.270+0000 7f11b2c9e700 1 -- 172.21.15.39:0/1583451664 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f119c0a1420 msgr2=0x7f119c0a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.270+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f119c0a1420 0x7f119c0a38e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11a40c9720 tx=0x7f11a4095450 comp rx=0 tx=0).stop 2024-05-24T11:53:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.270+0000 7f11b2c9e700 1 -- 172.21.15.39:0/1583451664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11ac105dd0 msgr2=0x7f11ac19d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.270+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11ac105dd0 0x7f11ac19d770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f11a400efc0 tx=0x7f11a400c3a0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.271+0000 7f11b2c9e700 1 -- 172.21.15.39:0/1583451664 shutdown_connections 2024-05-24T11:53:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.271+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11ac105dd0 0x7f11ac19d770 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.271+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f119c0a1420 0x7f119c0a38e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.271+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11ac0ff410 0x7f11ac1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.271+0000 7f11b2c9e700 1 --2- 172.21.15.39:0/1583451664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11ac0ffdd0 0x7f11ac1a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.271+0000 7f11b2c9e700 1 -- 172.21.15.39:0/1583451664 >> 172.21.15.39:0/1583451664 conn(0x7f11ac0763a0 msgr2=0x7f11ac1014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.271+0000 7f11b2c9e700 1 -- 172.21.15.39:0/1583451664 shutdown_connections 2024-05-24T11:53:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:46.271+0000 7f11b2c9e700 1 -- 172.21.15.39:0/1583451664 wait complete. 2024-05-24T11:53:46.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:45 smithi039 ceph-mon[42919]: pgmap v494: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-05-24T11:53:48.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:47 smithi053 ceph-mon[47448]: pgmap v495: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-05-24T11:53:48.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:47 smithi137 ceph-mon[47907]: pgmap v495: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-05-24T11:53:48.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:47 smithi039 ceph-mon[42919]: pgmap v495: 97 pgs: 97 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-05-24T11:53:49.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 865.0 s, 1079.4 tps, lat 92.592 ms stddev 34.337 2024-05-24T11:53:50.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:49 smithi053 ceph-mon[47448]: pgmap v496: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:53:50.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:49 smithi137 ceph-mon[47907]: pgmap v496: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:53:50.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:49 smithi039 ceph-mon[42919]: pgmap v496: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2024-05-24T11:53:52.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:51 smithi053 ceph-mon[47448]: pgmap v497: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-05-24T11:53:52.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:51 smithi137 ceph-mon[47907]: pgmap v497: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-05-24T11:53:52.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:51 smithi039 ceph-mon[42919]: pgmap v497: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 16 MiB/s wr, 1.29k op/s 2024-05-24T11:53:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 870.0 s, 994.4 tps, lat 100.415 ms stddev 40.761 2024-05-24T11:53:54.238 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:53 smithi053 ceph-mon[47448]: pgmap v498: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-05-24T11:53:54.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:53 smithi137 ceph-mon[47907]: pgmap v498: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-05-24T11:53:54.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:53 smithi039 ceph-mon[42919]: pgmap v498: 97 pgs: 97 active+clean; 5.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-05-24T11:53:56.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:55 smithi053 ceph-mon[47448]: pgmap v499: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-24T11:53:56.244 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:55 smithi137 ceph-mon[47907]: pgmap v499: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-24T11:53:56.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:55 smithi039 ceph-mon[42919]: pgmap v499: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 18 MiB/s wr, 1.49k op/s 2024-05-24T11:53:58.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:57 smithi053 ceph-mon[47448]: pgmap v500: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 11 MiB/s wr, 982 op/s 2024-05-24T11:53:58.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:57 smithi137 ceph-mon[47907]: pgmap v500: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 11 MiB/s wr, 982 op/s 2024-05-24T11:53:58.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:57 smithi039 ceph-mon[42919]: pgmap v500: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 11 MiB/s wr, 982 op/s 2024-05-24T11:53:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 875.0 s, 1042.8 tps, lat 95.980 ms stddev 38.096 2024-05-24T11:53:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.441+0000 7efc430b1700 1 -- 172.21.15.39:0/3041715022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c0ffda0 msgr2=0x7efc3c100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.442+0000 7efc430b1700 1 --2- 172.21.15.39:0/3041715022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c0ffda0 0x7efc3c100220 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efc2c01a9a0 tx=0x7efc2c01acb0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.442+0000 7efc430b1700 1 -- 172.21.15.39:0/3041715022 shutdown_connections 2024-05-24T11:53:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.442+0000 7efc430b1700 1 --2- 172.21.15.39:0/3041715022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c100760 0x7efc3c10cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.442+0000 7efc430b1700 1 --2- 172.21.15.39:0/3041715022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3c105e60 0x7efc3c106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.442+0000 7efc430b1700 1 --2- 172.21.15.39:0/3041715022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c0ffda0 0x7efc3c100220 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.442+0000 7efc430b1700 1 -- 172.21.15.39:0/3041715022 >> 172.21.15.39:0/3041715022 conn(0x7efc3c076f10 msgr2=0x7efc3c077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.442+0000 7efc430b1700 1 -- 172.21.15.39:0/3041715022 shutdown_connections 2024-05-24T11:53:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.442+0000 7efc430b1700 1 -- 172.21.15.39:0/3041715022 wait complete. 2024-05-24T11:53:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.443+0000 7efc430b1700 1 Processor -- start 2024-05-24T11:53:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.443+0000 7efc430b1700 1 -- start start 2024-05-24T11:53:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.443+0000 7efc430b1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c0ffda0 0x7efc3c1086e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.443+0000 7efc430b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c100760 0x7efc3c108c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.443+0000 7efc430b1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3c105e60 0x7efc3c1b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.443+0000 7efc430b1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1b5ab0 con 0x7efc3c100760 2024-05-24T11:53:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.443+0000 7efc430b1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1b5c20 con 0x7efc3c105e60 2024-05-24T11:53:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.443+0000 7efc430b1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1b5de0 con 0x7efc3c0ffda0 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.444+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c100760 0x7efc3c108c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.444+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c100760 0x7efc3c108c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53136/0 (socket says 172.21.15.39:53136) 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.444+0000 7efc3bfff700 1 -- 172.21.15.39:0/1910622116 learned_addr learned my addr 172.21.15.39:0/1910622116 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.444+0000 7efc40e4d700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c0ffda0 0x7efc3c1086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.444+0000 7efc3bfff700 1 -- 172.21.15.39:0/1910622116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c0ffda0 msgr2=0x7efc3c1086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.444+0000 7efc3bfff700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c0ffda0 0x7efc3c1086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.444+0000 7efc3bfff700 1 -- 172.21.15.39:0/1910622116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3c105e60 msgr2=0x7efc3c1b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.444+0000 7efc3bfff700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3c105e60 0x7efc3c1b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.444+0000 7efc3bfff700 1 -- 172.21.15.39:0/1910622116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c01a600 con 0x7efc3c100760 2024-05-24T11:53:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.445+0000 7efc3bfff700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c100760 0x7efc3c108c20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0055e0 tx=0x7efc2c0056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.445+0000 7efc39ffb700 1 -- 172.21.15.39:0/1910622116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c004c90 con 0x7efc3c100760 2024-05-24T11:53:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.445+0000 7efc39ffb700 1 -- 172.21.15.39:0/1910622116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc2c004df0 con 0x7efc3c100760 2024-05-24T11:53:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.445+0000 7efc39ffb700 1 -- 172.21.15.39:0/1910622116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00c620 con 0x7efc3c100760 2024-05-24T11:53:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.445+0000 7efc430b1700 1 -- 172.21.15.39:0/1910622116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c1b6060 con 0x7efc3c100760 2024-05-24T11:53:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.447+0000 7efc430b1700 1 -- 172.21.15.39:0/1910622116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c068460 con 0x7efc3c100760 2024-05-24T11:53:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.448+0000 7efc39ffb700 1 -- 172.21.15.39:0/1910622116 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efc2c00c780 con 0x7efc3c100760 2024-05-24T11:53:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.449+0000 7efc39ffb700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efc280a5760 0x7efc280a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.449+0000 7efc39ffb700 1 -- 172.21.15.39:0/1910622116 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8454+0+0 (secure 0 0 0) 0x7efc2c09b640 con 0x7efc3c100760 2024-05-24T11:53:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.449+0000 7efc40e4d700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efc280a5760 0x7efc280a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.450+0000 7efc40e4d700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efc280a5760 0x7efc280a7c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc24009910 tx=0x7efc24008040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.452+0000 7efc4f26cb80 1 Processor -- start 2024-05-24T11:53:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.452+0000 7efc4f26cb80 1 -- start start 2024-05-24T11:53:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.452+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8adf160 0x5636c8adf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.452+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ad9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.452+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8ad9c90 0x5636c8ada0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.452+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c8a280f0 con 0x5636c8ad9c90 2024-05-24T11:53:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.452+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c8a25ab0 con 0x5636c8adfa80 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.452+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c8a213a0 con 0x5636c8adf160 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ad9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ad9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:32900/0 (socket says 172.21.15.39:32900) 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc3bfff700 1 -- 172.21.15.39:0/3225398125 learned_addr learned my addr 172.21.15.39:0/3225398125 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc40e4d700 1 --2- 172.21.15.39:0/3225398125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8adf160 0x5636c8adf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc3bfff700 1 -- 172.21.15.39:0/3225398125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8adf160 msgr2=0x5636c8adf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc3bfff700 1 --2- 172.21.15.39:0/3225398125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8adf160 0x5636c8adf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc3bfff700 1 -- 172.21.15.39:0/3225398125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8ad9c90 msgr2=0x5636c8ada0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc3bfff700 1 --2- 172.21.15.39:0/3225398125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8ad9c90 0x5636c8ada0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc3bfff700 1 -- 172.21.15.39:0/3225398125 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636c88a2350 con 0x5636c8adfa80 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc3bfff700 1 --2- 172.21.15.39:0/3225398125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ad9750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efc2c01a1c0 tx=0x7efc2c09aa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99896996b2c333af server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.453+0000 7efc32ffd700 1 -- 172.21.15.39:0/3225398125 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c09af40 con 0x5636c8adfa80 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.454+0000 7efc32ffd700 1 -- 172.21.15.39:0/3225398125 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc2c09ed70 con 0x5636c8adfa80 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.454+0000 7efc32ffd700 1 -- 172.21.15.39:0/3225398125 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0cb770 con 0x5636c8adfa80 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.454+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3225398125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 msgr2=0x5636c8ad9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.454+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/3225398125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ad9750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efc2c01a1c0 tx=0x7efc2c09aa80 comp rx=0 tx=0).stop 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.454+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3225398125 shutdown_connections 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.454+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/3225398125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8adf160 0x5636c8adf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.454+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/3225398125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8ad9c90 0x5636c8ada0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.454+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/3225398125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ad9750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.454+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3225398125 >> 172.21.15.39:0/3225398125 conn(0x5636c8a2f1b0 msgr2=0x5636c8acb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.458+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3225398125 shutdown_connections 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.458+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3225398125 wait complete. 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.459+0000 7efc4f26cb80 1 -- start start 2024-05-24T11:53:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.459+0000 7efc4f26cb80 1 Processor -- start 2024-05-24T11:53:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8ae5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adf160 0x5636c8ae2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ae3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c89bd500 con 0x5636c8adf160 2024-05-24T11:53:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c89bd1a0 con 0x5636c8adfa80 2024-05-24T11:53:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c88a4640 con 0x5636c8ad9c90 2024-05-24T11:53:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4f26cb80 10 client.0 _ll_register_callbacks cb 0x5636c89c6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4164e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ae3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4164e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ae3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:32914/0 (socket says 172.21.15.39:32914) 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4164e700 1 -- 172.21.15.39:0/2916703487 learned_addr learned my addr 172.21.15.39:0/2916703487 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4164e700 1 -- 172.21.15.39:0/2916703487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 msgr2=0x5636c8ae5ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4164e700 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8ae5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4164e700 1 -- 172.21.15.39:0/2916703487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adf160 msgr2=0x5636c8ae2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4164e700 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adf160 0x5636c8ae2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.460+0000 7efc4164e700 1 -- 172.21.15.39:0/2916703487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c068460 con 0x5636c8adfa80 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.461+0000 7efc4164e700 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ae3530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc3400ebf0 tx=0x7efc3400c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d916ccf86b7dd880 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.461+0000 7efc1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.461+0000 7efc1bfff700 1 -- 172.21.15.39:0/2916703487 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34018070 con 0x5636c8adfa80 2024-05-24T11:53:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.461+0000 7efc1bfff700 1 -- 172.21.15.39:0/2916703487 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8454+0+0 (secure 0 0 0) 0x7efc340092e0 con 0x5636c8adfa80 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.461+0000 7efc4f26cb80 1 -- 172.21.15.39:0/2916703487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636c897a210 con 0x5636c8adfa80 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.462+0000 7efc1bfff700 1 -- 172.21.15.39:0/2916703487 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc34009e70 con 0x5636c8adfa80 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.462+0000 7efc4f26cb80 10 client.26216 fetch_fsmap learned FSMap version 40 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.462+0000 7efc4f26cb80 1 -- 172.21.15.39:0/2916703487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5636c884eb00 con 0x5636c8adfa80 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.462+0000 7efc1bfff700 1 -- 172.21.15.39:0/2916703487 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5636c897a210 con 0x5636c8adfa80 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.462+0000 7efc1bfff700 1 -- 172.21.15.39:0/2916703487 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efc3401b020 con 0x5636c8adfa80 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.462+0000 7efc4f26cb80 10 client.26216 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.462+0000 7efc4f26cb80 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.463+0000 7efc4f26cb80 20 client.26216 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.463+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5636c8ad1ec0 0x5636c8ad42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.463+0000 7efc4f26cb80 4 client.26216 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:53:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.463+0000 7efc4f26cb80 1 -- 172.21.15.39:0/2916703487 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636c8aa07b0 con 0x5636c8ad1ec0 2024-05-24T11:53:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.463+0000 7efc40e4d700 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5636c8ad1ec0 0x5636c8ad42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.464+0000 7efc40e4d700 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5636c8ad1ec0 0x5636c8ad42e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9cad6f340a42cc7 server_cookie=2e70b72a9a5064b8 in_seq=0 out_seq=0 2024-05-24T11:53:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.464+0000 7efc1bfff700 10 client.26216 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.464+0000 7efc1bfff700 1 -- 172.21.15.39:0/2916703487 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636c8aa07b0 con 0x5636c8ad1ec0 2024-05-24T11:53:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.464+0000 7efc1bfff700 10 client.26216 handle_command_reply: tid=0 2024-05-24T11:53:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.464+0000 7efc4f26cb80 1 client.26216 shutdown 2024-05-24T11:53:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.468+0000 7efc4f26cb80 1 -- 172.21.15.39:0/2916703487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 msgr2=0x5636c8ae3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.468+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ae3530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc3400ebf0 tx=0x7efc3400c0c0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.468+0000 7efc4f26cb80 1 -- 172.21.15.39:0/2916703487 shutdown_connections 2024-05-24T11:53:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.468+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8ae5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.468+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5636c8ad1ec0 0x5636c8ad42e0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.468+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8ae3530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.468+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/2916703487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adf160 0x5636c8ae2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.468+0000 7efc4f26cb80 1 -- 172.21.15.39:0/2916703487 >> 172.21.15.39:0/2916703487 conn(0x5636c8a2f1b0 msgr2=0x5636c8addcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.468+0000 7efc1bfff700 0 client.26216 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:53:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.470+0000 7efc4f26cb80 1 -- 172.21.15.39:0/2916703487 shutdown_connections 2024-05-24T11:53:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.471+0000 7efc4f26cb80 1 -- 172.21.15.39:0/2916703487 wait complete. 2024-05-24T11:53:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.471+0000 7efc4f26cb80 20 client.26216 trim_cache size 0 max 16384 2024-05-24T11:53:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.495+0000 7efc4f26cb80 1 Processor -- start 2024-05-24T11:53:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc4f26cb80 1 -- start start 2024-05-24T11:53:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8ada070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adf160 0x5636c8aeec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adfa80 0x5636c8ae2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c8ae3b40 con 0x5636c8adfa80 2024-05-24T11:53:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c8ae43f0 con 0x5636c8adf160 2024-05-24T11:53:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c8acd530 con 0x5636c8ad9c90 2024-05-24T11:53:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc40e4d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8ada070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc3bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adf160 0x5636c8aeec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc40e4d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8ada070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:36914/0 (socket says 172.21.15.39:36914) 2024-05-24T11:53:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc40e4d700 1 -- 172.21.15.39:0/3287370682 learned_addr learned my addr 172.21.15.39:0/3287370682 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc40e4d700 1 -- 172.21.15.39:0/3287370682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adf160 msgr2=0x5636c8aeec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc40e4d700 1 --2- 172.21.15.39:0/3287370682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adf160 0x5636c8aeec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc40e4d700 1 -- 172.21.15.39:0/3287370682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adfa80 msgr2=0x5636c8ae2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc40e4d700 1 --2- 172.21.15.39:0/3287370682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adfa80 0x5636c8ae2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.496+0000 7efc40e4d700 1 -- 172.21.15.39:0/3287370682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636c89e7500 con 0x5636c8ad9c90 2024-05-24T11:53:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.497+0000 7efc40e4d700 1 --2- 172.21.15.39:0/3287370682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8ada070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc24007fd0 tx=0x7efc24013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f4856164f0cf66 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.497+0000 7efc337fe700 1 -- 172.21.15.39:0/3287370682 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2400dd10 con 0x5636c8ad9c90 2024-05-24T11:53:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.497+0000 7efc337fe700 1 -- 172.21.15.39:0/3287370682 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc2400de70 con 0x5636c8ad9c90 2024-05-24T11:53:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.497+0000 7efc337fe700 1 -- 172.21.15.39:0/3287370682 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24014720 con 0x5636c8ad9c90 2024-05-24T11:53:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.498+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3287370682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 msgr2=0x5636c8ada070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:53:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.498+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/3287370682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8ada070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc24007fd0 tx=0x7efc24013000 comp rx=0 tx=0).stop 2024-05-24T11:53:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.499+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3287370682 shutdown_connections 2024-05-24T11:53:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.499+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/3287370682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8ada070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.499+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/3287370682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adfa80 0x5636c8ae2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.499+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/3287370682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adf160 0x5636c8aeec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.499+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3287370682 >> 172.21.15.39:0/3287370682 conn(0x5636c8a2f1b0 msgr2=0x5636c8ae2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:53:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.499+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3287370682 shutdown_connections 2024-05-24T11:53:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.499+0000 7efc4f26cb80 1 -- 172.21.15.39:0/3287370682 wait complete. 2024-05-24T11:53:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.500+0000 7efc4f26cb80 1 -- start start 2024-05-24T11:53:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.500+0000 7efc4f26cb80 1 Processor -- start 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8aea160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adf160 0x5636c8aea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc4f26cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8add000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c8add710 con 0x5636c8adf160 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c8add880 con 0x5636c8adfa80 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc4f26cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c8adda40 con 0x5636c8ad9c90 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc4f26cb80 10 client.0 _ll_register_callbacks cb 0x5636c89c6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc40e4d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8aea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc40e4d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8aea160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:36922/0 (socket says 172.21.15.39:36922) 2024-05-24T11:53:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc40e4d700 1 -- 172.21.15.39:0/1795093335 learned_addr learned my addr 172.21.15.39:0/1795093335 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc40e4d700 1 -- 172.21.15.39:0/1795093335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 msgr2=0x5636c8add000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc4164e700 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8add000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc40e4d700 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8add000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc40e4d700 1 -- 172.21.15.39:0/1795093335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adf160 msgr2=0x5636c8aea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc40e4d700 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adf160 0x5636c8aea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc40e4d700 1 -- 172.21.15.39:0/1795093335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2401c040 con 0x5636c8ad9c90 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc4164e700 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8add000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc40e4d700 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8aea160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc24004270 tx=0x7efc24004350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95589508ffe11456 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.501+0000 7efc1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.502+0000 7efc1bfff700 1 -- 172.21.15.39:0/1795093335 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24003740 con 0x5636c8ad9c90 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.503+0000 7efc1bfff700 1 -- 172.21.15.39:0/1795093335 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8454+0+0 (secure 0 0 0) 0x7efc240038a0 con 0x5636c8ad9c90 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.503+0000 7efc4f26cb80 1 -- 172.21.15.39:0/1795093335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc2c09ed70 con 0x5636c8ad9c90 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.503+0000 7efc1bfff700 1 -- 172.21.15.39:0/1795093335 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc24003a60 con 0x5636c8ad9c90 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.503+0000 7efc4f26cb80 10 client.26134 fetch_fsmap learned FSMap version 40 2024-05-24T11:53:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.503+0000 7efc4f26cb80 1 -- 172.21.15.39:0/1795093335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5636c88a6c70 con 0x5636c8ad9c90 2024-05-24T11:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.504+0000 7efc1bfff700 1 -- 172.21.15.39:0/1795093335 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc2c09ed70 con 0x5636c8ad9c90 2024-05-24T11:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.504+0000 7efc1bfff700 1 -- 172.21.15.39:0/1795093335 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efc24024460 con 0x5636c8ad9c90 2024-05-24T11:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.504+0000 7efc4f26cb80 10 client.26134 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.504+0000 7efc4f26cb80 10 client.26134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.504+0000 7efc4f26cb80 20 client.26134 populate_metadata read hostname 'smithi039' 2024-05-24T11:53:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.504+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5636c8b52b20 0x5636c8b54f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:53:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.504+0000 7efc4f26cb80 4 client.26134 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:53:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.504+0000 7efc4f26cb80 1 -- 172.21.15.39:0/1795093335 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5636c8a19e60 con 0x5636c8b52b20 2024-05-24T11:53:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.504+0000 7efc3bfff700 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5636c8b52b20 0x5636c8b54f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:53:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.506+0000 7efc3bfff700 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5636c8b52b20 0x5636c8b54f40 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388559ab936f6b59 server_cookie=bbcc2b16c3140779 in_seq=0 out_seq=0 2024-05-24T11:53:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:53:59.506+0000 7efc1bfff700 10 client.26134 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:00.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:53:59 smithi053 ceph-mon[47448]: pgmap v501: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:54:00.246 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:53:59 smithi137 ceph-mon[47907]: pgmap v501: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:54:00.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:53:59 smithi039 ceph-mon[42919]: pgmap v501: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 14 MiB/s wr, 1.27k op/s 2024-05-24T11:54:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.577+0000 7efc1bfff700 1 -- 172.21.15.39:0/1795093335 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5636c8a19e60 con 0x5636c8b52b20 2024-05-24T11:54:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.577+0000 7efc1bfff700 10 client.26134 handle_command_reply: tid=0 2024-05-24T11:54:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.577+0000 7efc4f26cb80 1 client.26134 shutdown 2024-05-24T11:54:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.578+0000 7efc4f26cb80 1 -- 172.21.15.39:0/1795093335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 msgr2=0x5636c8aea160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.578+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8aea160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc24004270 tx=0x7efc24004350 comp rx=0 tx=0).stop 2024-05-24T11:54:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.579+0000 7efc4f26cb80 1 -- 172.21.15.39:0/1795093335 shutdown_connections 2024-05-24T11:54:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.579+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636c8ad9c90 0x5636c8aea160 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.579+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x5636c8b52b20 0x5636c8b54f40 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.579+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636c8adfa80 0x5636c8add000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.579+0000 7efc4f26cb80 1 --2- 172.21.15.39:0/1795093335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636c8adf160 0x5636c8aea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.579+0000 7efc1bfff700 0 client.26134 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.579+0000 7efc4f26cb80 1 -- 172.21.15.39:0/1795093335 >> 172.21.15.39:0/1795093335 conn(0x5636c8a2f1b0 msgr2=0x5636c8ad8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.579+0000 7efc4f26cb80 1 -- 172.21.15.39:0/1795093335 shutdown_connections 2024-05-24T11:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.579+0000 7efc4f26cb80 1 -- 172.21.15.39:0/1795093335 wait complete. 2024-05-24T11:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.580+0000 7efc4f26cb80 20 client.26134 trim_cache size 0 max 16384 2024-05-24T11:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.580+0000 7efc430b1700 1 -- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efc280a5760 msgr2=0x7efc280a7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.580+0000 7efc430b1700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efc280a5760 0x7efc280a7c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc24009910 tx=0x7efc24008040 comp rx=0 tx=0).stop 2024-05-24T11:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.580+0000 7efc430b1700 1 -- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c100760 msgr2=0x7efc3c108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.580+0000 7efc430b1700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c100760 0x7efc3c108c20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0055e0 tx=0x7efc2c0056c0 comp rx=0 tx=0).stop 2024-05-24T11:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.581+0000 7efc430b1700 1 -- 172.21.15.39:0/1910622116 shutdown_connections 2024-05-24T11:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.581+0000 7efc430b1700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc3c0ffda0 0x7efc3c1086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.581+0000 7efc430b1700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7efc280a5760 0x7efc280a7c20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.581+0000 7efc430b1700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc3c105e60 0x7efc3c1b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.581+0000 7efc430b1700 1 --2- 172.21.15.39:0/1910622116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c100760 0x7efc3c108c20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.581+0000 7efc430b1700 1 -- 172.21.15.39:0/1910622116 >> 172.21.15.39:0/1910622116 conn(0x7efc3c076f10 msgr2=0x7efc3c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.581+0000 7efc430b1700 1 -- 172.21.15.39:0/1910622116 shutdown_connections 2024-05-24T11:54:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:00.581+0000 7efc430b1700 1 -- 172.21.15.39:0/1910622116 wait complete. 2024-05-24T11:54:02.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:01 smithi053 ceph-mon[47448]: pgmap v502: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:54:02.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:01 smithi137 ceph-mon[47907]: pgmap v502: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:54:02.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:01 smithi039 ceph-mon[42919]: pgmap v502: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-05-24T11:54:04.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 880.0 s, 1004.8 tps, lat 99.641 ms stddev 44.455 2024-05-24T11:54:04.239 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:03 smithi053 ceph-mon[47448]: pgmap v503: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-05-24T11:54:04.321 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:03 smithi137 ceph-mon[47907]: pgmap v503: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-05-24T11:54:04.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:03 smithi039 ceph-mon[42919]: pgmap v503: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-05-24T11:54:06.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:05 smithi053 ceph-mon[47448]: pgmap v504: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2024-05-24T11:54:06.243 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:05 smithi137 ceph-mon[47907]: pgmap v504: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2024-05-24T11:54:06.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:05 smithi039 ceph-mon[42919]: pgmap v504: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 16 MiB/s wr, 1.46k op/s 2024-05-24T11:54:08.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:08 smithi039 ceph-mon[42919]: pgmap v505: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 11 MiB/s wr, 985 op/s 2024-05-24T11:54:08.493 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:08 smithi053 ceph-mon[47448]: pgmap v505: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 11 MiB/s wr, 985 op/s 2024-05-24T11:54:08.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:08 smithi137 ceph-mon[47907]: pgmap v505: 97 pgs: 97 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 11 MiB/s wr, 985 op/s 2024-05-24T11:54:09.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 885.0 s, 916.1 tps, lat 108.785 ms stddev 87.825 2024-05-24T11:54:09.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:09 smithi039 ceph-mon[42919]: osdmap e106: 12 total, 12 up, 12 in 2024-05-24T11:54:09.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:09 smithi053 ceph-mon[47448]: osdmap e106: 12 total, 12 up, 12 in 2024-05-24T11:54:09.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:09 smithi137 ceph-mon[47907]: osdmap e106: 12 total, 12 up, 12 in 2024-05-24T11:54:10.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:10 smithi039 ceph-mon[42919]: pgmap v507: 97 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 74 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:54:10.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:10 smithi053 ceph-mon[47448]: pgmap v507: 97 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 74 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:54:10.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:10 smithi137 ceph-mon[47907]: pgmap v507: 97 pgs: 8 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 74 active+clean; 5.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 14 MiB/s wr, 1.21k op/s 2024-05-24T11:54:12.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:12 smithi039 ceph-mon[42919]: pgmap v508: 97 pgs: 13 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 59 active+clean; 5.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:54:12.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:12 smithi053 ceph-mon[47448]: pgmap v508: 97 pgs: 13 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 59 active+clean; 5.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:54:12.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:12 smithi137 ceph-mon[47907]: pgmap v508: 97 pgs: 13 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 59 active+clean; 5.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:54:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.755+0000 7f35b3613700 1 -- 172.21.15.39:0/3077244445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac069190 msgr2=0x7f35ac069570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.755+0000 7f35b3613700 1 --2- 172.21.15.39:0/3077244445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac069190 0x7f35ac069570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35a80098f0 tx=0x7f35a8009c00 comp rx=0 tx=0).stop 2024-05-24T11:54:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.756+0000 7f35b3613700 1 -- 172.21.15.39:0/3077244445 shutdown_connections 2024-05-24T11:54:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.756+0000 7f35b3613700 1 --2- 172.21.15.39:0/3077244445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac10d8f0 0x7f35ac10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.756+0000 7f35b3613700 1 --2- 172.21.15.39:0/3077244445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac069190 0x7f35ac069570 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35a80098f0 tx=0x7f35a8009c00 comp rx=0 tx=0).stop 2024-05-24T11:54:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.756+0000 7f35b3613700 1 --2- 172.21.15.39:0/3077244445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ac069ab0 0x7f35ac10d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.756+0000 7f35b3613700 1 -- 172.21.15.39:0/3077244445 >> 172.21.15.39:0/3077244445 conn(0x7f35ac076f40 msgr2=0x7f35ac077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.756+0000 7f35b3613700 1 -- 172.21.15.39:0/3077244445 shutdown_connections 2024-05-24T11:54:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.756+0000 7f35b3613700 1 -- 172.21.15.39:0/3077244445 wait complete. 2024-05-24T11:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b3613700 1 Processor -- start 2024-05-24T11:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b3613700 1 -- start start 2024-05-24T11:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b3613700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ac069ab0 0x7f35ac10ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b3613700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac10d8f0 0x7f35ac10bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b3613700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac10c5f0 0x7f35ac1a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b3613700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1aa150 con 0x7f35ac069ab0 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b3613700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac06a9f0 con 0x7f35ac10c5f0 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b3613700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac06abb0 con 0x7f35ac10d8f0 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b0bae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac10d8f0 0x7f35ac10bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b1bb0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac10c5f0 0x7f35ac1a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b13af700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ac069ab0 0x7f35ac10ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b1bb0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac10c5f0 0x7f35ac1a9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:56036/0 (socket says 172.21.15.39:56036) 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b13af700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ac069ab0 0x7f35ac10ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36432/0 (socket says 172.21.15.39:36432) 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.757+0000 7f35b1bb0700 1 -- 172.21.15.39:0/2823253629 learned_addr learned my addr 172.21.15.39:0/2823253629 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35b0bae700 1 -- 172.21.15.39:0/2823253629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac10c5f0 msgr2=0x7f35ac1a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35b0bae700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac10c5f0 0x7f35ac1a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35b0bae700 1 -- 172.21.15.39:0/2823253629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ac069ab0 msgr2=0x7f35ac10ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35b0bae700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ac069ab0 0x7f35ac10ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35b0bae700 1 -- 172.21.15.39:0/2823253629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0095f0 con 0x7f35ac10d8f0 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35b1bb0700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac10c5f0 0x7f35ac1a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35b0bae700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac10d8f0 0x7f35ac10bf50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f359c00eb90 tx=0x7f359c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35a27fc700 1 -- 172.21.15.39:0/2823253629 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c009970 con 0x7f35ac10d8f0 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35a27fc700 1 -- 172.21.15.39:0/2823253629 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f359c0044b0 con 0x7f35ac10d8f0 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35a27fc700 1 -- 172.21.15.39:0/2823253629 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c003d70 con 0x7f35ac10d8f0 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.758+0000 7f35b3613700 1 -- 172.21.15.39:0/2823253629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a8009520 con 0x7f35ac10d8f0 2024-05-24T11:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.759+0000 7f35b3613700 1 -- 172.21.15.39:0/2823253629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac06b1c0 con 0x7f35ac10d8f0 2024-05-24T11:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.763+0000 7f35bf7ceb80 1 Processor -- start 2024-05-24T11:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.763+0000 7f35a27fc700 1 -- 172.21.15.39:0/2823253629 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f359c003ef0 con 0x7f35ac10d8f0 2024-05-24T11:54:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.764+0000 7f35a27fc700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f35980a1390 0x7f35980a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.764+0000 7f35a27fc700 1 -- 172.21.15.39:0/2823253629 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f359c014070 con 0x7f35ac10d8f0 2024-05-24T11:54:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35bf7ceb80 1 -- start start 2024-05-24T11:54:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a2950 0x556d331a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a3270 0x556d3319cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d3319d480 0x556d3319d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d330f40f0 con 0x556d3319d480 2024-05-24T11:54:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d330f1ab0 con 0x556d331a2950 2024-05-24T11:54:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d330ed3a0 con 0x556d331a3270 2024-05-24T11:54:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35b0bae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a3270 0x556d3319cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35b0bae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a3270 0x556d3319cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:43054/0 (socket says 172.21.15.39:43054) 2024-05-24T11:54:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35b0bae700 1 -- 172.21.15.39:0/1487769507 learned_addr learned my addr 172.21.15.39:0/1487769507 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.765+0000 7f35b13af700 1 --2- 172.21.15.39:0/1487769507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a2950 0x556d331a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35b0bae700 1 -- 172.21.15.39:0/1487769507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a2950 msgr2=0x556d331a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35b0bae700 1 --2- 172.21.15.39:0/1487769507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a2950 0x556d331a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35b0bae700 1 -- 172.21.15.39:0/1487769507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d3319d480 msgr2=0x556d3319d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35b0bae700 1 --2- 172.21.15.39:0/1487769507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d3319d480 0x556d3319d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35b0bae700 1 -- 172.21.15.39:0/1487769507 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d32f6e350 con 0x556d331a3270 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35b0bae700 1 --2- 172.21.15.39:0/1487769507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a3270 0x556d3319cf40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f359c00c5b0 tx=0x7f359c09a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50811c7dd6e80f99 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35977fe700 1 -- 172.21.15.39:0/1487769507 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c016330 con 0x556d331a3270 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35977fe700 1 -- 172.21.15.39:0/1487769507 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f359c09a900 con 0x556d331a3270 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35977fe700 1 -- 172.21.15.39:0/1487769507 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c096910 con 0x556d331a3270 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.766+0000 7f35b1bb0700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f35980a1390 0x7f35980a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/1487769507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a3270 msgr2=0x556d3319cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/1487769507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a3270 0x556d3319cf40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f359c00c5b0 tx=0x7f359c09a0a0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35b1bb0700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f35980a1390 0x7f35980a3850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35ac10e8b0 tx=0x7f35a4009920 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/1487769507 shutdown_connections 2024-05-24T11:54:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/1487769507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d3319d480 0x556d3319d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/1487769507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a2950 0x556d331a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/1487769507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a3270 0x556d3319cf40 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f359c00c5b0 tx=0x7f359c09a0a0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/1487769507 >> 172.21.15.39:0/1487769507 conn(0x556d330fb1b0 msgr2=0x556d33197840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/1487769507 shutdown_connections 2024-05-24T11:54:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.767+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/1487769507 wait complete. 2024-05-24T11:54:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.768+0000 7f35bf7ceb80 1 -- start start 2024-05-24T11:54:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.768+0000 7f35bf7ceb80 1 Processor -- start 2024-05-24T11:54:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.768+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d331a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a2950 0x556d331a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a7270 0x556d331b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d33089500 con 0x556d331a7270 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d330891a0 con 0x556d3319d480 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d32f70640 con 0x556d331a2950 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35bf7ceb80 10 client.0 _ll_register_callbacks cb 0x556d33092ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b13af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d331a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b13af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d331a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:56060/0 (socket says 172.21.15.39:56060) 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b13af700 1 -- 172.21.15.39:0/2480448440 learned_addr learned my addr 172.21.15.39:0/2480448440 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b0bae700 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a2950 0x556d331a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b1bb0700 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a7270 0x556d331b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b13af700 1 -- 172.21.15.39:0/2480448440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a2950 msgr2=0x556d331a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b13af700 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a2950 0x556d331a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b13af700 1 -- 172.21.15.39:0/2480448440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a7270 msgr2=0x556d331b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b13af700 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a7270 0x556d331b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.769+0000 7f35b13af700 1 -- 172.21.15.39:0/2480448440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac06b1c0 con 0x556d3319d480 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.770+0000 7f35b13af700 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d331a67f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f35ac078220 tx=0x7f35a800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c3d3cf35ffc607 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.770+0000 7f3594ff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.770+0000 7f3594ff9700 1 -- 172.21.15.39:0/2480448440 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a801c070 con 0x556d3319d480 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.770+0000 7f3594ff9700 1 -- 172.21.15.39:0/2480448440 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f35a800f8b0 con 0x556d3319d480 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.770+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/2480448440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d33046210 con 0x556d3319d480 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.770+0000 7f3594ff9700 1 -- 172.21.15.39:0/2480448440 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a800bec0 con 0x556d3319d480 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.770+0000 7f35bf7ceb80 10 client.26231 fetch_fsmap learned FSMap version 40 2024-05-24T11:54:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.770+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/2480448440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556d32f1ab00 con 0x556d3319d480 2024-05-24T11:54:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.771+0000 7f3594ff9700 1 -- 172.21.15.39:0/2480448440 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556d33046210 con 0x556d3319d480 2024-05-24T11:54:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.771+0000 7f3594ff9700 1 -- 172.21.15.39:0/2480448440 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f35a8015020 con 0x556d3319d480 2024-05-24T11:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.773+0000 7f35bf7ceb80 10 client.26231 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.773+0000 7f35bf7ceb80 10 client.26231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.773+0000 7f35bf7ceb80 20 client.26231 populate_metadata read hostname 'smithi039' 2024-05-24T11:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.773+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556d331b7200 0x556d331b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.773+0000 7f35bf7ceb80 4 client.26231 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.773+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/2480448440 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d3316c7b0 con 0x556d331b7200 2024-05-24T11:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.773+0000 7f35b0bae700 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556d331b7200 0x556d331b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.773+0000 7f35b0bae700 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556d331b7200 0x556d331b9620 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13c3d1e26b01e8e6 server_cookie=9b3a49b1b291ce39 in_seq=0 out_seq=0 2024-05-24T11:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.774+0000 7f3594ff9700 10 client.26231 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.774+0000 7f3594ff9700 1 -- 172.21.15.39:0/2480448440 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d3316c7b0 con 0x556d331b7200 2024-05-24T11:54:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.774+0000 7f3594ff9700 10 client.26231 handle_command_reply: tid=0 2024-05-24T11:54:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.774+0000 7f35bf7ceb80 1 client.26231 shutdown 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.774+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/2480448440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 msgr2=0x556d331a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.774+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d331a67f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f35ac078220 tx=0x7f35a800f690 comp rx=0 tx=0).stop 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/2480448440 shutdown_connections 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556d331b7200 0x556d331b9620 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a7270 0x556d331b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d331a67f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f3594ff9700 0 client.26231 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/2480448440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a2950 0x556d331a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/2480448440 >> 172.21.15.39:0/2480448440 conn(0x556d330fb1b0 msgr2=0x556d331a14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/2480448440 shutdown_connections 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/2480448440 wait complete. 2024-05-24T11:54:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.775+0000 7f35bf7ceb80 20 client.26231 trim_cache size 0 max 16384 2024-05-24T11:54:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.796+0000 7f35bf7ceb80 1 Processor -- start 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.796+0000 7f35bf7ceb80 1 -- start start 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d3319af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a2950 0x556d33197910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a7270 0x556d33199520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d331a7090 con 0x556d331a2950 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3319c380 con 0x556d3319d480 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d331a9920 con 0x556d331a7270 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35b13af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d3319af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35b13af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d3319af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:56068/0 (socket says 172.21.15.39:56068) 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35b13af700 1 -- 172.21.15.39:0/821861330 learned_addr learned my addr 172.21.15.39:0/821861330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35b0bae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a2950 0x556d33197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35b1bb0700 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a7270 0x556d33199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.797+0000 7f35b0bae700 1 -- 172.21.15.39:0/821861330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a7270 msgr2=0x556d33199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f35b0bae700 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a7270 0x556d33199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f35b0bae700 1 -- 172.21.15.39:0/821861330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 msgr2=0x556d3319af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f35b0bae700 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d3319af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f35b0bae700 1 -- 172.21.15.39:0/821861330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d330b3500 con 0x556d331a2950 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f35b1bb0700 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a7270 0x556d33199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f35b13af700 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d3319af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f35b0bae700 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a2950 0x556d33197910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f359c09e1f0 tx=0x7f359c099e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355d2c369a8f654a server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f3597fff700 1 -- 172.21.15.39:0/821861330 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c0a2680 con 0x556d331a2950 2024-05-24T11:54:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f3597fff700 1 -- 172.21.15.39:0/821861330 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f359c082070 con 0x556d331a2950 2024-05-24T11:54:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.798+0000 7f3597fff700 1 -- 172.21.15.39:0/821861330 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c096910 con 0x556d331a2950 2024-05-24T11:54:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.799+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/821861330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a2950 msgr2=0x556d33197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.799+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a2950 0x556d33197910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f359c09e1f0 tx=0x7f359c099e70 comp rx=0 tx=0).stop 2024-05-24T11:54:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.799+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/821861330 shutdown_connections 2024-05-24T11:54:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.799+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d331a7270 0x556d33199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.799+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d3319d480 0x556d3319af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.799+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/821861330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331a2950 0x556d33197910 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.799+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/821861330 >> 172.21.15.39:0/821861330 conn(0x556d330fb1b0 msgr2=0x556d331a1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.800+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/821861330 shutdown_connections 2024-05-24T11:54:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.800+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/821861330 wait complete. 2024-05-24T11:54:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.800+0000 7f35bf7ceb80 1 -- start start 2024-05-24T11:54:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.800+0000 7f35bf7ceb80 1 Processor -- start 2024-05-24T11:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.801+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d3319d480 0x556d331ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.801+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a7270 0x556d331a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.801+0000 7f35bf7ceb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331aa280 0x556d3320c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.801+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d331aa8d0 con 0x556d331aa280 2024-05-24T11:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.801+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d331aaa40 con 0x556d331a7270 2024-05-24T11:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.801+0000 7f35bf7ceb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3320cc30 con 0x556d3319d480 2024-05-24T11:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.801+0000 7f35bf7ceb80 10 client.0 _ll_register_callbacks cb 0x556d33092ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.802+0000 7f35b0bae700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a7270 0x556d331a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.802+0000 7f35b0bae700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a7270 0x556d331a9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:56080/0 (socket says 172.21.15.39:56080) 2024-05-24T11:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.802+0000 7f35b0bae700 1 -- 172.21.15.39:0/3134363077 learned_addr learned my addr 172.21.15.39:0/3134363077 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.802+0000 7f35b1bb0700 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331aa280 0x556d3320c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.802+0000 7f35b0bae700 1 -- 172.21.15.39:0/3134363077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d3319d480 msgr2=0x556d331ae120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-24T11:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.802+0000 7f35b0bae700 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d3319d480 0x556d331ae120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.802+0000 7f35b0bae700 1 -- 172.21.15.39:0/3134363077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331aa280 msgr2=0x556d3320c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.802+0000 7f35b0bae700 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331aa280 0x556d3320c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.802+0000 7f35b0bae700 1 -- 172.21.15.39:0/3134363077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d32f1ab00 con 0x556d331a7270 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f35b0bae700 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a7270 0x556d331a9be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f359c098550 tx=0x7f359c0a2730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6e0b9b05c05fc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f3594ff9700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f3594ff9700 1 -- 172.21.15.39:0/3134363077 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c016330 con 0x556d331a7270 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f3594ff9700 1 -- 172.21.15.39:0/3134363077 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f359c09a200 con 0x556d331a7270 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/3134363077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f359c09a900 con 0x556d331a7270 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f3594ff9700 1 -- 172.21.15.39:0/3134363077 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f359c096910 con 0x556d331a7270 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f35bf7ceb80 10 client.26237 fetch_fsmap learned FSMap version 40 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/3134363077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556d32f6e350 con 0x556d331a7270 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f3594ff9700 1 -- 172.21.15.39:0/3134363077 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f359c09a900 con 0x556d331a7270 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.803+0000 7f3594ff9700 1 -- 172.21.15.39:0/3134363077 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f359c09d030 con 0x556d331a7270 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.804+0000 7f35bf7ceb80 10 client.26237 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.804+0000 7f35bf7ceb80 10 client.26237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:54:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.804+0000 7f35bf7ceb80 20 client.26237 populate_metadata read hostname 'smithi039' 2024-05-24T11:54:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.804+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556d3320de10 0x556d33210200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.804+0000 7f35bf7ceb80 4 client.26237 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:54:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.804+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/3134363077 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556d330e5e60 con 0x556d3320de10 2024-05-24T11:54:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.804+0000 7f35b13af700 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556d3320de10 0x556d33210200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.804+0000 7f3594ff9700 10 client.26237 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.804+0000 7f35b13af700 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556d3320de10 0x556d33210200 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed6430cae4187a4 server_cookie=b3f763101d36986a in_seq=0 out_seq=0 2024-05-24T11:54:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.849+0000 7f3594ff9700 1 -- 172.21.15.39:0/3134363077 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556d330e5e60 con 0x556d3320de10 2024-05-24T11:54:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.849+0000 7f3594ff9700 10 client.26237 handle_command_reply: tid=0 2024-05-24T11:54:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.850+0000 7f35bf7ceb80 1 client.26237 shutdown 2024-05-24T11:54:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.851+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/3134363077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a7270 msgr2=0x556d331a9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.851+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a7270 0x556d331a9be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f359c098550 tx=0x7f359c0a2730 comp rx=0 tx=0).stop 2024-05-24T11:54:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.851+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/3134363077 shutdown_connections 2024-05-24T11:54:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.851+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d3319d480 0x556d331ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.851+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x556d3320de10 0x556d33210200 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.851+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d331aa280 0x556d3320c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.851+0000 7f35bf7ceb80 1 --2- 172.21.15.39:0/3134363077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556d331a7270 0x556d331a9be0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.851+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/3134363077 >> 172.21.15.39:0/3134363077 conn(0x556d330fb1b0 msgr2=0x556d3319c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.851+0000 7f3594ff9700 0 client.26237 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.852+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/3134363077 shutdown_connections 2024-05-24T11:54:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.852+0000 7f35bf7ceb80 1 -- 172.21.15.39:0/3134363077 wait complete. 2024-05-24T11:54:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.852+0000 7f35bf7ceb80 20 client.26237 trim_cache size 0 max 16384 2024-05-24T11:54:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:54:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:54:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:54:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:54:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 -- 172.21.15.39:0/2823253629 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f35980a1390 msgr2=0x7f35980a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f35980a1390 0x7f35980a3850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35ac10e8b0 tx=0x7f35a4009920 comp rx=0 tx=0).stop 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 -- 172.21.15.39:0/2823253629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac10d8f0 msgr2=0x7f35ac10bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac10d8f0 0x7f35ac10bf50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f359c00eb90 tx=0x7f359c00eea0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 -- 172.21.15.39:0/2823253629 shutdown_connections 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f35980a1390 0x7f35980a3850 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35ac10c5f0 0x7f35ac1a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ac069ab0 0x7f35ac10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 --2- 172.21.15.39:0/2823253629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac10d8f0 0x7f35ac10bf50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 -- 172.21.15.39:0/2823253629 >> 172.21.15.39:0/2823253629 conn(0x7f35ac076f40 msgr2=0x7f35ac10f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 -- 172.21.15.39:0/2823253629 shutdown_connections 2024-05-24T11:54:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:13.853+0000 7f35b3613700 1 -- 172.21.15.39:0/2823253629 wait complete. 2024-05-24T11:54:14.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 890.0 s, 1089.5 tps, lat 92.288 ms stddev 45.159 2024-05-24T11:54:14.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:14 smithi039 ceph-mon[42919]: pgmap v509: 97 pgs: 13 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 59 active+clean; 5.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:54:14.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:14 smithi053 ceph-mon[47448]: pgmap v509: 97 pgs: 13 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 59 active+clean; 5.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:54:14.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:14 smithi137 ceph-mon[47907]: pgmap v509: 97 pgs: 13 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 59 active+clean; 5.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-05-24T11:54:16.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:16 smithi039 ceph-mon[42919]: pgmap v510: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-05-24T11:54:16.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:16 smithi053 ceph-mon[47448]: pgmap v510: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-05-24T11:54:16.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:16 smithi137 ceph-mon[47907]: pgmap v510: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-05-24T11:54:18.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:18 smithi039 ceph-mon[42919]: pgmap v511: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-05-24T11:54:18.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:18 smithi053 ceph-mon[47448]: pgmap v511: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-05-24T11:54:18.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:18 smithi137 ceph-mon[47907]: pgmap v511: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-05-24T11:54:19.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 895.0 s, 1073.8 tps, lat 92.711 ms stddev 42.202 2024-05-24T11:54:20.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:20 smithi039 ceph-mon[42919]: pgmap v512: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 16 MiB/s wr, 1.45k op/s 2024-05-24T11:54:20.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:20 smithi039 ceph-mon[42919]: osdmap e107: 12 total, 12 up, 12 in 2024-05-24T11:54:20.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:20 smithi053 ceph-mon[47448]: pgmap v512: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 16 MiB/s wr, 1.45k op/s 2024-05-24T11:54:20.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:20 smithi053 ceph-mon[47448]: osdmap e107: 12 total, 12 up, 12 in 2024-05-24T11:54:20.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:20 smithi137 ceph-mon[47907]: pgmap v512: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 16 MiB/s wr, 1.45k op/s 2024-05-24T11:54:20.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:20 smithi137 ceph-mon[47907]: osdmap e107: 12 total, 12 up, 12 in 2024-05-24T11:54:22.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:22 smithi039 ceph-mon[42919]: pgmap v514: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-05-24T11:54:22.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:22 smithi053 ceph-mon[47448]: pgmap v514: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-05-24T11:54:22.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:22 smithi137 ceph-mon[47907]: pgmap v514: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-05-24T11:54:24.113 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 900.0 s, 1060.4 tps, lat 94.705 ms stddev 32.145 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:transaction type: 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:scaling factor: 32 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:query mode: simple 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:number of clients: 100 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:number of threads: 4 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:duration: 900 s 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:number of transactions actually processed: 963662 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:latency average = 93.377 ms 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:latency stddev = 106.236 ms 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:tps = 1070.483637 (including connections establishing) 2024-05-24T11:54:24.325 INFO:teuthology.orchestra.run.smithi039.stdout:tps = 1070.496754 (excluding connections establishing) 2024-05-24T11:54:24.360 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:24 smithi039 ceph-mon[42919]: pgmap v515: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-05-24T11:54:24.363 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-24T11:54:24.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:24 smithi053 ceph-mon[47448]: pgmap v515: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-05-24T11:54:24.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:24 smithi137 ceph-mon[47907]: pgmap v515: 97 pgs: 97 active+clean; 5.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-05-24T11:54:26.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:26 smithi053 ceph-mon[47448]: pgmap v516: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-05-24T11:54:26.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:26 smithi137 ceph-mon[47907]: pgmap v516: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-05-24T11:54:26.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:26 smithi039 ceph-mon[42919]: pgmap v516: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-05-24T11:54:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.023+0000 7fe8b625f700 1 -- 172.21.15.39:0/2726837935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 msgr2=0x7fe8b010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.023+0000 7fe8b625f700 1 --2- 172.21.15.39:0/2726837935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 0x7fe8b010ddb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800b0a0 tx=0x7fe8a800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.023+0000 7fe8b625f700 1 -- 172.21.15.39:0/2726837935 shutdown_connections 2024-05-24T11:54:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.023+0000 7fe8b625f700 1 --2- 172.21.15.39:0/2726837935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8b0101110 0x7fe8b01014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.023+0000 7fe8b625f700 1 --2- 172.21.15.39:0/2726837935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 0x7fe8b010ddb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.023+0000 7fe8b625f700 1 --2- 172.21.15.39:0/2726837935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b0101ac0 0x7fe8b0105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.023+0000 7fe8b625f700 1 -- 172.21.15.39:0/2726837935 >> 172.21.15.39:0/2726837935 conn(0x7fe8b00fc9b0 msgr2=0x7fe8b00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.023+0000 7fe8b625f700 1 -- 172.21.15.39:0/2726837935 shutdown_connections 2024-05-24T11:54:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.024+0000 7fe8b625f700 1 -- 172.21.15.39:0/2726837935 wait complete. 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.025+0000 7fe8b625f700 1 Processor -- start 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.025+0000 7fe8b625f700 1 -- start start 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.025+0000 7fe8b625f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b0101110 0x7fe8b01973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.025+0000 7fe8b625f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 0x7fe8b0197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.025+0000 7fe8b625f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8b019bc70 0x7fe8b019c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.025+0000 7fe8b625f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019c6f0 con 0x7fe8b0106230 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.025+0000 7fe8b625f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019c860 con 0x7fe8b0101110 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.025+0000 7fe8b625f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019ca20 con 0x7fe8b019bc70 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8aeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 0x7fe8b0197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b0101110 0x7fe8b01973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8aeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 0x7fe8b0197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50230/0 (socket says 172.21.15.39:50230) 2024-05-24T11:54:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8aeffd700 1 -- 172.21.15.39:0/2394947307 learned_addr learned my addr 172.21.15.39:0/2394947307 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8affff700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8b019bc70 0x7fe8b019c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8af7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b0101110 0x7fe8b01973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:37548/0 (socket says 172.21.15.39:37548) 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8affff700 1 -- 172.21.15.39:0/2394947307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b0101110 msgr2=0x7fe8b01973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8affff700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b0101110 0x7fe8b01973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8affff700 1 -- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 msgr2=0x7fe8b0197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8affff700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 0x7fe8b0197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8affff700 1 -- 172.21.15.39:0/2394947307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a8009d00 con 0x7fe8b019bc70 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8af7fe700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b0101110 0x7fe8b01973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 0x7fe8b0197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.026+0000 7fe8affff700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8b019bc70 0x7fe8b019c0f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800b780 tx=0x7fe8a8004e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.027+0000 7fe8acff9700 1 -- 172.21.15.39:0/2394947307 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800e040 con 0x7fe8b019bc70 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.027+0000 7fe8acff9700 1 -- 172.21.15.39:0/2394947307 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8a801b450 con 0x7fe8b019bc70 2024-05-24T11:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.027+0000 7fe8acff9700 1 -- 172.21.15.39:0/2394947307 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80052a0 con 0x7fe8b019bc70 2024-05-24T11:54:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.027+0000 7fe8b625f700 1 -- 172.21.15.39:0/2394947307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b01a3f20 con 0x7fe8b019bc70 2024-05-24T11:54:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.027+0000 7fe8b625f700 1 -- 172.21.15.39:0/2394947307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b01a4350 con 0x7fe8b019bc70 2024-05-24T11:54:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8c241ab80 1 Processor -- start 2024-05-24T11:54:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8c241ab80 1 -- start start 2024-05-24T11:54:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608058ad0 0x563608058eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 0x563608052f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 0x5636080538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563607faa0f0 con 0x563608058ad0 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563607fa7ab0 con 0x563608053480 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563607fa33a0 con 0x5636080593f0 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8aeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608058ad0 0x563608058eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8aeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608058ad0 0x563608058eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50232/0 (socket says 172.21.15.39:50232) 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8aeffd700 1 -- 172.21.15.39:0/208082944 learned_addr learned my addr 172.21.15.39:0/208082944 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.032+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/208082944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 0x5636080538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.033+0000 7fe8aeffd700 1 -- 172.21.15.39:0/208082944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 msgr2=0x563608052f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.033+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/208082944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 0x563608052f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.033+0000 7fe8aeffd700 1 -- 172.21.15.39:0/208082944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 msgr2=0x5636080538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.033+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/208082944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 0x5636080538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.033+0000 7fe8aeffd700 1 -- 172.21.15.39:0/208082944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563607e24350 con 0x563608058ad0 2024-05-24T11:54:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.033+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/208082944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608058ad0 0x563608058eb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009880 tx=0x7fe8a4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7c781b97742db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.033+0000 7fe8957fa700 1 -- 172.21.15.39:0/208082944 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4004030 con 0x563608058ad0 2024-05-24T11:54:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.033+0000 7fe8957fa700 1 -- 172.21.15.39:0/208082944 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8a4004830 con 0x563608058ad0 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.033+0000 7fe8957fa700 1 -- 172.21.15.39:0/208082944 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4005220 con 0x563608058ad0 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.034+0000 7fe8c241ab80 1 -- 172.21.15.39:0/208082944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608058ad0 msgr2=0x563608058eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.034+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/208082944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608058ad0 0x563608058eb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009880 tx=0x7fe8a4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.034+0000 7fe8acff9700 1 -- 172.21.15.39:0/2394947307 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe8a801b830 con 0x7fe8b019bc70 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8c241ab80 1 -- 172.21.15.39:0/208082944 shutdown_connections 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/208082944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 0x5636080538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/208082944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608058ad0 0x563608058eb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/208082944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 0x563608052f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8c241ab80 1 -- 172.21.15.39:0/208082944 >> 172.21.15.39:0/208082944 conn(0x563607fb11b0 msgr2=0x56360804db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8c241ab80 1 -- 172.21.15.39:0/208082944 shutdown_connections 2024-05-24T11:54:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8c241ab80 1 -- 172.21.15.39:0/208082944 wait complete. 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8c241ab80 1 -- start start 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8c241ab80 1 Processor -- start 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.035+0000 7fe8acff9700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe8980a12f0 0x7fe8980a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 0x56360805f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608058ad0 0x56360805c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 0x56360805ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563607f3f500 con 0x5636080593f0 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563607f3f1a0 con 0x563608053480 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563607e26640 con 0x563608058ad0 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8acff9700 1 -- 172.21.15.39:0/2394947307 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe8a80cdff0 con 0x7fe8b019bc70 2024-05-24T11:54:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8c241ab80 10 client.0 _ll_register_callbacks cb 0x563607f48ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe8980a12f0 0x7fe8980a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8aeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608058ad0 0x56360805c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8aeffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608058ad0 0x56360805c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:36350/0 (socket says 172.21.15.39:36350) 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8aeffd700 1 -- 172.21.15.39:0/2933326409 learned_addr learned my addr 172.21.15.39:0/2933326409 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8affff700 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 0x56360805ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8af7fe700 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 0x56360805f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8affff700 1 -- 172.21.15.39:0/2933326409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608058ad0 msgr2=0x56360805c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8affff700 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608058ad0 0x56360805c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8affff700 1 -- 172.21.15.39:0/2933326409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 msgr2=0x56360805f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.036+0000 7fe8affff700 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 0x56360805f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.037+0000 7fe8affff700 1 -- 172.21.15.39:0/2933326409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a401b040 con 0x5636080593f0 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.037+0000 7fe8affff700 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 0x56360805ca30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80cb6f0 tx=0x7fe8a809af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e2f5d6b969812e server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.037+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe8980a12f0 0x7fe8980a37b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4005100 tx=0x7fe8a4004f40 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.039+0000 7fe8937f6700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.039+0000 7fe8937f6700 1 -- 172.21.15.39:0/2933326409 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a809b840 con 0x5636080593f0 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.039+0000 7fe8937f6700 1 -- 172.21.15.39:0/2933326409 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe8a80cbe40 con 0x5636080593f0 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.039+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2933326409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563607efc210 con 0x5636080593f0 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.039+0000 7fe8937f6700 1 -- 172.21.15.39:0/2933326409 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a809b140 con 0x5636080593f0 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.039+0000 7fe8c241ab80 10 client.16476 fetch_fsmap learned FSMap version 40 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.039+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2933326409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563607dd0b00 con 0x5636080593f0 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.040+0000 7fe8937f6700 1 -- 172.21.15.39:0/2933326409 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563607efc210 con 0x5636080593f0 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.040+0000 7fe8937f6700 1 -- 172.21.15.39:0/2933326409 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe8a80b9240 con 0x5636080593f0 2024-05-24T11:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.040+0000 7fe8c241ab80 10 client.16476 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.040+0000 7fe8c241ab80 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.040+0000 7fe8c241ab80 20 client.16476 populate_metadata read hostname 'smithi039' 2024-05-24T11:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.040+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563608070e70 0x563608073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.040+0000 7fe8c241ab80 4 client.16476 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-24T11:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.040+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2933326409 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636080227b0 con 0x563608070e70 2024-05-24T11:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.040+0000 7fe8af7fe700 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563608070e70 0x563608073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.046+0000 7fe8af7fe700 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563608070e70 0x563608073290 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ec09441ef9d863 server_cookie=9e8c34f413a2da5a in_seq=0 out_seq=0 2024-05-24T11:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.046+0000 7fe8937f6700 10 client.16476 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.047+0000 7fe8937f6700 1 -- 172.21.15.39:0/2933326409 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636080227b0 con 0x563608070e70 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.047+0000 7fe8937f6700 10 client.16476 handle_command_reply: tid=0 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.047+0000 7fe8c241ab80 1 client.16476 shutdown 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.049+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2933326409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 msgr2=0x56360805ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.049+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 0x56360805ca30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80cb6f0 tx=0x7fe8a809af10 comp rx=0 tx=0).stop 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.049+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2933326409 shutdown_connections 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.049+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563608070e70 0x563608073290 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.049+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 0x56360805ca30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.049+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608053480 0x56360805f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.049+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2933326409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608058ad0 0x56360805c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.049+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2933326409 >> 172.21.15.39:0/2933326409 conn(0x563607fb11b0 msgr2=0x56360804d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.049+0000 7fe8937f6700 0 client.16476 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.053+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2933326409 shutdown_connections 2024-05-24T11:54:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.053+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2933326409 wait complete. 2024-05-24T11:54:27.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.053+0000 7fe8c241ab80 20 client.16476 trim_cache size 0 max 16384 2024-05-24T11:54:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8c241ab80 1 Processor -- start 2024-05-24T11:54:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8c241ab80 1 -- start start 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608053480 0x56360804d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x56360805eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 0x563608068240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563608078480 con 0x5636080593f0 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360805cd90 con 0x563608058ad0 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360805aa40 con 0x563608053480 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8aeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x56360805eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8aeffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x56360805eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:37576/0 (socket says 172.21.15.39:37576) 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8aeffd700 1 -- 172.21.15.39:0/2236743766 learned_addr learned my addr 172.21.15.39:0/2236743766 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8affff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 0x563608068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8af7fe700 1 --2- 172.21.15.39:0/2236743766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608053480 0x56360804d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8aeffd700 1 -- 172.21.15.39:0/2236743766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608053480 msgr2=0x56360804d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/2236743766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608053480 0x56360804d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8aeffd700 1 -- 172.21.15.39:0/2236743766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 msgr2=0x563608068240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/2236743766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 0x563608068240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.073+0000 7fe8aeffd700 1 -- 172.21.15.39:0/2236743766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563607f69500 con 0x563608058ad0 2024-05-24T11:54:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.074+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/2236743766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x56360805eda0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4004ff0 tx=0x7fe8a4005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a412f872a2a11e16 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.074+0000 7fe895ffb700 1 -- 172.21.15.39:0/2236743766 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4013890 con 0x563608058ad0 2024-05-24T11:54:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.074+0000 7fe895ffb700 1 -- 172.21.15.39:0/2236743766 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8a4011e30 con 0x563608058ad0 2024-05-24T11:54:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.074+0000 7fe895ffb700 1 -- 172.21.15.39:0/2236743766 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4020ae0 con 0x563608058ad0 2024-05-24T11:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2236743766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 msgr2=0x56360805eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2236743766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x56360805eda0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4004ff0 tx=0x7fe8a4005910 comp rx=0 tx=0).stop 2024-05-24T11:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2236743766 shutdown_connections 2024-05-24T11:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2236743766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563608053480 0x56360804d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2236743766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636080593f0 0x563608068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/2236743766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x56360805eda0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2236743766 >> 172.21.15.39:0/2236743766 conn(0x563607fb11b0 msgr2=0x5636080567d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2236743766 shutdown_connections 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 -- 172.21.15.39:0/2236743766 wait complete. 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 -- start start 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.075+0000 7fe8c241ab80 1 Processor -- start 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608053480 0x563608055c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x563608068210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8c241ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 0x563608068750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636080c2340 con 0x563608053480 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636080c24b0 con 0x563608058ad0 2024-05-24T11:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8c241ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636080c2620 con 0x5636080593f0 2024-05-24T11:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8c241ab80 10 client.0 _ll_register_callbacks cb 0x563607f48ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-24T11:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8affff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 0x563608068750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8affff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 0x563608068750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.39:36364/0 (socket says 172.21.15.39:36364) 2024-05-24T11:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.076+0000 7fe8affff700 1 -- 172.21.15.39:0/3145959101 learned_addr learned my addr 172.21.15.39:0/3145959101 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.077+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x563608068210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.077+0000 7fe8aeffd700 1 -- 172.21.15.39:0/3145959101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 msgr2=0x563608068750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.077+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 0x563608068750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.077+0000 7fe8aeffd700 1 -- 172.21.15.39:0/3145959101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608053480 msgr2=0x563608055c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.077+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608053480 0x563608055c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.077+0000 7fe8aeffd700 1 -- 172.21.15.39:0/3145959101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a4028040 con 0x563608058ad0 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.077+0000 7fe8aeffd700 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x563608068210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x56360805d0f0 tx=0x7fe8a40208c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31af542314befea server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.077+0000 7fe8937f6700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8937f6700 1 -- 172.21.15.39:0/3145959101 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4013890 con 0x563608058ad0 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8937f6700 1 -- 172.21.15.39:0/3145959101 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe8a4004ac0 con 0x563608058ad0 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8c241ab80 1 -- 172.21.15.39:0/3145959101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8a4004830 con 0x563608058ad0 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8937f6700 1 -- 172.21.15.39:0/3145959101 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a4005650 con 0x563608058ad0 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8c241ab80 10 client.26255 fetch_fsmap learned FSMap version 40 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8c241ab80 1 -- 172.21.15.39:0/3145959101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563607e28c70 con 0x563608058ad0 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8937f6700 1 -- 172.21.15.39:0/3145959101 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8a404b080 con 0x563608058ad0 2024-05-24T11:54:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8937f6700 1 -- 172.21.15.39:0/3145959101 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 40) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe8a402e810 con 0x563608058ad0 2024-05-24T11:54:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8c241ab80 10 client.26255 fetch_fsmap finished waiting for FSMap version 40 2024-05-24T11:54:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8c241ab80 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-24T11:54:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8c241ab80 20 client.26255 populate_metadata read hostname 'smithi039' 2024-05-24T11:54:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563608054b20 0x563608054f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8c241ab80 4 client.26255 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-24T11:54:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.078+0000 7fe8c241ab80 1 -- 172.21.15.39:0/3145959101 --> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563607f9be60 con 0x563608054b20 2024-05-24T11:54:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.079+0000 7fe8af7fe700 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563608054b20 0x563608054f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.079+0000 7fe8937f6700 10 client.26255 ms_handle_connect on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.079+0000 7fe8af7fe700 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563608054b20 0x563608054f00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4536c6c109b824 server_cookie=ad84307a4d077216 in_seq=0 out_seq=0 2024-05-24T11:54:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.259+0000 7fe8937f6700 1 -- 172.21.15.39:0/3145959101 <== mds.0 v2:172.21.15.53:6838/3196235839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563607f9be60 con 0x563608054b20 2024-05-24T11:54:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.259+0000 7fe8937f6700 10 client.26255 handle_command_reply: tid=0 2024-05-24T11:54:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.259+0000 7fe8c241ab80 1 client.26255 shutdown 2024-05-24T11:54:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.264+0000 7fe8c241ab80 1 -- 172.21.15.39:0/3145959101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 msgr2=0x563608068210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.264+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x563608068210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x56360805d0f0 tx=0x7fe8a40208c0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8c241ab80 1 -- 172.21.15.39:0/3145959101 shutdown_connections 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636080593f0 0x563608068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.53:6838/3196235839,v1:172.21.15.53:6839/3196235839] conn(0x563608054b20 0x563608054f00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563608053480 0x563608055c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8c241ab80 1 --2- 172.21.15.39:0/3145959101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563608058ad0 0x563608068210 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8c241ab80 1 -- 172.21.15.39:0/3145959101 >> 172.21.15.39:0/3145959101 conn(0x563607fb11b0 msgr2=0x56360805ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8937f6700 0 client.26255 ms_handle_reset on v2:172.21.15.53:6838/3196235839 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8c241ab80 1 -- 172.21.15.39:0/3145959101 shutdown_connections 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8c241ab80 1 -- 172.21.15.39:0/3145959101 wait complete. 2024-05-24T11:54:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.265+0000 7fe8c241ab80 20 client.26255 trim_cache size 0 max 16384 2024-05-24T11:54:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-24T11:54:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-24T11:54:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-24T11:54:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-24T11:54:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 -- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe8980a12f0 msgr2=0x7fe8980a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe8980a12f0 0x7fe8980a37b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4005100 tx=0x7fe8a4004f40 comp rx=0 tx=0).stop 2024-05-24T11:54:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 -- 172.21.15.39:0/2394947307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8b019bc70 msgr2=0x7fe8b019c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8b019bc70 0x7fe8b019c0f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800b780 tx=0x7fe8a8004e20 comp rx=0 tx=0).stop 2024-05-24T11:54:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 -- 172.21.15.39:0/2394947307 shutdown_connections 2024-05-24T11:54:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8b019bc70 0x7fe8b019c0f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fe8980a12f0 0x7fe8980a37b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b0101110 0x7fe8b01973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 --2- 172.21.15.39:0/2394947307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8b0106230 0x7fe8b0197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 -- 172.21.15.39:0/2394947307 >> 172.21.15.39:0/2394947307 conn(0x7fe8b00fc9b0 msgr2=0x7fe8b00fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 -- 172.21.15.39:0/2394947307 shutdown_connections 2024-05-24T11:54:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-24T11:54:27.266+0000 7fe8967fc700 1 -- 172.21.15.39:0/2394947307 wait complete. 2024-05-24T11:54:28.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:28 smithi053 ceph-mon[47448]: pgmap v517: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-05-24T11:54:28.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:28 smithi137 ceph-mon[47907]: pgmap v517: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-05-24T11:54:28.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:28 smithi039 ceph-mon[42919]: pgmap v517: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-05-24T11:54:30.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:30 smithi053 ceph-mon[47448]: pgmap v518: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 12 MiB/s wr, 1.26k op/s 2024-05-24T11:54:30.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:30 smithi137 ceph-mon[47907]: pgmap v518: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 12 MiB/s wr, 1.26k op/s 2024-05-24T11:54:30.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:30 smithi039 ceph-mon[42919]: pgmap v518: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 12 MiB/s wr, 1.26k op/s 2024-05-24T11:54:30.673 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-24T11:54:31.039 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/: 2024-05-24T11:54:31.039 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-24T11:54:31.039 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1788082128 May 24 11:38 . 2024-05-24T11:54:31.039 INFO:teuthology.orchestra.run.smithi039.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1788082128 May 24 11:38 .. 2024-05-24T11:54:31.039 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 data 2024-05-24T11:54:31.039 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.039 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data: 2024-05-24T11:54:31.039 INFO:teuthology.orchestra.run.smithi039.stdout:total 41 2024-05-24T11:54:31.039 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 . 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1788082128 May 24 11:38 .. 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 746750093 May 24 11:39 base 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 24 11:38 current_logfiles 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 587404 May 24 11:38 global 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 522 May 24 11:38 log 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_commit_ts 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_dynshmem 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 May 24 11:38 pg_hba.conf 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 24 11:38 pg_ident.conf 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:54 pg_logical 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 pg_multixact 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 pg_notify 2024-05-24T11:54:31.040 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_replslot 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_serial 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_snapshots 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 May 24 11:54 pg_stat 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:54 pg_stat_tmp 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 May 24 11:54 pg_subtrans 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_tblspc 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 pg_twophase 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 24 11:38 PG_VERSION 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1056964608 May 24 11:54 pg_wal 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 245760 May 24 11:38 pg_xact 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 24 11:38 postgresql.auto.conf 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 May 24 11:38 postgresql.conf 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 24 11:38 postmaster.opts 2024-05-24T11:54:31.041 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base: 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:total 3 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 746750093 May 24 11:39 . 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 1 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 12346 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 732003463 May 24 11:39 12347 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:39 pgsql_tmp 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:total 7202 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 . 2024-05-24T11:54:31.042 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 746750093 May 24 11:39 .. 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 112 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 113 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 24 11:38 12182 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12182_fsm 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12182_vm 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12184 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12186 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12187_fsm 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187_vm 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12189 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12191 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192 2024-05-24T11:54:31.043 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12192_fsm 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192_vm 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12194 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12196 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12197_fsm 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197_vm 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12199 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12201 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12202_fsm 2024-05-24T11:54:31.044 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202_vm 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12204 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12206 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12207_fsm 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207_vm 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12209 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12211 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12212 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12214 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12216 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 24 11:38 1247 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1247_fsm 2024-05-24T11:54:31.045 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1247_vm 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 24 11:38 1249 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1249_fsm 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1249_vm 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 24 11:38 1255 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1255_fsm 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1255_vm 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 24 11:38 1259 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1259_fsm 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1259_vm 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417_vm 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418 2024-05-24T11:54:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418_vm 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 174 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 175 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2187 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224_vm 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328_vm 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336_vm 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2337 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2600 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2600_fsm 2024-05-24T11:54:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2600_vm 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2601_fsm 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601_vm 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2602 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2602_fsm 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2602_vm 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2603 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2603_fsm 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2603_vm 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604_vm 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2605 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2605_fsm 2024-05-24T11:54:31.048 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2605_vm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2606_fsm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606_vm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607_fsm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2607_vm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 24 11:38 2608 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2608_fsm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2608_vm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 24 11:38 2609 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2609_fsm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2609_vm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610_fsm 2024-05-24T11:54:31.049 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2610_vm 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611_vm 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2612_fsm 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612_vm 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613_vm 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2615_fsm 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615_vm 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616_fsm 2024-05-24T11:54:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2616_vm 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 24 11:38 2617 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2617_fsm 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2617_vm 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 24 11:38 2618 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2618_fsm 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2618_vm 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 24 11:38 2619 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2619_fsm 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2619_vm 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620_vm 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2650 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2651 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2652 2024-05-24T11:54:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2653 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2654 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2655 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2656 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2657 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 24 11:38 2658 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2659 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2660 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2661 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2662 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2663 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2664 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2665 2024-05-24T11:54:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2666 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2667 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2668 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2669 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2670 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 May 24 11:38 2673 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 May 24 11:38 2674 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 24 11:38 2675 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2678 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2679 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2680 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2681 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2682 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2683 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2684 2024-05-24T11:54:31.053 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2685 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2686 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2687 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2688 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2689 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2690 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 24 11:38 2691 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2692 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2693 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2696 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2699 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2701 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2702 2024-05-24T11:54:31.054 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2703 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2704 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2753 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2753_fsm 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2753_vm 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2754 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2755 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2756 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2757 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830_vm 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2831 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832_vm 2024-05-24T11:54:31.055 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2833 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834_vm 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2835 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836_vm 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2837 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 24 11:38 2838 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2838_fsm 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2838_vm 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2839 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840_fsm 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2840_vm 2024-05-24T11:54:31.056 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2841 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995_vm 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2996 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3079_fsm 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079_vm 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3080 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3081 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3085 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118_vm 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3119 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3164 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256 2024-05-24T11:54:31.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256_vm 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3257 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3258 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350_vm 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3351 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3379 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3380 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381_vm 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3394 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3394_fsm 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3394_vm 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3395 2024-05-24T11:54:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439_vm 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3440 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 3455 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3456 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3456_fsm 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3456_vm 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466_vm 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3467 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3468 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501_vm 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3502 2024-05-24T11:54:31.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3503 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3534 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3541_fsm 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541_vm 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3542 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3574 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3575 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576_vm 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596_vm 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3597 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598_vm 2024-05-24T11:54:31.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3599 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3600_fsm 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600_vm 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3601_fsm 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601_vm 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3602_fsm 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602_vm 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3603 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3603_fsm 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3603_vm 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3604 2024-05-24T11:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3605 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3606 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3607 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3608 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 3609 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3712 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3764_fsm 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764_vm 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3766 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3767 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3997 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 5002 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 548 2024-05-24T11:54:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 549 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102_vm 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104_vm 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106_vm 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6110 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6111 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6112 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6113 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6117 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826_vm 2024-05-24T11:54:31.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 827 2024-05-24T11:54:31.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 828 2024-05-24T11:54:31.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 24 11:38 pg_filenode.map 2024-05-24T11:54:31.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 24 11:38 PG_VERSION 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/12346: 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:total 7202 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 May 24 11:38 . 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 746750093 May 24 11:39 .. 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 112 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 113 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 24 11:38 12182 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12182_fsm 2024-05-24T11:54:31.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12182_vm 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12184 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12186 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12187_fsm 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187_vm 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12189 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12191 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12192_fsm 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192_vm 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12194 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12196 2024-05-24T11:54:31.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12197_fsm 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197_vm 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12199 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12201 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12202_fsm 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202_vm 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12204 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12206 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12207_fsm 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207_vm 2024-05-24T11:54:31.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12209 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12211 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12212 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12214 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12216 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 24 11:38 1247 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1247_fsm 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1247_vm 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 24 11:38 1249 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1249_fsm 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1249_vm 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 24 11:38 1255 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1255_fsm 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1255_vm 2024-05-24T11:54:31.323 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 24 11:38 1259 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1259_fsm 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1259_vm 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417_vm 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418_vm 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 174 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 175 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2187 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224_vm 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328 2024-05-24T11:54:31.324 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328_vm 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336_vm 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2337 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2600 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2600_fsm 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2600_vm 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2601_fsm 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601_vm 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2602 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2602_fsm 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2602_vm 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2603 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2603_fsm 2024-05-24T11:54:31.325 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2603_vm 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604_vm 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2605 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2605_fsm 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2605_vm 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2606_fsm 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2606_vm 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607_fsm 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2607_vm 2024-05-24T11:54:31.326 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 May 24 11:38 2608 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2608_fsm 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2608_vm 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 24 11:38 2609 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2609_fsm 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2609_vm 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2610_fsm 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2610_vm 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611_vm 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2612_fsm 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612_vm 2024-05-24T11:54:31.327 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613_vm 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2615_fsm 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615_vm 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616_fsm 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2616_vm 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 24 11:38 2617 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2617_fsm 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2617_vm 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 24 11:38 2618 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2618_fsm 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2618_vm 2024-05-24T11:54:31.328 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 24 11:38 2619 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2619_fsm 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2619_vm 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620_vm 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2650 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2651 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2652 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2653 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2654 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2655 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2656 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2657 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 24 11:38 2658 2024-05-24T11:54:31.329 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2659 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2660 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2661 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2662 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2663 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2664 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2665 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2666 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2667 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2668 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2669 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2670 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 May 24 11:38 2673 2024-05-24T11:54:31.330 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 May 24 11:38 2674 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 24 11:38 2675 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2678 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2679 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2680 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2681 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2682 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2683 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2684 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2685 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2686 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2687 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2688 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2689 2024-05-24T11:54:31.331 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2690 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 24 11:38 2691 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2692 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2693 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2696 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2699 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2701 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2702 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2703 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2704 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2753 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2753_fsm 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2753_vm 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2754 2024-05-24T11:54:31.332 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2755 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2756 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2757 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830_vm 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2831 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832_vm 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2833 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834_vm 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2835 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836_vm 2024-05-24T11:54:31.333 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2837 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 24 11:38 2838 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2838_fsm 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2838_vm 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2839 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840_fsm 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2840_vm 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2841 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995_vm 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2996 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3079_fsm 2024-05-24T11:54:31.334 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079_vm 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3080 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3081 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3085 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118_vm 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3119 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3164 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256_vm 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3257 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3258 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350 2024-05-24T11:54:31.335 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350_vm 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3351 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3379 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3380 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381_vm 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3394 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3394_fsm 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3394_vm 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3395 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439_vm 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3440 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 3455 2024-05-24T11:54:31.336 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3456 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3456_fsm 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3456_vm 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466_vm 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3467 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3468 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501_vm 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3502 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3503 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3534 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3541_fsm 2024-05-24T11:54:31.337 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541_vm 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3542 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3574 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3575 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576_vm 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596_vm 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3597 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598_vm 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3599 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600 2024-05-24T11:54:31.338 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3600_fsm 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600_vm 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3601_fsm 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601_vm 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3602_fsm 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602_vm 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3603 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3603_fsm 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3603_vm 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3604 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3605 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3606 2024-05-24T11:54:31.339 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3607 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3608 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 3609 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3712 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3764_fsm 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764_vm 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3766 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3767 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3997 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 5002 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 548 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 549 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102 2024-05-24T11:54:31.340 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102_vm 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104_vm 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106_vm 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6110 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6111 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6112 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6113 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6117 2024-05-24T11:54:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826 2024-05-24T11:54:31.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826_vm 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 827 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 828 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 24 11:38 pg_filenode.map 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 24 11:38 PG_VERSION 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/12347: 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:total 714849 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 732003463 May 24 11:39 . 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 746750093 May 24 11:39 .. 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 112 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 113 2024-05-24T11:54:31.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 24 11:38 12182 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12182_fsm 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12182_vm 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12184 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12186 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12187_fsm 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12187_vm 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12189 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12191 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12192_fsm 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12192_vm 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12194 2024-05-24T11:54:31.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12196 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12197_fsm 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12197_vm 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12199 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12201 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12202_fsm 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12202_vm 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12204 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12206 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 12207_fsm 2024-05-24T11:54:31.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12207_vm 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12209 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12211 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12212 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 12214 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 12216 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 May 24 11:39 1247 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:39 1247_fsm 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:39 1247_vm 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 May 24 11:39 1249 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:39 1249_fsm 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:39 1249_vm 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 May 24 11:38 1255 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1255_fsm 2024-05-24T11:54:31.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1255_vm 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 24 11:54 1259 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1259_fsm 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:39 1259_vm 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1417_vm 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 1418_vm 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21708800 May 24 11:54 16387 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:54 16387_fsm 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:54 16387_vm 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 17694720 May 24 11:54 16393 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:54 16393_fsm 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:54 16393_vm 2024-05-24T11:54:31.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 493477888 May 24 11:54 16396 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 139264 May 24 11:53 16396_fsm 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:48 16396_vm 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 24 11:54 16397 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 647168 May 24 11:54 16399 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 138493952 May 24 11:54 16401 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 52011008 May 24 11:54 16403 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:54 16403_fsm 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 174 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 175 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2187 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2224_vm 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328 2024-05-24T11:54:31.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2328_vm 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2336_vm 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2337 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2600 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2600_fsm 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2600_vm 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2601_fsm 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2601_vm 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2602 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2602_fsm 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2602_vm 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2603 2024-05-24T11:54:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2603_fsm 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2603_vm 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2604_vm 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2605 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2605_fsm 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2605_vm 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:39 2606 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2606_fsm 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:39 2606_vm 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2607_fsm 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2607_vm 2024-05-24T11:54:31.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 May 24 11:39 2608 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:39 2608_fsm 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:39 2608_vm 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 May 24 11:38 2609 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2609_fsm 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2609_vm 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:39 2610 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:39 2610_fsm 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:39 2610_vm 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2611_vm 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2612_fsm 2024-05-24T11:54:31.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2612_vm 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2613_vm 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2615_fsm 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2615_vm 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2616_fsm 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2616_vm 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 24 11:38 2617 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2617_fsm 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2617_vm 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 24 11:38 2618 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2618_fsm 2024-05-24T11:54:31.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2618_vm 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 24 11:54 2619 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:53 2619_fsm 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:53 2619_vm 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2620_vm 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2650 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2651 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2652 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2653 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2654 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2655 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2656 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2657 2024-05-24T11:54:31.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 24 11:39 2658 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:39 2659 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2660 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2661 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:39 2662 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:39 2663 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:39 2664 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:39 2665 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:39 2666 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:39 2667 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2668 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2669 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2670 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 May 24 11:39 2673 2024-05-24T11:54:31.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 May 24 11:39 2674 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 24 11:38 2675 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:39 2678 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:39 2679 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2680 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2681 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2682 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2683 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2684 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2685 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2686 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2687 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:38 2688 2024-05-24T11:54:31.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 24 11:38 2689 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 24 11:38 2690 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 May 24 11:38 2691 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2692 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2693 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:54 2696 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2699 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2701 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2702 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:39 2703 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:39 2704 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2753 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2753_fsm 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2753_vm 2024-05-24T11:54:31.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2754 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2755 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2756 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 2757 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2830_vm 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2831 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2832_vm 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2833 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2834_vm 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2835 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836 2024-05-24T11:54:31.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2836_vm 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2837 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 May 24 11:38 2838 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2838_fsm 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2838_vm 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2839 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2840_fsm 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2840_vm 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2841 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2995_vm 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2996 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3079_fsm 2024-05-24T11:54:31.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3079_vm 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3080 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3081 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3085 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3118_vm 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3119 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3164 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3256_vm 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3257 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3258 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350 2024-05-24T11:54:31.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3350_vm 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3351 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3379 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3380 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3381_vm 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3394 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3394_fsm 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3394_vm 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3395 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3439_vm 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3440 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 24 11:39 3455 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3456 2024-05-24T11:54:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3456_fsm 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3456_vm 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3466_vm 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3467 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3468 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3501_vm 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3502 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3503 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3534 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3541_fsm 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3541_vm 2024-05-24T11:54:31.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3542 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3574 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3575 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3576_vm 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3596_vm 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3597 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3598_vm 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3599 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3600_fsm 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3600_vm 2024-05-24T11:54:31.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3601_fsm 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3601_vm 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3602_fsm 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3602_vm 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3603 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3603_fsm 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3603_vm 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3604 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3605 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3606 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3607 2024-05-24T11:54:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3608 2024-05-24T11:54:31.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 24 11:38 3609 2024-05-24T11:54:31.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3712 2024-05-24T11:54:31.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764 2024-05-24T11:54:31.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 3764_fsm 2024-05-24T11:54:31.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3764_vm 2024-05-24T11:54:31.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3766 2024-05-24T11:54:31.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 3767 2024-05-24T11:54:31.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3997 2024-05-24T11:54:31.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 5002 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 548 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 549 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6102_vm 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6104_vm 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6106_vm 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6110 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6111 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6112 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6113 2024-05-24T11:54:31.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6117 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 826_vm 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 827 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 828 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 24 11:38 pg_filenode.map 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 May 24 11:38 pg_internal.init 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 24 11:38 PG_VERSION 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:39 . 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 746750093 May 24 11:39 .. 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.733 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/global: 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:total 575 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 587404 May 24 11:38 . 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1136 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1136_fsm 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1136_vm 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 1137 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1213 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1213_fsm 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1213_vm 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1214 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1214_fsm 2024-05-24T11:54:31.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1214_vm 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 1232 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 1233 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1260 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1260_fsm 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1260_vm 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1261 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1261_fsm 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1261_vm 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1262 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 1262_fsm 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 1262_vm 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2396 2024-05-24T11:54:31.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 24 11:38 2396_fsm 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2396_vm 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2397 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2671 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2672 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2676 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2677 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2694 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2695 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2697 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 2698 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2846 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2846_vm 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2847 2024-05-24T11:54:31.736 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2964 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2964_vm 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2965 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2966 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 2966_vm 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 2967 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3592 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 3592_vm 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 3593 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 4060 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 4060_vm 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 4061 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6000 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6000_vm 2024-05-24T11:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6001 2024-05-24T11:54:31.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6002 2024-05-24T11:54:31.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6100 2024-05-24T11:54:31.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 6100_vm 2024-05-24T11:54:31.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6114 2024-05-24T11:54:31.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 6115 2024-05-24T11:54:31.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:54 pg_control 2024-05-24T11:54:31.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 24 11:38 pg_filenode.map 2024-05-24T11:54:31.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 May 24 11:38 pg_internal.init 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/log: 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 522 May 24 11:38 . 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 594 May 24 11:54 postgresql-Fri.log 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.777 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:total 3 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:54 . 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 mappings 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:54 replorigin_checkpoint 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 snapshots 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:54 .. 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 24 11:54 .. 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 . 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 members 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 offsets 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 . 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 .. 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 0000 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 . 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 24 11:38 .. 2024-05-24T11:54:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:39 0000 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 . 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 24 11:38 0000 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.863 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.863 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.864 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:total 14 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 May 24 11:54 . 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 May 24 11:54 db_0.stat 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 May 24 11:54 db_12347.stat 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 May 24 11:54 global.stat 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.865 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:54 . 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout:total 185 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 May 24 11:54 . 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 May 24 11:54 000E 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.866 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout:total 1032194 2024-05-24T11:54:31.867 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1056964608 May 24 11:54 . 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000046 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000047 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000048 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000049 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 00000001000000010000004A 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 00000001000000010000004B 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 00000001000000010000004C 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 00000001000000010000004D 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 00000001000000010000004E 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 00000001000000010000004F 2024-05-24T11:54:31.868 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000050 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000051 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000052 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000053 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000054 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000055 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000056 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000057 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:54 000000010000000100000058 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:54 000000010000000100000059 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:54 00000001000000010000005A 2024-05-24T11:54:31.869 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:54 00000001000000010000005B 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:54 00000001000000010000005C 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:50 00000001000000010000005D 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:50 00000001000000010000005E 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:50 00000001000000010000005F 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 000000010000000100000060 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:50 000000010000000100000061 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:50 000000010000000100000062 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:50 000000010000000100000063 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 000000010000000100000064 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:50 000000010000000100000065 2024-05-24T11:54:31.870 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:50 000000010000000100000066 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 000000010000000100000067 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 000000010000000100000068 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:50 000000010000000100000069 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 00000001000000010000006A 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 00000001000000010000006B 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 00000001000000010000006C 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 00000001000000010000006D 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 00000001000000010000006E 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 00000001000000010000006F 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000070 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000071 2024-05-24T11:54:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000072 2024-05-24T11:54:31.872 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 000000010000000100000073 2024-05-24T11:54:31.872 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000074 2024-05-24T11:54:31.872 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 000000010000000100000075 2024-05-24T11:54:31.872 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:53 000000010000000100000076 2024-05-24T11:54:31.872 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000077 2024-05-24T11:54:31.872 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 000000010000000100000078 2024-05-24T11:54:31.872 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000079 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 00000001000000010000007A 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 00000001000000010000007B 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 00000001000000010000007C 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 00000001000000010000007D 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 00000001000000010000007E 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:51 00000001000000010000007F 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000080 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000081 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000082 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000083 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 24 11:52 000000010000000100000084 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 archive_status 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.873 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-24T11:54:31.874 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-24T11:54:31.874 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 24 11:38 . 2024-05-24T11:54:31.874 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1056964608 May 24 11:54 .. 2024-05-24T11:54:31.874 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:31.874 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-24T11:54:31.874 INFO:teuthology.orchestra.run.smithi039.stdout:total 241 2024-05-24T11:54:31.874 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 245760 May 24 11:38 . 2024-05-24T11:54:31.874 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1804802224 May 24 11:54 .. 2024-05-24T11:54:31.874 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 245760 May 24 11:54 0000 2024-05-24T11:54:31.875 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-24T11:54:31.887 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:54:31.907 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-24T11:54:31.921 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-24T11:54:31.921 INFO:teuthology.task.exec:Executing custom commands... 2024-05-24T11:54:31.922 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:54:31.922 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-24T11:54:31.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-24T11:54:32.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-24T11:54:32.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.239+0000 7faf6d43c700 1 -- 172.21.15.39:0/373928384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 msgr2=0x7faf6810dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.239+0000 7faf6d43c700 1 --2- 172.21.15.39:0/373928384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 0x7faf6810dca0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faf6000b0a0 tx=0x7faf6000b3b0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.239+0000 7faf6d43c700 1 -- 172.21.15.39:0/373928384 shutdown_connections 2024-05-24T11:54:32.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.239+0000 7faf6d43c700 1 --2- 172.21.15.39:0/373928384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf68107000 0x7faf681073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.239+0000 7faf6d43c700 1 --2- 172.21.15.39:0/373928384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 0x7faf6810dca0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.239+0000 7faf6d43c700 1 --2- 172.21.15.39:0/373928384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf680682e0 0x7faf68068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.239+0000 7faf6d43c700 1 -- 172.21.15.39:0/373928384 >> 172.21.15.39:0/373928384 conn(0x7faf68076f10 msgr2=0x7faf68077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.239+0000 7faf6d43c700 1 -- 172.21.15.39:0/373928384 shutdown_connections 2024-05-24T11:54:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.239+0000 7faf6d43c700 1 -- 172.21.15.39:0/373928384 wait complete. 2024-05-24T11:54:32.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf6d43c700 1 Processor -- start 2024-05-24T11:54:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf6d43c700 1 -- start start 2024-05-24T11:54:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf6d43c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf680682e0 0x7faf681972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf6d43c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 0x7faf681977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf6d43c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf68107000 0x7faf6819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf6d43c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6819c1c0 con 0x7faf68068ca0 2024-05-24T11:54:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf6d43c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6819c330 con 0x7faf68107000 2024-05-24T11:54:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf6d43c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6819c4f0 con 0x7faf680682e0 2024-05-24T11:54:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf667fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 0x7faf681977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf66ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf680682e0 0x7faf681972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf667fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 0x7faf681977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47148/0 (socket says 172.21.15.39:47148) 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf667fc700 1 -- 172.21.15.39:0/3347878619 learned_addr learned my addr 172.21.15.39:0/3347878619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.240+0000 7faf677fe700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf68107000 0x7faf6819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf667fc700 1 -- 172.21.15.39:0/3347878619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf680682e0 msgr2=0x7faf681972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf667fc700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf680682e0 0x7faf681972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf667fc700 1 -- 172.21.15.39:0/3347878619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf68107000 msgr2=0x7faf6819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf667fc700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf68107000 0x7faf6819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf667fc700 1 -- 172.21.15.39:0/3347878619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60009d00 con 0x7faf68068ca0 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf677fe700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf68107000 0x7faf6819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf66ffd700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf680682e0 0x7faf681972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-24T11:54:32.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf667fc700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 0x7faf681977e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00e9a0 tx=0x7faf5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:32.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf4ffff700 1 -- 172.21.15.39:0/3347878619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c009790 con 0x7faf68068ca0 2024-05-24T11:54:32.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf4ffff700 1 -- 172.21.15.39:0/3347878619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf5c010430 con 0x7faf68068ca0 2024-05-24T11:54:32.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6819c7d0 con 0x7faf68068ca0 2024-05-24T11:54:32.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf4ffff700 1 -- 172.21.15.39:0/3347878619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00cea0 con 0x7faf68068ca0 2024-05-24T11:54:32.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.241+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf681a4010 con 0x7faf68068ca0 2024-05-24T11:54:32.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.242+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf68109380 con 0x7faf68068ca0 2024-05-24T11:54:32.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.244+0000 7faf4ffff700 1 -- 172.21.15.39:0/3347878619 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faf5c0105a0 con 0x7faf68068ca0 2024-05-24T11:54:32.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.245+0000 7faf4ffff700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7faf500a1290 0x7faf500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:32.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.245+0000 7faf66ffd700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7faf500a1290 0x7faf500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:32.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.245+0000 7faf4ffff700 1 -- 172.21.15.39:0/3347878619 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7faf5c014070 con 0x7faf68068ca0 2024-05-24T11:54:32.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.245+0000 7faf66ffd700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7faf500a1290 0x7faf500a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf58011490 tx=0x7faf58009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:32.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.248+0000 7faf4ffff700 1 -- 172.21.15.39:0/3347878619 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf5c096510 con 0x7faf68068ca0 2024-05-24T11:54:32.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:32 smithi053 ceph-mon[47448]: pgmap v519: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-24T11:54:32.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:32 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:54:32.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:32 smithi137 ceph-mon[47907]: pgmap v519: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-24T11:54:32.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:32 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:54:32.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.490+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 --> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7faf68110c20 con 0x7faf500a1290 2024-05-24T11:54:32.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.498+0000 7faf4ffff700 1 -- 172.21.15.39:0/3347878619 <== mgr.15360 v2:172.21.15.39:6800/1882625876 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7faf68110c20 con 0x7faf500a1290 2024-05-24T11:54:32.501 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-24T00:00:00", "created": "2024-05-24T11:38:13", "first": "2024-05-24T11:39:00", "last": "2024-05-24T11:54:00", "last_pruned": "2024-05-24T11:54:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-05-24T11:54:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7faf500a1290 msgr2=0x7faf500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7faf500a1290 0x7faf500a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf58011490 tx=0x7faf58009260 comp rx=0 tx=0).stop 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 msgr2=0x7faf681977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 0x7faf681977e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00e9a0 tx=0x7faf5c00ecb0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 shutdown_connections 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf680682e0 0x7faf681972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7faf500a1290 0x7faf500a3750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf68107000 0x7faf6819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 --2- 172.21.15.39:0/3347878619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf68068ca0 0x7faf681977e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 >> 172.21.15.39:0/3347878619 conn(0x7faf68076f10 msgr2=0x7faf680fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 shutdown_connections 2024-05-24T11:54:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.500+0000 7faf6d43c700 1 -- 172.21.15.39:0/3347878619 wait complete. 2024-05-24T11:54:32.514 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:32 smithi039 ceph-mon[42919]: pgmap v519: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-24T11:54:32.515 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:32 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-24T11:54:32.516 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-24T11:54:32.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.649+0000 7f4f85f26700 1 -- 172.21.15.39:0/687582114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f80100760 msgr2=0x7f4f8010cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.649+0000 7f4f85f26700 1 --2- 172.21.15.39:0/687582114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f80100760 0x7f4f8010cc00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800b0a0 tx=0x7f4f7800b3b0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.649+0000 7f4f85f26700 1 -- 172.21.15.39:0/687582114 shutdown_connections 2024-05-24T11:54:32.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.649+0000 7f4f85f26700 1 --2- 172.21.15.39:0/687582114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f80105e60 0x7f4f80106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.649+0000 7f4f85f26700 1 --2- 172.21.15.39:0/687582114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f80100760 0x7f4f8010cc00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.649+0000 7f4f85f26700 1 --2- 172.21.15.39:0/687582114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f800ffda0 0x7f4f80100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.649+0000 7f4f85f26700 1 -- 172.21.15.39:0/687582114 >> 172.21.15.39:0/687582114 conn(0x7f4f80076f10 msgr2=0x7f4f80077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:32.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.649+0000 7f4f85f26700 1 -- 172.21.15.39:0/687582114 shutdown_connections 2024-05-24T11:54:32.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.649+0000 7f4f85f26700 1 -- 172.21.15.39:0/687582114 wait complete. 2024-05-24T11:54:32.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.650+0000 7f4f85f26700 1 Processor -- start 2024-05-24T11:54:32.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.650+0000 7f4f85f26700 1 -- start start 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f85f26700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f800ffda0 0x7f4f80196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f85f26700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f80100760 0x7f4f80196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f85f26700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f80105e60 0x7f4f8019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f85f26700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8019b0e0 con 0x7f4f800ffda0 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f85f26700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8019b250 con 0x7f4f80105e60 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f85f26700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8019b410 con 0x7f4f80100760 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f800ffda0 0x7f4f80196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f800ffda0 0x7f4f80196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47172/0 (socket says 172.21.15.39:47172) 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7f7fe700 1 -- 172.21.15.39:0/538760987 learned_addr learned my addr 172.21.15.39:0/538760987 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7effd700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f80100760 0x7f4f80196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7ffff700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f80105e60 0x7f4f8019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:32.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7f7fe700 1 -- 172.21.15.39:0/538760987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f80100760 msgr2=0x7f4f80196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7f7fe700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f80100760 0x7f4f80196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7f7fe700 1 -- 172.21.15.39:0/538760987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f80105e60 msgr2=0x7f4f8019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7f7fe700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f80105e60 0x7f4f8019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7f7fe700 1 -- 172.21.15.39:0/538760987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f78009d00 con 0x7f4f800ffda0 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7ffff700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f80105e60 0x7f4f8019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7f7fe700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f800ffda0 0x7f4f80196210 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000e9a0 tx=0x7f4f7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7cff9700 1 -- 172.21.15.39:0/538760987 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70009790 con 0x7f4f800ffda0 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7cff9700 1 -- 172.21.15.39:0/538760987 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f7000cbd0 con 0x7f4f800ffda0 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8019b6f0 con 0x7f4f800ffda0 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f7cff9700 1 -- 172.21.15.39:0/538760987 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70018680 con 0x7f4f800ffda0 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.651+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f801a2f30 con 0x7f4f800ffda0 2024-05-24T11:54:32.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.652+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f801081d0 con 0x7f4f800ffda0 2024-05-24T11:54:32.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.653+0000 7f4f7cff9700 1 -- 172.21.15.39:0/538760987 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4f7000cd40 con 0x7f4f800ffda0 2024-05-24T11:54:32.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.654+0000 7f4f7cff9700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4f680a1290 0x7f4f680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:32.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.654+0000 7f4f7effd700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4f680a1290 0x7f4f680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:32.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.654+0000 7f4f7cff9700 1 -- 172.21.15.39:0/538760987 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4f70014070 con 0x7f4f800ffda0 2024-05-24T11:54:32.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.655+0000 7f4f7effd700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4f680a1290 0x7f4f680a3750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4f80197710 tx=0x7f4f7400b410 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:32.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.658+0000 7f4f7cff9700 1 -- 172.21.15.39:0/538760987 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4f70096770 con 0x7f4f800ffda0 2024-05-24T11:54:32.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.899+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 --> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4f8010fb30 con 0x7f4f680a1290 2024-05-24T11:54:32.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.905+0000 7f4f7cff9700 1 -- 172.21.15.39:0/538760987 <== mgr.15360 v2:172.21.15.39:6800/1882625876 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f4f8010fb30 con 0x7f4f680a1290 2024-05-24T11:54:32.908 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1m 6m3h 2024-05-24T11:54:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4f680a1290 msgr2=0x7f4f680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4f680a1290 0x7f4f680a3750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4f80197710 tx=0x7f4f7400b410 comp rx=0 tx=0).stop 2024-05-24T11:54:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f800ffda0 msgr2=0x7f4f80196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f800ffda0 0x7f4f80196210 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000e9a0 tx=0x7f4f7000ecb0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 shutdown_connections 2024-05-24T11:54:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f4f680a1290 0x7f4f680a3750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f80105e60 0x7f4f8019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f800ffda0 0x7f4f80196210 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 --2- 172.21.15.39:0/538760987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f80100760 0x7f4f80196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 >> 172.21.15.39:0/538760987 conn(0x7f4f80076f10 msgr2=0x7f4f800fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 shutdown_connections 2024-05-24T11:54:32.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:32.908+0000 7f4f85f26700 1 -- 172.21.15.39:0/538760987 wait complete. 2024-05-24T11:54:32.923 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-24T11:54:32.933 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:54:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.073+0000 7fd3c6801700 1 -- 172.21.15.39:0/4199071317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3c0102920 msgr2=0x7fd3c010ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.073+0000 7fd3c6801700 1 --2- 172.21.15.39:0/4199071317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3c0102920 0x7fd3c010ae00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009990 tx=0x7fd3b4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.073+0000 7fd3c6801700 1 -- 172.21.15.39:0/4199071317 shutdown_connections 2024-05-24T11:54:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.073+0000 7fd3c6801700 1 --2- 172.21.15.39:0/4199071317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3c0102000 0x7fd3c01023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.073+0000 7fd3c6801700 1 --2- 172.21.15.39:0/4199071317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c010b3d0 0x7fd3c010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.073+0000 7fd3c6801700 1 --2- 172.21.15.39:0/4199071317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3c0102920 0x7fd3c010ae00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.073+0000 7fd3c6801700 1 -- 172.21.15.39:0/4199071317 >> 172.21.15.39:0/4199071317 conn(0x7fd3c00fb7e0 msgr2=0x7fd3c00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.073+0000 7fd3c6801700 1 -- 172.21.15.39:0/4199071317 shutdown_connections 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.073+0000 7fd3c6801700 1 -- 172.21.15.39:0/4199071317 wait complete. 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3c6801700 1 Processor -- start 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3c6801700 1 -- start start 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3c6801700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c0102000 0x7fd3c01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3c6801700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3c0102920 0x7fd3c01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3c6801700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3c010b3d0 0x7fd3c019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3c6801700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c019c1d0 con 0x7fd3c010b3d0 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3c6801700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c019c340 con 0x7fd3c0102000 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3c6801700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c019c500 con 0x7fd3c0102920 2024-05-24T11:54:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c0102000 0x7fd3c01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3c4d9e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3c010b3d0 0x7fd3c019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c0102000 0x7fd3c01972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.39:54456/0 (socket says 172.21.15.39:54456) 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3bffff700 1 -- 172.21.15.39:0/2913845251 learned_addr learned my addr 172.21.15.39:0/2913845251 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.074+0000 7fd3bf7fe700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3c0102920 0x7fd3c01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3c4d9e700 1 -- 172.21.15.39:0/2913845251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3c0102920 msgr2=0x7fd3c01977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3c4d9e700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3c0102920 0x7fd3c01977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3c4d9e700 1 -- 172.21.15.39:0/2913845251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c0102000 msgr2=0x7fd3c01972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3c4d9e700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c0102000 0x7fd3c01972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3c4d9e700 1 -- 172.21.15.39:0/2913845251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b40095f0 con 0x7fd3c010b3d0 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3c4d9e700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3c010b3d0 0x7fd3c019bb90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800be20 tx=0x7fd3b8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3bd7fa700 1 -- 172.21.15.39:0/2913845251 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800cb90 con 0x7fd3c010b3d0 2024-05-24T11:54:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c019c7e0 con 0x7fd3c010b3d0 2024-05-24T11:54:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3bd7fa700 1 -- 172.21.15.39:0/2913845251 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd3b800ccf0 con 0x7fd3c010b3d0 2024-05-24T11:54:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3bd7fa700 1 -- 172.21.15.39:0/2913845251 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80125a0 con 0x7fd3c010b3d0 2024-05-24T11:54:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.075+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c01a4020 con 0x7fd3c010b3d0 2024-05-24T11:54:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.076+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c01064f0 con 0x7fd3c010b3d0 2024-05-24T11:54:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.077+0000 7fd3bd7fa700 1 -- 172.21.15.39:0/2913845251 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd3b8014440 con 0x7fd3c010b3d0 2024-05-24T11:54:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.078+0000 7fd3bd7fa700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd3a80a1290 0x7fd3a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.078+0000 7fd3bffff700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd3a80a1290 0x7fd3a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.078+0000 7fd3bd7fa700 1 -- 172.21.15.39:0/2913845251 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd3b8006e40 con 0x7fd3c010b3d0 2024-05-24T11:54:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.078+0000 7fd3bffff700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd3a80a1290 0x7fd3a80a3750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0005950 tx=0x7fd3b000a400 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.082+0000 7fd3bd7fa700 1 -- 172.21.15.39:0/2913845251 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd3b80959d0 con 0x7fd3c010b3d0 2024-05-24T11:54:33.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.291+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3c0068870 con 0x7fd3c010b3d0 2024-05-24T11:54:33.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.292+0000 7fd3bd7fa700 1 -- 172.21.15.39:0/2913845251 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+22875 (secure 0 0 0) 0x7fd3b800ce60 con 0x7fd3c010b3d0 2024-05-24T11:54:33.296 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:33.296 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":107,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:54:19.813504+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/338214439":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/870992183":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/3248772673":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/3290494740":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/333059399":"2024-05-25T11:38:12.183530+0000","172.21.15.39:6801/2090515453":"2024-05-25T11:38:12.183530+0000","172.21.15.39:6800/2090515453":"2024-05-25T11:38:12.183530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:54:33.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd3a80a1290 msgr2=0x7fd3a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd3a80a1290 0x7fd3a80a3750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0005950 tx=0x7fd3b000a400 comp rx=0 tx=0).stop 2024-05-24T11:54:33.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3c010b3d0 msgr2=0x7fd3c019bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3c010b3d0 0x7fd3c019bb90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800be20 tx=0x7fd3b8009ed0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 shutdown_connections 2024-05-24T11:54:33.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd3a80a1290 0x7fd3a80a3750 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3c010b3d0 0x7fd3c019bb90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c0102000 0x7fd3c01972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 --2- 172.21.15.39:0/2913845251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3c0102920 0x7fd3c01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 >> 172.21.15.39:0/2913845251 conn(0x7fd3c00fb7e0 msgr2=0x7fd3c00fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:33.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 shutdown_connections 2024-05-24T11:54:33.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.295+0000 7fd3c6801700 1 -- 172.21.15.39:0/2913845251 wait complete. 2024-05-24T11:54:33.309 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:33 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:54:33.310 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:33 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:54:33.310 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:33 smithi039 ceph-mon[42919]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:54:33.311 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-24T11:36:46.509049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-24T11:36:47.866055+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:54:33.312 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-24T11:54:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.439+0000 7f2bb21c4700 1 -- 172.21.15.39:0/2352919176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac105160 msgr2=0x7f2bac10cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.439+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/2352919176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac105160 0x7f2bac10cce0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b0a0 tx=0x7f2ba400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.439+0000 7f2bb21c4700 1 -- 172.21.15.39:0/2352919176 shutdown_connections 2024-05-24T11:54:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.439+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/2352919176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac100060 0x7f2bac100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.439+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/2352919176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac105160 0x7f2bac10cce0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.439+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/2352919176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bac100a10 0x7f2bac104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.439+0000 7f2bb21c4700 1 -- 172.21.15.39:0/2352919176 >> 172.21.15.39:0/2352919176 conn(0x7f2bac076f40 msgr2=0x7f2bac077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.439+0000 7f2bb21c4700 1 -- 172.21.15.39:0/2352919176 shutdown_connections 2024-05-24T11:54:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.439+0000 7f2bb21c4700 1 -- 172.21.15.39:0/2352919176 wait complete. 2024-05-24T11:54:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bb21c4700 1 Processor -- start 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bb21c4700 1 -- start start 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bb21c4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac100060 0x7f2bac1973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bb21c4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac105160 0x7f2bac197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bb21c4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bac19bcd0 0x7f2bac19c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bb21c4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac19c750 con 0x7f2bac100060 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bb21c4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac19c8c0 con 0x7f2bac19bcd0 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bb21c4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac19ca80 con 0x7f2bac105160 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bab7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac100060 0x7f2bac1973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bab7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac100060 0x7f2bac1973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47210/0 (socket says 172.21.15.39:47210) 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bab7fe700 1 -- 172.21.15.39:0/3846389837 learned_addr learned my addr 172.21.15.39:0/3846389837 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:33.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2babfff700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bac19bcd0 0x7f2bac19c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2baaffd700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac105160 0x7f2bac197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bab7fe700 1 -- 172.21.15.39:0/3846389837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac105160 msgr2=0x7f2bac197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bab7fe700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac105160 0x7f2bac197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bab7fe700 1 -- 172.21.15.39:0/3846389837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bac19bcd0 msgr2=0x7f2bac19c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.440+0000 7f2bab7fe700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bac19bcd0 0x7f2bac19c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.441+0000 7f2bab7fe700 1 -- 172.21.15.39:0/3846389837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba4009d00 con 0x7f2bac100060 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.441+0000 7f2bab7fe700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac100060 0x7f2bac1973f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000d950 tx=0x7f2ba000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.441+0000 7f2ba8ff9700 1 -- 172.21.15.39:0/3846389837 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0009940 con 0x7f2bac100060 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.441+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bac1a3fe0 con 0x7f2bac100060 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.441+0000 7f2ba8ff9700 1 -- 172.21.15.39:0/3846389837 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ba0010460 con 0x7f2bac100060 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.441+0000 7f2ba8ff9700 1 -- 172.21.15.39:0/3846389837 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000f620 con 0x7f2bac100060 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.441+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac1a44b0 con 0x7f2bac100060 2024-05-24T11:54:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.442+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac109440 con 0x7f2bac100060 2024-05-24T11:54:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.444+0000 7f2ba8ff9700 1 -- 172.21.15.39:0/3846389837 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2ba00105d0 con 0x7f2bac100060 2024-05-24T11:54:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.444+0000 7f2ba8ff9700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2b940a1290 0x7f2b940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.444+0000 7f2baaffd700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2b940a1290 0x7f2b940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.444+0000 7f2ba8ff9700 1 -- 172.21.15.39:0/3846389837 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2ba00ccb40 con 0x7f2bac100060 2024-05-24T11:54:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.445+0000 7f2baaffd700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2b940a1290 0x7f2b940a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c005fd0 tx=0x7f2b9c005ee0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.448+0000 7f2ba8ff9700 1 -- 172.21.15.39:0/3846389837 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2ba0095fa0 con 0x7f2bac100060 2024-05-24T11:54:33.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:33 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:54:33.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:33 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:54:33.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:33 smithi053 ceph-mon[47448]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:54:33.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:33 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-24T11:54:33.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:33 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-24T11:54:33.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:33 smithi137 ceph-mon[47907]: from='mgr.15360 172.21.15.39:0/867873879' entity='mgr.x' 2024-05-24T11:54:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.652+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2bac1986f0 con 0x7f2bac100060 2024-05-24T11:54:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.652+0000 7f2ba8ff9700 1 -- 172.21.15.39:0/3846389837 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ba00160a0 con 0x7f2bac100060 2024-05-24T11:54:33.656 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:54:33.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2b940a1290 msgr2=0x7f2b940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2b940a1290 0x7f2b940a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c005fd0 tx=0x7f2b9c005ee0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac100060 msgr2=0x7f2bac1973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac100060 0x7f2bac1973f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000d950 tx=0x7f2ba000dc60 comp rx=0 tx=0).stop 2024-05-24T11:54:33.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 shutdown_connections 2024-05-24T11:54:33.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2b940a1290 0x7f2b940a3750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bac19bcd0 0x7f2bac19c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac100060 0x7f2bac1973f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 --2- 172.21.15.39:0/3846389837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac105160 0x7f2bac197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 >> 172.21.15.39:0/3846389837 conn(0x7f2bac076f40 msgr2=0x7f2bac068390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 shutdown_connections 2024-05-24T11:54:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.655+0000 7f2bb21c4700 1 -- 172.21.15.39:0/3846389837 wait complete. 2024-05-24T11:54:33.671 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-24T11:54:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.797+0000 7fc8bd127700 1 -- 172.21.15.39:0/3250221061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8101b10 msgr2=0x7fc8b8105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.797+0000 7fc8bd127700 1 --2- 172.21.15.39:0/3250221061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8101b10 0x7fc8b8105b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009990 tx=0x7fc8a8009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.797+0000 7fc8bd127700 1 -- 172.21.15.39:0/3250221061 shutdown_connections 2024-05-24T11:54:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.797+0000 7fc8bd127700 1 --2- 172.21.15.39:0/3250221061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b8106280 0x7fc8b810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.797+0000 7fc8bd127700 1 --2- 172.21.15.39:0/3250221061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b8101160 0x7fc8b8101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.797+0000 7fc8bd127700 1 --2- 172.21.15.39:0/3250221061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8101b10 0x7fc8b8105b50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.797+0000 7fc8bd127700 1 -- 172.21.15.39:0/3250221061 >> 172.21.15.39:0/3250221061 conn(0x7fc8b80fc9e0 msgr2=0x7fc8b80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.798+0000 7fc8bd127700 1 -- 172.21.15.39:0/3250221061 shutdown_connections 2024-05-24T11:54:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.798+0000 7fc8bd127700 1 -- 172.21.15.39:0/3250221061 wait complete. 2024-05-24T11:54:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.798+0000 7fc8bd127700 1 Processor -- start 2024-05-24T11:54:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8bd127700 1 -- start start 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8bd127700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b8101160 0x7fc8b8199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8bd127700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8106280 0x7fc8b8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8bd127700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b819de50 0x7fc8b819e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8bd127700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b819e8d0 con 0x7fc8b8106280 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8bd127700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b819ea40 con 0x7fc8b8101160 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8bd127700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b819ec00 con 0x7fc8b819de50 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8106280 0x7fc8b8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8106280 0x7fc8b8199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47220/0 (socket says 172.21.15.39:47220) 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b659c700 1 -- 172.21.15.39:0/1530620025 learned_addr learned my addr 172.21.15.39:0/1530620025 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b6d9d700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b8101160 0x7fc8b8199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b759e700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b819de50 0x7fc8b819e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b659c700 1 -- 172.21.15.39:0/1530620025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b819de50 msgr2=0x7fc8b819e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b659c700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b819de50 0x7fc8b819e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b659c700 1 -- 172.21.15.39:0/1530620025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b8101160 msgr2=0x7fc8b8199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b659c700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b8101160 0x7fc8b8199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b659c700 1 -- 172.21.15.39:0/1530620025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80095f0 con 0x7fc8b8106280 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.799+0000 7fc8b659c700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8106280 0x7fc8b8199ab0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005910 tx=0x7fc8a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.800+0000 7fc89ffff700 1 -- 172.21.15.39:0/1530620025 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a801d070 con 0x7fc8b8106280 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.800+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b81a6100 con 0x7fc8b8106280 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.800+0000 7fc89ffff700 1 -- 172.21.15.39:0/1530620025 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8a800bea0 con 0x7fc8b8106280 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.800+0000 7fc89ffff700 1 -- 172.21.15.39:0/1530620025 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a80173f0 con 0x7fc8b8106280 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.800+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b81a6520 con 0x7fc8b8106280 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.801+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b8102630 con 0x7fc8b8106280 2024-05-24T11:54:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.803+0000 7fc89ffff700 1 -- 172.21.15.39:0/1530620025 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc8a8022650 con 0x7fc8b8106280 2024-05-24T11:54:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.803+0000 7fc89ffff700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fc8a00a1290 0x7fc8a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.803+0000 7fc8b6d9d700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fc8a00a1290 0x7fc8a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.803+0000 7fc89ffff700 1 -- 172.21.15.39:0/1530620025 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc8a80ce6f0 con 0x7fc8b8106280 2024-05-24T11:54:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.804+0000 7fc8b6d9d700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fc8a00a1290 0x7fc8a00a3750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009c60 tx=0x7fc8ac009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:33.807+0000 7fc89ffff700 1 -- 172.21.15.39:0/1530620025 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc8a8097cf0 con 0x7fc8b8106280 2024-05-24T11:54:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.014+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8b819a870 con 0x7fc8b8106280 2024-05-24T11:54:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.015+0000 7fc89ffff700 1 -- 172.21.15.39:0/1530620025 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8a809b310 con 0x7fc8b8106280 2024-05-24T11:54:34.018 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fc8a00a1290 msgr2=0x7fc8a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fc8a00a1290 0x7fc8a00a3750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009c60 tx=0x7fc8ac009260 comp rx=0 tx=0).stop 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8106280 msgr2=0x7fc8b8199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8106280 0x7fc8b8199ab0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005910 tx=0x7fc8a8004730 comp rx=0 tx=0).stop 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 shutdown_connections 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b819de50 0x7fc8b819e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fc8a00a1290 0x7fc8a00a3750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8b8101160 0x7fc8b8199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 --2- 172.21.15.39:0/1530620025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8b8106280 0x7fc8b8199ab0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 >> 172.21.15.39:0/1530620025 conn(0x7fc8b80fc9e0 msgr2=0x7fc8b80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 shutdown_connections 2024-05-24T11:54:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.017+0000 7fc8bd127700 1 -- 172.21.15.39:0/1530620025 wait complete. 2024-05-24T11:54:34.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-24T11:54:34.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.161+0000 7fcf141ee700 1 -- 172.21.15.39:0/2912944566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c10d7f0 msgr2=0x7fcf0c10fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.161+0000 7fcf141ee700 1 --2- 172.21.15.39:0/2912944566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c10d7f0 0x7fcf0c10fbe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b0a0 tx=0x7fcf0400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.161+0000 7fcf141ee700 1 -- 172.21.15.39:0/2912944566 shutdown_connections 2024-05-24T11:54:34.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.161+0000 7fcf141ee700 1 --2- 172.21.15.39:0/2912944566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf0c101e20 0x7fcf0c0691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.161+0000 7fcf141ee700 1 --2- 172.21.15.39:0/2912944566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c10d7f0 0x7fcf0c10fbe0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.161+0000 7fcf141ee700 1 --2- 172.21.15.39:0/2912944566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf0c069710 0x7fcf0c069b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.161+0000 7fcf141ee700 1 -- 172.21.15.39:0/2912944566 >> 172.21.15.39:0/2912944566 conn(0x7fcf0c076ba0 msgr2=0x7fcf0c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.161+0000 7fcf141ee700 1 -- 172.21.15.39:0/2912944566 shutdown_connections 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.161+0000 7fcf141ee700 1 -- 172.21.15.39:0/2912944566 wait complete. 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf141ee700 1 Processor -- start 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf141ee700 1 -- start start 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf141ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf0c069710 0x7fcf0c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf141ee700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c101e20 0x7fcf0c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf141ee700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf0c10d7f0 0x7fcf0c19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf141ee700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c19e520 con 0x7fcf0c101e20 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf141ee700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c19e690 con 0x7fcf0c10d7f0 2024-05-24T11:54:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf141ee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c19e850 con 0x7fcf0c069710 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf11789700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c101e20 0x7fcf0c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf11789700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c101e20 0x7fcf0c199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47244/0 (socket says 172.21.15.39:47244) 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf11789700 1 -- 172.21.15.39:0/1611099621 learned_addr learned my addr 172.21.15.39:0/1611099621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf1278b700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf0c10d7f0 0x7fcf0c19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf11f8a700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf0c069710 0x7fcf0c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf11789700 1 -- 172.21.15.39:0/1611099621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf0c069710 msgr2=0x7fcf0c199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf11789700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf0c069710 0x7fcf0c199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf11789700 1 -- 172.21.15.39:0/1611099621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf0c10d7f0 msgr2=0x7fcf0c19dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf11789700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf0c10d7f0 0x7fcf0c19dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.162+0000 7fcf11789700 1 -- 172.21.15.39:0/1611099621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04009d00 con 0x7fcf0c101e20 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.163+0000 7fcf11789700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c101e20 0x7fcf0c199b40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00d980 tx=0x7fcefc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.163+0000 7fcf02ffd700 1 -- 172.21.15.39:0/1611099621 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc004220 con 0x7fcf0c101e20 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.163+0000 7fcf02ffd700 1 -- 172.21.15.39:0/1611099621 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcefc007640 con 0x7fcf0c101e20 2024-05-24T11:54:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.163+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c19eb30 con 0x7fcf0c101e20 2024-05-24T11:54:34.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.163+0000 7fcf02ffd700 1 -- 172.21.15.39:0/1611099621 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc005660 con 0x7fcf0c101e20 2024-05-24T11:54:34.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.163+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c1a6370 con 0x7fcf0c101e20 2024-05-24T11:54:34.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.164+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c100440 con 0x7fcf0c101e20 2024-05-24T11:54:34.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.165+0000 7fcf02ffd700 1 -- 172.21.15.39:0/1611099621 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcefc004750 con 0x7fcf0c101e20 2024-05-24T11:54:34.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.165+0000 7fcf02ffd700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcef80a1290 0x7fcef80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:34.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.166+0000 7fcf11f8a700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcef80a1290 0x7fcef80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:34.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.166+0000 7fcf02ffd700 1 -- 172.21.15.39:0/1611099621 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcefc0cc9e0 con 0x7fcf0c101e20 2024-05-24T11:54:34.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.166+0000 7fcf11f8a700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcef80a1290 0x7fcef80a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009be0 tx=0x7fcf08009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:34.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.170+0000 7fcf02ffd700 1 -- 172.21.15.39:0/1611099621 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcefc095e40 con 0x7fcf0c101e20 2024-05-24T11:54:34.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:34 smithi039 ceph-mon[42919]: from='client.16485 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:54:34.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:34 smithi039 ceph-mon[42919]: pgmap v520: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-05-24T11:54:34.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:34 smithi039 ceph-mon[42919]: from='client.16491 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:54:34.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:34 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2913845251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:54:34.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:34 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3846389837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:54:34.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:34 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1530620025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:54:34.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.380+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf0c19a930 con 0x7fcf0c101e20 2024-05-24T11:54:34.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.381+0000 7fcf02ffd700 1 -- 172.21.15.39:0/1611099621 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcefc016020 con 0x7fcf0c101e20 2024-05-24T11:54:34.384 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-05-24T11:54:34.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcef80a1290 msgr2=0x7fcef80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcef80a1290 0x7fcef80a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009be0 tx=0x7fcf08009260 comp rx=0 tx=0).stop 2024-05-24T11:54:34.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c101e20 msgr2=0x7fcf0c199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c101e20 0x7fcf0c199b40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00d980 tx=0x7fcefc00dc90 comp rx=0 tx=0).stop 2024-05-24T11:54:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 shutdown_connections 2024-05-24T11:54:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf0c069710 0x7fcf0c199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fcef80a1290 0x7fcef80a3750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf0c10d7f0 0x7fcf0c19dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 --2- 172.21.15.39:0/1611099621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0c101e20 0x7fcf0c199b40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 >> 172.21.15.39:0/1611099621 conn(0x7fcf0c076ba0 msgr2=0x7fcf0c10e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 shutdown_connections 2024-05-24T11:54:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.383+0000 7fcf141ee700 1 -- 172.21.15.39:0/1611099621 wait complete. 2024-05-24T11:54:34.398 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-24T11:54:34.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:34 smithi053 ceph-mon[47448]: from='client.16485 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:54:34.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:34 smithi053 ceph-mon[47448]: pgmap v520: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-05-24T11:54:34.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:34 smithi053 ceph-mon[47448]: from='client.16491 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:54:34.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:34 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2913845251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:54:34.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:34 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3846389837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:54:34.488 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:34 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1530620025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:54:34.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:34 smithi137 ceph-mon[47907]: from='client.16485 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:54:34.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:34 smithi137 ceph-mon[47907]: pgmap v520: 97 pgs: 97 active+clean; 5.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 12 MiB/s wr, 1.25k op/s 2024-05-24T11:54:34.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:34 smithi137 ceph-mon[47907]: from='client.16491 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-24T11:54:34.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:34 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2913845251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:54:34.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:34 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3846389837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:54:34.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:34 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1530620025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:54:34.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.522+0000 7eff1ff99700 1 -- 172.21.15.39:0/1073530211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff18102940 msgr2=0x7eff1810ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.522+0000 7eff1ff99700 1 --2- 172.21.15.39:0/1073530211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff18102940 0x7eff1810ae20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7eff14009990 tx=0x7eff14009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.522+0000 7eff1ff99700 1 -- 172.21.15.39:0/1073530211 shutdown_connections 2024-05-24T11:54:34.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.522+0000 7eff1ff99700 1 --2- 172.21.15.39:0/1073530211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18102020 0x7eff18102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.522+0000 7eff1ff99700 1 --2- 172.21.15.39:0/1073530211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff1810b3f0 0x7eff1810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.522+0000 7eff1ff99700 1 --2- 172.21.15.39:0/1073530211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff18102940 0x7eff1810ae20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.522+0000 7eff1ff99700 1 -- 172.21.15.39:0/1073530211 >> 172.21.15.39:0/1073530211 conn(0x7eff180fb7e0 msgr2=0x7eff180fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.522+0000 7eff1ff99700 1 -- 172.21.15.39:0/1073530211 shutdown_connections 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.522+0000 7eff1ff99700 1 -- 172.21.15.39:0/1073530211 wait complete. 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1ff99700 1 Processor -- start 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1ff99700 1 -- start start 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1ff99700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18102020 0x7eff181972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1ff99700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff18102940 0x7eff181977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1ff99700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1810b3f0 0x7eff1819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1ff99700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1819c170 con 0x7eff1810b3f0 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1ff99700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1819c2e0 con 0x7eff18102940 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1ff99700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1819c4a0 con 0x7eff18102020 2024-05-24T11:54:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1e536700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1810b3f0 0x7eff1819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1e536700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1810b3f0 0x7eff1819bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47254/0 (socket says 172.21.15.39:47254) 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1e536700 1 -- 172.21.15.39:0/3033758279 learned_addr learned my addr 172.21.15.39:0/3033758279 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1dd35700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18102020 0x7eff181972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1d534700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff18102940 0x7eff181977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1e536700 1 -- 172.21.15.39:0/3033758279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18102020 msgr2=0x7eff181972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1e536700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18102020 0x7eff181972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1e536700 1 -- 172.21.15.39:0/3033758279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff18102940 msgr2=0x7eff181977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1e536700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff18102940 0x7eff181977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1e536700 1 -- 172.21.15.39:0/3033758279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff140095f0 con 0x7eff1810b3f0 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.523+0000 7eff1e536700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1810b3f0 0x7eff1819bb30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7eff10007ee0 tx=0x7eff1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.524+0000 7eff0effd700 1 -- 172.21.15.39:0/3033758279 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10017070 con 0x7eff1810b3f0 2024-05-24T11:54:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.524+0000 7eff0effd700 1 -- 172.21.15.39:0/3033758279 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff1000f040 con 0x7eff1810b3f0 2024-05-24T11:54:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.524+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1819c780 con 0x7eff1810b3f0 2024-05-24T11:54:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.524+0000 7eff0effd700 1 -- 172.21.15.39:0/3033758279 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10005570 con 0x7eff1810b3f0 2024-05-24T11:54:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.524+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff181a3fc0 con 0x7eff1810b3f0 2024-05-24T11:54:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.525+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff18106510 con 0x7eff1810b3f0 2024-05-24T11:54:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.526+0000 7eff0effd700 1 -- 172.21.15.39:0/3033758279 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7eff1000db20 con 0x7eff1810b3f0 2024-05-24T11:54:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.526+0000 7eff0effd700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7eff040a1310 0x7eff040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.527+0000 7eff1dd35700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7eff040a1310 0x7eff040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.527+0000 7eff0effd700 1 -- 172.21.15.39:0/3033758279 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7eff100cc9d0 con 0x7eff1810b3f0 2024-05-24T11:54:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.527+0000 7eff1dd35700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7eff040a1310 0x7eff040a37d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff08007be0 tx=0x7eff080073d0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.530+0000 7eff0effd700 1 -- 172.21.15.39:0/3033758279 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7eff10095dd0 con 0x7eff1810b3f0 2024-05-24T11:54:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.800+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7eff18198550 con 0x7eff1810b3f0 2024-05-24T11:54:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.808+0000 7eff0effd700 1 -- 172.21.15.39:0/3033758279 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733942 (secure 0 0 0) 0x7eff100993f0 con 0x7eff1810b3f0 2024-05-24T11:54:34.821 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:34.821 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:54:34.821 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 25, 2024-05-24T11:54:34.822 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.822 INFO:teuthology.orchestra.run.smithi039.stdout: "active_gid": 15360, 2024-05-24T11:54:34.822 INFO:teuthology.orchestra.run.smithi039.stdout: "active_name": "x", 2024-05-24T11:54:34.822 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addrs": { 2024-05-24T11:54:34.822 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-24T11:54:34.822 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:34.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-24T11:54:34.822 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-05-24T11:54:34.822 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1882625876 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1882625876 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addr": "172.21.15.39:6801/1882625876", 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: "active_change": "2024-05-24T11:38:12.183744+0000", 2024-05-24T11:54:34.823 INFO:teuthology.orchestra.run.smithi039.stdout: "active_mgr_features": 4540138322906710015, 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "available": true, 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "standbys": [ 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 24976, 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "z", 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540138322906710015, 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:34.824 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:34.825 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:34.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.825 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.825 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:34.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.825 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.825 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.825 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.826 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.827 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.828 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.829 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:34.830 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.831 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.832 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-05-24T11:54:34.833 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.834 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.834 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.834 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-05-24T11:54:34.834 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.834 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.834 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.835 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-05-24T11:54:34.836 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.836 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.836 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.836 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-05-24T11:54:34.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-05-24T11:54:34.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-05-24T11:54:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-05-24T11:54:34.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.838 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.838 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.838 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-05-24T11:54:34.838 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.838 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.838 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.839 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.840 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-05-24T11:54:34.840 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.840 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.840 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.841 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-05-24T11:54:34.841 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-05-24T11:54:34.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.841 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.841 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.841 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.841 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.841 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.841 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.842 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:34.843 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.844 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.845 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.846 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-05-24T11:54:34.846 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-05-24T11:54:34.846 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:34.846 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.847 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:34.847 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-24T11:54:34.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-24T11:54:34.847 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.847 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-24T11:54:34.847 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-24T11:54:34.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.848 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.848 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-05-24T11:54:34.848 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-05-24T11:54:34.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:34.848 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.848 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.848 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-05-24T11:54:34.848 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-05-24T11:54:34.849 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.850 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.850 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.850 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-24T11:54:34.850 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-24T11:54:34.850 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.850 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.850 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-05-24T11:54:34.850 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.851 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-24T11:54:34.852 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-24T11:54:34.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:34.854 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:34.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap" 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.865 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.866 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-24T11:54:34.867 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.867 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.867 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.867 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-24T11:54:34.867 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.867 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.869 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-05-24T11:54:34.869 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-05-24T11:54:34.869 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.869 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.869 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.869 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-05-24T11:54:34.869 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.871 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.871 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.871 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.871 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.871 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.871 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-24T11:54:34.871 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-24T11:54:34.871 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.871 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-24T11:54:34.872 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.873 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-24T11:54:34.874 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.874 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.874 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.874 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.876 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.876 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-05-24T11:54:34.876 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-05-24T11:54:34.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.877 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-24T11:54:34.877 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.878 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.878 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.878 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-05-24T11:54:34.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-05-24T11:54:34.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.879 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.879 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.879 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.879 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.879 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.880 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-24T11:54:34.880 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.880 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.880 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-05-24T11:54:34.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-05-24T11:54:34.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.881 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.881 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.881 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-24T11:54:34.881 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-05-24T11:54:34.882 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.883 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.886 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-24T11:54:34.887 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.888 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.890 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.891 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.892 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.893 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.894 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.894 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-05-24T11:54:34.894 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-05-24T11:54:34.894 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.895 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.896 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-24T11:54:34.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.898 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.899 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-05-24T11:54:34.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.903 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-05-24T11:54:34.903 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-05-24T11:54:34.903 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.903 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.903 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-05-24T11:54:34.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.904 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.905 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.906 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.906 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:34.906 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.906 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.906 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.906 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-05-24T11:54:34.906 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.906 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.906 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.908 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-05-24T11:54:34.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.910 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.911 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:34.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:34.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.911 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.911 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.911 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.912 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:34.912 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:34.912 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:34.912 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:34.912 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:34.912 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:34.913 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.913 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.913 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.913 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.913 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.913 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-05-24T11:54:34.913 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-05-24T11:54:34.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.914 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.914 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.914 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.914 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.914 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.914 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.915 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.916 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.917 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.918 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:34.918 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:34.918 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.918 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.918 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.919 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.919 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.919 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.919 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.919 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.919 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.921 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-05-24T11:54:34.922 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.923 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.923 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.923 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-05-24T11:54:34.923 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-05-24T11:54:34.923 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.923 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.923 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.923 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:34.924 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.924 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.924 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.924 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-24T11:54:34.924 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.924 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.925 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.925 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-05-24T11:54:34.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-05-24T11:54:34.925 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.925 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.925 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.926 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.926 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.926 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.926 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-05-24T11:54:34.926 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.927 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.927 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-24T11:54:34.927 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-24T11:54:34.927 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.928 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.928 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.928 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-05-24T11:54:34.928 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.928 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.928 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.929 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-05-24T11:54:34.929 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-05-24T11:54:34.929 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.929 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-05-24T11:54:34.929 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.929 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.930 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.930 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-05-24T11:54:34.930 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-05-24T11:54:34.930 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.930 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.930 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.930 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.931 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.934 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.935 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.936 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.936 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.936 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-24T11:54:34.936 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.936 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.937 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.938 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-24T11:54:34.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.940 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-05-24T11:54:34.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-05-24T11:54:34.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.940 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-05-24T11:54:34.941 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.942 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-24T11:54:34.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.943 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.943 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-05-24T11:54:34.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-05-24T11:54:34.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-05-24T11:54:34.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.945 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-24T11:54:34.946 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.947 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.947 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:34.947 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:34.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:34.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-05-24T11:54:34.947 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:34.947 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:34.948 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:34.948 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:34.948 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:34.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.948 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.948 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.950 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.950 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:34.950 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:34.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.950 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.950 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.951 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.951 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.952 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:34.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:34.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:34.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:34.953 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:34.953 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:34.953 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:34.953 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:34.953 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:34.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-05-24T11:54:34.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-05-24T11:54:34.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.957 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.958 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.961 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-24T11:54:34.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-24T11:54:34.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-24T11:54:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.968 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.970 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.971 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.972 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.974 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-24T11:54:34.974 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.974 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.974 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.974 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.974 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.974 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.974 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.976 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.976 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-05-24T11:54:34.976 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-05-24T11:54:34.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.977 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.978 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.979 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.980 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.980 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.980 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.980 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.980 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.980 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-24T11:54:34.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-24T11:54:34.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.981 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.981 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.982 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-24T11:54:34.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.984 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.984 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.984 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.984 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.984 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.984 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.985 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.985 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.985 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-05-24T11:54:34.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-24T11:54:34.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.986 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.986 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:34.986 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.986 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.986 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.986 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.987 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.987 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.987 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-24T11:54:34.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-24T11:54:34.988 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-24T11:54:34.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.990 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.992 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.993 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-24T11:54:34.994 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-24T11:54:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-24T11:54:34.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:34.997 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:34.999 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-24T11:54:35.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-24T11:54:35.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.003 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.008 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.009 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.011 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.012 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-05-24T11:54:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-05-24T11:54:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-05-24T11:54:35.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.018 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.020 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.022 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-05-24T11:54:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-05-24T11:54:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.026 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-05-24T11:54:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-05-24T11:54:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-05-24T11:54:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-24T11:54:35.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-24T11:54:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-24T11:54:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.030 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-24T11:54:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-24T11:54:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.031 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-05-24T11:54:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-24T11:54:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-24T11:54:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-05-24T11:54:35.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-05-24T11:54:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-05-24T11:54:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-24T11:54:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-24T11:54:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.036 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.037 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.038 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.039 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.039 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-05-24T11:54:35.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-05-24T11:54:35.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.040 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.042 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.044 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.046 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.046 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.046 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.046 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.046 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.046 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.046 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.048 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.048 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.048 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.048 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.048 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.048 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-05-24T11:54:35.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-05-24T11:54:35.049 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.049 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.049 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-05-24T11:54:35.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-24T11:54:35.050 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.050 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.050 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-05-24T11:54:35.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-05-24T11:54:35.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-05-24T11:54:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-24T11:54:35.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.054 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-05-24T11:54:35.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-05-24T11:54:35.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.056 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.056 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.056 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.057 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-05-24T11:54:35.057 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.057 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.057 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-05-24T11:54:35.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-05-24T11:54:35.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-05-24T11:54:35.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-24T11:54:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.060 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.061 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.063 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.064 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-05-24T11:54:35.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.067 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-05-24T11:54:35.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-05-24T11:54:35.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.067 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-05-24T11:54:35.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.068 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.069 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.071 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-05-24T11:54:35.071 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-05-24T11:54:35.071 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-05-24T11:54:35.071 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.071 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-05-24T11:54:35.071 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-05-24T11:54:35.071 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.072 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.072 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-05-24T11:54:35.072 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.072 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.072 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.072 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-24T11:54:35.072 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-05-24T11:54:35.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.074 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.076 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-05-24T11:54:35.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-05-24T11:54:35.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-24T11:54:35.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.076 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.077 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.078 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.080 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.080 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.080 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.083 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.085 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-05-24T11:54:35.085 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.085 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.085 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-05-24T11:54:35.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.085 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.086 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-24T11:54:35.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-24T11:54:35.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-05-24T11:54:35.086 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.087 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-24T11:54:35.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.087 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-05-24T11:54:35.087 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-05-24T11:54:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-05-24T11:54:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-05-24T11:54:35.090 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-05-24T11:54:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-05-24T11:54:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.091 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.092 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-05-24T11:54:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.093 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.095 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.097 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.099 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.100 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.103 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.105 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.106 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.107 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.109 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.110 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.112 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.116 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.117 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.118 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.120 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.122 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-05-24T11:54:35.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-05-24T11:54:35.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-05-24T11:54:35.125 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.126 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.128 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.129 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.131 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.133 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.136 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.139 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.140 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.142 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.143 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.144 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.146 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.149 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.150 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-05-24T11:54:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-05-24T11:54:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-05-24T11:54:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.153 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.154 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.157 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.158 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.160 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.164 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.166 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.205 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.205 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.205 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.205 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.206 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-05-24T11:54:35.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-05-24T11:54:35.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-24T11:54:35.212 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.214 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.216 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.217 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.219 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.220 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.222 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.225 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.226 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.227 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-05-24T11:54:35.228 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-24T11:54:35.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-05-24T11:54:35.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.233 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.234 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.235 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.240 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.242 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.243 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.246 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.247 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.248 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.249 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.250 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.251 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.252 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.254 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.255 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.256 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.257 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.258 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.260 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.261 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.262 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.263 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.264 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.265 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-05-24T11:54:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.267 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.268 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.271 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.272 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.273 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.274 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.275 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-05-24T11:54:35.276 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-05-24T11:54:35.277 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.280 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.281 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.282 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.283 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.284 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.285 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.286 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-05-24T11:54:35.287 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.288 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.290 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.291 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.291 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.291 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.291 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.291 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.291 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.291 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.291 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.293 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.295 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.296 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.297 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.299 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.300 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.301 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.302 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.303 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.304 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-05-24T11:54:35.305 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.307 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.308 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.309 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.311 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.312 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.313 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.315 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.316 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.317 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.318 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.320 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.322 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.324 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.325 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.326 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.327 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.328 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.329 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.330 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-05-24T11:54:35.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-05-24T11:54:35.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.333 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.334 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.335 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.336 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.337 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.338 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.339 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.340 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.341 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.342 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.343 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.344 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.345 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.346 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.347 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-05-24T11:54:35.348 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.349 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.350 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.351 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.354 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.357 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.358 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 25073, 2024-05-24T11:54:35.359 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "y", 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540138322906710015, 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.360 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.361 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.364 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.365 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.366 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.367 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.368 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.369 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-05-24T11:54:35.370 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-05-24T11:54:35.371 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.372 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.373 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.374 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-05-24T11:54:35.375 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.376 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-05-24T11:54:35.377 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-05-24T11:54:35.378 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-05-24T11:54:35.379 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.380 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-24T11:54:35.381 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.382 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.383 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.384 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.385 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.386 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.388 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap" 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.389 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.390 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-05-24T11:54:35.391 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-05-24T11:54:35.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.394 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.395 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.396 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.397 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.398 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.399 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.400 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.401 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.403 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-24T11:54:35.405 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.406 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.407 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.408 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-05-24T11:54:35.410 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.411 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-05-24T11:54:35.412 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-05-24T11:54:35.413 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.414 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.418 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.421 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.422 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.424 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.425 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.426 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-05-24T11:54:35.427 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.428 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.429 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.430 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.431 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.432 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.433 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.434 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.435 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.436 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-24T11:54:35.438 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-05-24T11:54:35.439 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-05-24T11:54:35.440 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-05-24T11:54:35.441 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-05-24T11:54:35.443 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-24T11:54:35.444 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-24T11:54:35.445 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.446 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.447 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.448 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.449 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.450 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.451 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.452 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.453 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.455 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.456 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.457 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.458 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.459 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.460 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.461 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.462 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.463 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.464 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.465 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.466 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.467 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.470 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.471 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.472 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.473 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.474 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.475 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.476 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.477 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.478 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.480 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-24T11:54:35.481 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-24T11:54:35.482 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-05-24T11:54:35.483 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-05-24T11:54:35.484 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-24T11:54:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.486 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.487 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.489 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.490 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.491 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.492 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.493 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.494 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.495 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.496 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-24T11:54:35.497 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-24T11:54:35.498 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-24T11:54:35.499 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.499 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.499 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.499 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.499 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.499 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.499 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.499 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.499 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.500 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.501 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.503 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.504 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.505 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-24T11:54:35.506 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-05-24T11:54:35.507 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-05-24T11:54:35.508 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.509 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.510 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.511 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.512 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.513 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.514 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.514 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.514 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.514 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.514 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.514 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-05-24T11:54:35.514 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-05-24T11:54:35.514 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-05-24T11:54:35.515 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.516 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-05-24T11:54:35.517 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.519 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.520 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.520 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.520 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.520 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.520 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.520 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.520 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.520 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.521 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.522 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.523 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.524 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.524 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.524 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.524 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.524 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.524 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.524 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.524 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.524 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.525 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.526 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.526 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.526 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.526 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.526 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.526 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.526 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.526 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.527 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-05-24T11:54:35.527 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-05-24T11:54:35.527 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.527 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.527 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.527 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.527 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.528 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.529 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.530 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-24T11:54:35.531 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-05-24T11:54:35.532 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.533 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.534 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-24T11:54:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.537 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.538 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.539 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.541 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.541 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.541 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.541 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.541 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.541 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.542 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.543 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.544 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.545 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.546 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-05-24T11:54:35.547 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-24T11:54:35.548 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.549 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.550 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.551 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.552 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.553 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.554 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.555 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.556 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.556 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.556 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.558 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.559 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.560 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.560 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.560 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.560 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.561 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-24T11:54:35.562 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.563 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.564 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.565 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.566 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.567 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.568 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.569 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.571 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.572 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.573 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.574 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.575 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.576 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.577 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-05-24T11:54:35.578 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.579 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.580 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-05-24T11:54:35.581 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-05-24T11:54:35.582 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.583 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.584 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.585 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.586 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.587 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.589 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.590 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.591 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.592 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.595 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.596 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.597 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.598 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.599 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.600 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.602 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.602 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.602 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.602 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.602 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.602 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.602 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.602 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.602 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.603 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.604 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.606 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.608 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.609 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.610 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-24T11:54:35.611 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.612 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-05-24T11:54:35.613 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.614 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.615 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.616 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.617 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.618 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.619 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.620 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.621 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.622 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.623 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.624 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.625 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.626 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.629 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.630 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.633 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.635 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.636 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.637 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.638 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.639 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.640 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.640 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.640 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.640 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.640 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.640 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.641 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.642 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-05-24T11:54:35.643 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.644 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.645 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.646 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.647 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.649 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.650 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.651 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.652 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.653 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.654 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.655 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.656 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.657 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.658 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.659 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.660 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.661 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-24T11:54:35.662 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.663 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.665 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.666 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.666 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.666 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.666 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.666 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.666 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.666 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.666 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.667 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.668 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.668 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.668 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.668 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.668 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.668 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.668 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.669 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.670 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.671 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.672 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.673 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.674 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-24T11:54:35.676 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.677 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.678 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.679 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.680 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.681 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.682 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.683 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-05-24T11:54:35.684 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.685 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.686 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.687 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.688 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.688 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-05-24T11:54:35.688 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-05-24T11:54:35.688 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.688 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.688 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.688 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-05-24T11:54:35.688 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.689 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.690 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-24T11:54:35.691 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.692 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-05-24T11:54:35.693 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.693 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.693 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.693 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.693 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.693 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.693 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.693 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.693 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.694 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.695 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.696 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.697 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.698 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.699 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.700 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.701 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.702 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.704 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.705 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.706 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.707 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.708 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.709 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.710 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.712 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.713 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.714 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.715 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.717 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.718 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.719 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.720 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.721 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.722 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.723 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.724 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.725 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.726 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.728 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.729 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.730 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.731 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.732 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-05-24T11:54:35.733 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.734 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.735 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.737 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.738 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.739 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.740 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.741 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.742 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-05-24T11:54:35.743 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.745 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.746 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.746 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.746 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-05-24T11:54:35.746 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.746 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.746 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.746 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.747 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.748 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.749 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.753 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.754 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.755 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.756 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.757 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.758 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.759 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.760 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.761 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.762 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.763 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.765 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.767 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.768 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.769 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.770 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.771 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.772 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.773 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.774 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.775 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.776 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.777 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.778 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.779 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.780 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.781 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.782 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.783 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.784 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.785 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.788 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.789 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.790 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.791 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.806 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-05-24T11:54:35.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-05-24T11:54:35.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.810 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.811 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.812 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.813 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.818 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.819 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.820 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.821 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: "modules": [ 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard", 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: "iostat", 2024-05-24T11:54:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: "nfs", 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "restful", 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_schedule" 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.823 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.824 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.825 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.826 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.827 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.828 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.829 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.830 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-05-24T11:54:35.833 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-05-24T11:54:35.834 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.835 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.836 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.837 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.838 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-24T11:54:35.841 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:35.842 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-05-24T11:54:35.843 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-05-24T11:54:35.844 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-05-24T11:54:35.845 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.846 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.848 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.849 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.850 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.852 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-05-24T11:54:35.853 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap" 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-05-24T11:54:35.854 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:35.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.856 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.857 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.858 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-24T11:54:35.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.860 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.861 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.862 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.863 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.864 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.866 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.867 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-05-24T11:54:35.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-05-24T11:54:35.869 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-05-24T11:54:35.870 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-05-24T11:54:35.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-05-24T11:54:35.872 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-05-24T11:54:35.873 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.874 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.875 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.876 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.877 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-24T11:54:35.881 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-05-24T11:54:35.882 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.883 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.886 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.887 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.889 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.890 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-24T11:54:35.892 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-24T11:54:35.893 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.894 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.895 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.898 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-24T11:54:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-24T11:54:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-05-24T11:54:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-05-24T11:54:35.905 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.906 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.908 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.909 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.911 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.912 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.913 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.914 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.915 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.916 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.918 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.920 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.922 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.923 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.924 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.926 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.927 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.928 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.930 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-24T11:54:35.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.935 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-24T11:54:35.936 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-24T11:54:35.937 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.938 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.939 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-24T11:54:35.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-24T11:54:35.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.942 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.943 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-05-24T11:54:35.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.945 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.948 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.949 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.950 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.953 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.956 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.961 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.962 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.963 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.965 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.967 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.970 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.971 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.972 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-05-24T11:54:35.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.974 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.977 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.978 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.980 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.981 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.982 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-05-24T11:54:35.983 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.984 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.986 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.987 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-05-24T11:54:35.989 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-24T11:54:35.990 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.991 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-05-24T11:54:35.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:35.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.994 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.995 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.996 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:35.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.998 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-05-24T11:54:35.999 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-05-24T11:54:36.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-05-24T11:54:36.001 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-05-24T11:54:36.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:36.003 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-05-24T11:54:36.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.006 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.009 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.011 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-05-24T11:54:36.014 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-24T11:54:36.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.017 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.018 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.021 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.022 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-05-24T11:54:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-05-24T11:54:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.026 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.028 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:36.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-05-24T11:54:36.031 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.032 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.033 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-05-24T11:54:36.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-05-24T11:54:36.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-05-24T11:54:36.060 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.062 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.064 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.066 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.068 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-05-24T11:54:36.069 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.069 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.070 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.071 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.072 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.073 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.074 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.076 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.079 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.081 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.087 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.089 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.091 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.092 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.094 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.095 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.096 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.099 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.100 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.103 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.105 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.106 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.107 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.109 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.111 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-05-24T11:54:36.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-24T11:54:36.113 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.115 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.119 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-05-24T11:54:36.121 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-05-24T11:54:36.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-05-24T11:54:36.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.124 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.125 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.126 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.127 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.131 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.131 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.131 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-24T11:54:36.131 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-24T11:54:36.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.131 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-24T11:54:36.134 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-05-24T11:54:36.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-24T11:54:36.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.140 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.142 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.143 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-05-24T11:54:36.147 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.149 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.150 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.151 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-05-24T11:54:36.162 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.163 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.167 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.168 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.169 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.170 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.172 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.174 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.176 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-24T11:54:36.178 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.179 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-05-24T11:54:36.180 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-05-24T11:54:36.181 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-05-24T11:54:36.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-05-24T11:54:36.183 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-05-24T11:54:36.184 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-05-24T11:54:36.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.187 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.188 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.189 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.190 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.191 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.192 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.194 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-05-24T11:54:36.195 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.195 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.195 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.195 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.195 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.195 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.195 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.195 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.196 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.197 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.198 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.199 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.201 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.202 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.204 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.205 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-05-24T11:54:36.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:36.207 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.209 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.210 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.212 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.214 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.215 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.218 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.219 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.222 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-05-24T11:54:36.223 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.224 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.225 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.226 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.227 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.228 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.232 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.233 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.236 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-05-24T11:54:36.237 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.238 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.239 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.242 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.244 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.246 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.247 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.249 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.251 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.253 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-24T11:54:36.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-24T11:54:36.255 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.256 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.257 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-24T11:54:36.259 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-24T11:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-05-24T11:54:36.263 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-24T11:54:36.264 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "services": { 2024-05-24T11:54:36.265 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard": "https://172.21.15.39:8443/" 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "always_on_modules": { 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "octopus": [ 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-05-24T11:54:36.266 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "pacific": [ 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.267 INFO:teuthology.orchestra.run.smithi039.stdout: "quincy": [ 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-05-24T11:54:36.268 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-24T11:54:36.269 INFO:teuthology.orchestra.run.smithi039.stdout: "reef": [ 2024-05-24T11:54:36.269 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-05-24T11:54:36.269 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-05-24T11:54:36.269 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-05-24T11:54:36.269 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-05-24T11:54:36.269 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-05-24T11:54:36.269 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-05-24T11:54:36.269 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-05-24T11:54:36.269 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-05-24T11:54:36.270 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-05-24T11:54:36.270 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-05-24T11:54:36.270 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:36.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.270 INFO:teuthology.orchestra.run.smithi039.stdout: "last_failure_osd_epoch": 80, 2024-05-24T11:54:36.270 INFO:teuthology.orchestra.run.smithi039.stdout: "active_clients": [ 2024-05-24T11:54:36.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-05-24T11:54:36.270 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2579039544 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "libcephsqlite", 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1133136766 2024-05-24T11:54:36.271 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3489502349 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3034750634 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3837598099 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3550777397 2024-05-24T11:54:36.274 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.275 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:36.275 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:36.275 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-24T11:54:36.275 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:54:36.276 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:35 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1611099621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:54:36.276 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:35 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3033758279' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-24T11:54:36.276 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:35 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1611099621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:54:36.277 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:35 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3033758279' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-24T11:54:36.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7eff040a1310 msgr2=0x7eff040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:36.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7eff040a1310 0x7eff040a37d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff08007be0 tx=0x7eff080073d0 comp rx=0 tx=0).stop 2024-05-24T11:54:36.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1810b3f0 msgr2=0x7eff1819bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:36.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1810b3f0 0x7eff1819bb30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7eff10007ee0 tx=0x7eff1000d3b0 comp rx=0 tx=0).stop 2024-05-24T11:54:36.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 shutdown_connections 2024-05-24T11:54:36.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18102020 0x7eff181972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:36.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7eff040a1310 0x7eff040a37d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:36.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1810b3f0 0x7eff1819bb30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:36.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 --2- 172.21.15.39:0/3033758279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff18102940 0x7eff181977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:36.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.813+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 >> 172.21.15.39:0/3033758279 conn(0x7eff180fb7e0 msgr2=0x7eff180fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:36.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.814+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 shutdown_connections 2024-05-24T11:54:36.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:34.814+0000 7eff1ff99700 1 -- 172.21.15.39:0/3033758279 wait complete. 2024-05-24T11:54:36.283 DEBUG:tasks.check_counter:Getting stats from a 2024-05-24T11:54:36.284 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-24T11:54:36.284 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:35 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1611099621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:54:36.284 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:35 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3033758279' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-24T11:54:36.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:36 smithi053 ceph-mon[47448]: pgmap v521: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 13 MiB/s wr, 1.39k op/s 2024-05-24T11:54:36.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:36 smithi137 ceph-mon[47907]: pgmap v521: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 13 MiB/s wr, 1.39k op/s 2024-05-24T11:54:36.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:36 smithi039 ceph-mon[42919]: pgmap v521: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 13 MiB/s wr, 1.39k op/s 2024-05-24T11:54:36.689 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:54:38.485 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:54:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1410, 2024-05-24T11:54:38.485 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1384, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1925120, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 500746, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 5, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.153855077, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.047642013, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.063009746, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.041859018, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1383, 2024-05-24T11:54:38.486 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.170596927, 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000123352 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1568288, 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 170464 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 998, 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1216, 2024-05-24T11:54:38.487 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 245988, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 851088, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 8, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.098546823, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.031508896, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.020440643, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.037529799, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1213, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.133752155, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000110265 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:38.488 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 541264 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1072, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1072, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 284642, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 332597, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 5, 2024-05-24T11:54:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.077832909, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.023595009, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.021158675, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.025667783, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1071, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.095131977, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000088825 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.490 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 448 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4986, 2024-05-24T11:54:38.491 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.139315275, 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000027941 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-05-24T11:54:38.492 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-05-24T11:54:38.493 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 1, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-05-24T11:54:38.494 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2024-05-24T11:54:38.495 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:38.496 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-05-24T11:54:38.497 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:38.498 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 5, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-05-24T11:54:38.499 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 1, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 4196257, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 4195864, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 5 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-05-24T11:54:38.500 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 3, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 3, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 0, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 1, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 1, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 0, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 0, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 0, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 0, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 49620, 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 207156 2024-05-24T11:54:38.501 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-05-24T11:54:38.502 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.503 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.505 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-05-24T11:54:38.507 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-05-24T11:54:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.508 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.510 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:38.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.512 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.513 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 3, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 1, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 3781, 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:38.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:38.516 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 189433, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 52, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:38.517 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 184, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-24T11:54:38.518 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 8320, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 13, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 3126, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 3126, 2024-05-24T11:54:38.519 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3126, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2.222958850, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000711119 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3126, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3127 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 3126, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 3126, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-05-24T11:54:38.520 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 2055, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 1069, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:38.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-05-24T11:54:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-05-24T11:54:38.523 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 107, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 30, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 5, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 7, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 2, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-05-24T11:54:38.524 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 3476, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 2122341, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 3476, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3476, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 2122341, 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3126, 2024-05-24T11:54:38.527 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 1958, 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3126, 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1958, 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:38.528 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3126, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3126, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3126, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3126, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.529 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:38.530 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55e7f0b2e500": { 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:38.531 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-24T11:54:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-24T11:54:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:38.532 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:38.532 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:54:38.533 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:38 smithi137 ceph-mon[47907]: pgmap v522: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 8.3 MiB/s wr, 900 op/s 2024-05-24T11:54:38.533 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:38 smithi053 ceph-mon[47448]: pgmap v522: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 8.3 MiB/s wr, 900 op/s 2024-05-24T11:54:38.533 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:38 smithi039 ceph-mon[42919]: pgmap v522: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 8.3 MiB/s wr, 900 op/s 2024-05-24T11:54:38.852 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-24T11:54:38.853 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-24T11:54:38.853 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-24T11:54:38.853 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-24T11:54:38.853 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-24T11:54:38.853 DEBUG:tasks.check_counter:Getting stats from d 2024-05-24T11:54:38.853 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-24T11:54:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:54:40.116 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:54:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1412, 2024-05-24T11:54:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1389, 2024-05-24T11:54:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1911200, 2024-05-24T11:54:40.116 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 503169, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 5, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.163004232, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.056780583, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.063727182, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.041573702, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1388, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.129599408, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000093371 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:40.117 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1554032, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 172576 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 2070, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2289, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 530001, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 1184201, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-05-24T11:54:40.118 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.148289462, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.045697891, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.034928673, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.056101340, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2287, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.179729442, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000078587 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 542240 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 4, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 6, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1629, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 1256, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.002070689, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000056499, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.001628632, 2024-05-24T11:54:40.120 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000115027, 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4, 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001996434, 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000499108 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 5020, 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.129316057, 2024-05-24T11:54:40.122 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000025760 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-05-24T11:54:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 0, 2024-05-24T11:54:40.124 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 1, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:40.125 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-05-24T11:54:40.127 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-05-24T11:54:40.128 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-05-24T11:54:40.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 5, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 1, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-05-24T11:54:40.130 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 4196257, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 4196116, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 5 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 3, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 3, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 0, 2024-05-24T11:54:40.131 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 1, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 1, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 0, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 0, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 0, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 0, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 46720, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 207156 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-05-24T11:54:40.132 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:40.133 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.134 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.135 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.136 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-05-24T11:54:40.137 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.138 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.140 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:40.141 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.142 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.143 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 3, 2024-05-24T11:54:40.144 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 1, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 3, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 3142, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:40.145 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:40.146 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-24T11:54:40.147 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 198314, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 59, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:40.148 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 184, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 8320, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 13, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.149 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 3131, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 3131, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3131, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2.077961543, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000663673 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-05-24T11:54:40.150 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3131, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3132 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 3131, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 3131, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 2058, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 1071, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-05-24T11:54:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-05-24T11:54:40.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-05-24T11:54:40.153 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 107, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 30, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 6, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 8, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 2, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-05-24T11:54:40.154 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-05-24T11:54:40.155 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 3482, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 2108837, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 3482, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:40.156 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3482, 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 2108837, 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:40.157 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3131, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 1958, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3131, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1958, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:40.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3131, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3131, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3131, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3131, 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:40.159 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:40.160 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x5592c1b90500": { 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:40.161 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:40.162 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:54:40.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:40 smithi039 ceph-mon[42919]: pgmap v523: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 8.3 MiB/s wr, 904 op/s 2024-05-24T11:54:40.486 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-05-24T11:54:40.486 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-05-24T11:54:40.486 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-05-24T11:54:40.486 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-05-24T11:54:40.486 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-05-24T11:54:40.487 DEBUG:tasks.check_counter:Getting stats from g 2024-05-24T11:54:40.487 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-24T11:54:40.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:40 smithi053 ceph-mon[47448]: pgmap v523: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 8.3 MiB/s wr, 904 op/s 2024-05-24T11:54:40.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:40 smithi137 ceph-mon[47907]: pgmap v523: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 8.3 MiB/s wr, 904 op/s 2024-05-24T11:54:40.727 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:54:41.674 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:54:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 702, 2024-05-24T11:54:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 770, 2024-05-24T11:54:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 220637, 2024-05-24T11:54:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 502393, 2024-05-24T11:54:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 9, 2024-05-24T11:54:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-05-24T11:54:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.081304786, 2024-05-24T11:54:41.675 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.030269187, 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.047758073, 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.006542763, 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 768, 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.089206211, 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000116153 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1424, 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000398064, 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000279 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.676 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2208, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 9152 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 30, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 78, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 5964, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 169587, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 17, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.010900578, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.002486367, 2024-05-24T11:54:41.677 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.006910977, 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000229689, 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 75, 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.017346011, 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000231280 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 62, 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000017471, 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000281 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.678 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 113168 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 399, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 570, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1576449, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 614716, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.064591094, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.026872730, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.038227555, 2024-05-24T11:54:41.679 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.006899404, 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 570, 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.121392737, 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000212969 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1572368, 2024-05-24T11:54:41.680 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 607456 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 48, 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001105689, 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000023035 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:41.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2, 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000010489, 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000005244 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-05-24T11:54:41.682 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 11, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 10, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-05-24T11:54:41.683 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 10, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 10, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 1, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:41.684 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 1039, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 1804859590, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 6, 2024-05-24T11:54:41.685 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:41.686 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 1, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-05-24T11:54:41.687 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:41.688 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 5, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 5, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-05-24T11:54:41.689 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 1, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 1, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 4196257, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 0, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 5, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.008692877, 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001738575 2024-05-24T11:54:41.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 0 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 13, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 13, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 0, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 11, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 12, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 1, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 10, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 10, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 0, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-05-24T11:54:41.691 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 51104, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 198964 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 28, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.692 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.693 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-05-24T11:54:41.694 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.695 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.697 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.698 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.700 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.701 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.702 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:41.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 3, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 1, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 966, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:41.704 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:41.705 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:41.706 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 246151, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 79, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:41.707 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 184, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:41.708 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 50040, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 198, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-05-24T11:54:41.709 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 42, 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 12345, 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 33, 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 42, 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 42, 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.241995471, 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.005761796 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 42, 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 64 2024-05-24T11:54:41.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 42, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 0, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 42, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 0, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 10, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 0, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 5, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 8, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-05-24T11:54:41.711 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 6, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 20, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:41.712 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-05-24T11:54:41.713 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 107, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 30, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 9, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 13, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 4, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 14, 2024-05-24T11:54:41.714 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 0, 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 1 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 1, 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:41.715 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 1127, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1678955, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 1127, 2024-05-24T11:54:41.716 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 1127, 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1678955, 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-24T11:54:41.717 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 42, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 12345, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 42, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 12345, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:41.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 42, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 42, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 42, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 42, 2024-05-24T11:54:41.719 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:41.720 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x5652a09520a0": { 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-24T11:54:41.721 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 5, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1953, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 5, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 5, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1953, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:41.722 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:41.723 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:54:42.034 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-24T11:54:42.034 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-24T11:54:42.034 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-05-24T11:54:42.035 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-24T11:54:42.035 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-24T11:54:42.035 DEBUG:tasks.check_counter:Getting stats from j 2024-05-24T11:54:42.035 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-24T11:54:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.a/config 2024-05-24T11:54:42.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:42 smithi053 ceph-mon[47448]: pgmap v524: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-05-24T11:54:42.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:42 smithi137 ceph-mon[47907]: pgmap v524: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-05-24T11:54:42.534 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:42 smithi039 ceph-mon[42919]: pgmap v524: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 5.0 MiB/s wr, 536 op/s 2024-05-24T11:54:43.241 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 0, 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2, 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 0, 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 0, 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.001431617, 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000000000, 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.000255253, 2024-05-24T11:54:43.242 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.243 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 3, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 225, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 480, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 268224, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.015404520, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.010606207, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.002417022, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 222, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.032434578, 2024-05-24T11:54:43.244 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000146101 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 268224 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 307, 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 295, 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1449792, 2024-05-24T11:54:43.245 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 168096, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 2, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.048612668, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.015692667, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.033826041, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.006801243, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 295, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.034272547, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000116178 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:43.246 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1449792, 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 168096 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:43.248 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:43.249 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 148731, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 31, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:43.250 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 928, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:43.251 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 306, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1404096, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 306, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 306, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1404096, 2024-05-24T11:54:43.252 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-24T11:54:43.253 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-24T11:54:43.253 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-24T11:54:43.253 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-24T11:54:43.253 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:43.253 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-24T11:54:43.253 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-24T11:54:43.617 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-24T11:54:43.617 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-24T11:54:43.617 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-24T11:54:43.617 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-24T11:54:43.617 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-24T11:54:43.618 DEBUG:tasks.check_counter:Getting stats from b 2024-05-24T11:54:43.618 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-24T11:54:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:54:44.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:44 smithi053 ceph-mon[47448]: pgmap v525: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-05-24T11:54:44.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:44 smithi137 ceph-mon[47907]: pgmap v525: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-05-24T11:54:44.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:44 smithi039 ceph-mon[42919]: pgmap v525: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-05-24T11:54:44.928 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-24T11:54:44.928 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:44.928 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 5835, 2024-05-24T11:54:44.928 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 6156, 2024-05-24T11:54:44.928 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 3631462, 2024-05-24T11:54:44.928 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 116697193, 2024-05-24T11:54:44.928 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 11, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.653662506, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.263744782, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.181115317, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.182142854, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 6155, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 3.847307338, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000625070 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3141, 2024-05-24T11:54:44.929 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000846030, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000269 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1593888, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 620032 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 61277, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 60826, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 20729797, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 128130818, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 23, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-05-24T11:54:44.930 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 4.619108136, 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 1.713152231, 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 3.403911020, 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.253478925, 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 60823, 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 8.014094026, 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000131760 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 108530, 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.023417133, 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000215 2024-05-24T11:54:44.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 544016, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 272416 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 3952, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 4243, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1269594, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 66504062, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 147, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.469318247, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.167249978, 2024-05-24T11:54:44.932 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.159582063, 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.120510534, 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4241, 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.337746354, 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000315431 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2654, 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000715593, 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000269 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 95792, 2024-05-24T11:54:44.933 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 33376 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 10358, 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 15.394581414, 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001486250 2024-05-24T11:54:44.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1019, 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.045389579, 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000044543 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 13570, 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 13570, 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-24T11:54:44.935 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 13570, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 21.666331069, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001596634 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 1596, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 8, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 2060, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 617, 2024-05-24T11:54:44.936 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 1443, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 1178, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 1065, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 1065, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 2, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 23901, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 21783, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 51, 2024-05-24T11:54:44.937 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 1037, 2024-05-24T11:54:44.938 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 1804802542, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 6, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 2537, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 3717, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:44.939 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 40938, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 34496, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 79, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 3473 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 7, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-24T11:54:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 656, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 268, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 388, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 8, 2024-05-24T11:54:44.941 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 44422, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 44106, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 44106, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 316, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 243, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 139, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 137, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 137, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 2, 2024-05-24T11:54:44.942 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 1, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 276184116, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 278804240, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5161, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 25.398819103, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.004921298 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 1727, 2024-05-24T11:54:44.943 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 1995, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 268, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 103, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 111, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 8, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 2060, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 3053, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 993, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 1065, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 1346, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 281, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 79036, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-24T11:54:44.944 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 15873, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 3, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 13570, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 660, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 6, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1305, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 5.697879057, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.004366190 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 682, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.199830223, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000293006 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.945 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 388, 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.868827701, 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002239246 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7430, 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 7.029134243, 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000946047 2024-05-24T11:54:44.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 16, 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.003076373, 2024-05-24T11:54:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000192273 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 35, 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.042635390, 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001218154 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 16, 2024-05-24T11:54:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.360028128, 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.022501758 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2395, 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.974283231, 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000824335 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 228, 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.405662665, 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001779222 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-24T11:54:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 632, 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4.154073623, 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.006572901 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 10, 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.104339451, 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.010433945 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002403680, 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002403680 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 6, 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.017057794, 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002842965 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-24T11:54:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 426, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.807099510, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001894599 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 5, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 3, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 1718, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 859, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 970, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-24T11:54:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 844991, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 259, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-24T11:54:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 184, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 5915251, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 85298, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-24T11:54:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 10210, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 282437974, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 37, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 10210, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 10210, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 108.801782359, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.010656393 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 10210, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 16963 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 10210, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 134, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 10076, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1485, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 1548, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 134, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 5295, 2024-05-24T11:54:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 372, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 5, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 927, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 3096, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-24T11:54:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 107, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 31, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-05-24T11:54:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 3696, 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 405 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 1 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 19, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 6, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 268, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 71060, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 19145538, 2024-05-24T11:54:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 71060, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 71060, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 19145538, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-24T11:54:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 10210, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 282465930, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 10210, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 282465930, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 10210, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 10210, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 10210, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 10210, 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 268, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 27956, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 268, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 134, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 27956, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55f5235f40a0": { 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 44422, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 274609936, 2024-05-24T11:54:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 44422, 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 5161, 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 274609936, 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:44.973 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-24T11:54:45.373 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-24T11:54:45.373 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-24T11:54:45.373 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-24T11:54:45.373 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-05-24T11:54:45.373 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-05-24T11:54:45.373 DEBUG:tasks.check_counter:Getting stats from e 2024-05-24T11:54:45.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-24T11:54:45.631 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:54:46.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:46 smithi053 ceph-mon[47448]: pgmap v526: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-05-24T11:54:46.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:46 smithi137 ceph-mon[47907]: pgmap v526: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-05-24T11:54:46.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:46 smithi039 ceph-mon[42919]: pgmap v526: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-05-24T11:54:46.768 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-24T11:54:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 3154, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 3154, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 894055, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 980105, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.229932345, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.066371483, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.059372678, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.085268827, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3153, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.108934968, 2024-05-24T11:54:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000034549 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 5, 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 226, 2024-05-24T11:54:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1036, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 629548, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 7, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.014401611, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.008672462, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.003107567, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000070405, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 223, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.024420761, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000109510 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 628864 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 355, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 329, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1570812, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 174071, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-05-24T11:54:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.056508368, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.016914232, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.042358276, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.007030475, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 328, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.018455303, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000056266 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1570544, 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 173760 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5281, 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.131500496, 2024-05-24T11:54:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000024900 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-24T11:54:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-24T11:54:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 1, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-05-24T11:54:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2024-05-24T11:54:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-24T11:54:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-24T11:54:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 6, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-24T11:54:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 1, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 4196398, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 4196257, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 6 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 3, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 3, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 0, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 1, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 1, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 0, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 0, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 0, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 0, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 47920, 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2024-05-24T11:54:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-24T11:54:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 3, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 2, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 6928, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 184159, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 51, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-24T11:54:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 184, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 8320, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 13, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-24T11:54:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-24T11:54:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-24T11:54:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-24T11:54:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-24T11:54:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 3153, 2024-05-24T11:54:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2024-05-24T11:54:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2024-05-24T11:54:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 3153, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3153, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.718965440, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000545184 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3153, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 3154 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 3153, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 3153, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-05-24T11:54:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 2073, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 1078, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-24T11:54:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-24T11:54:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 107, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 30, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 4, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 5, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 1, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-05-24T11:54:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 3510, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2129092, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 3510, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3510, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2129092, 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-24T11:54:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 3153, 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 2099, 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3153, 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2099, 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 3153, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 3153, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3153, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 3153, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x5639361c0500": { 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-24T11:54:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:46.815 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-24T11:54:47.148 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-24T11:54:47.148 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-24T11:54:47.148 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-05-24T11:54:47.149 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-24T11:54:47.149 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-24T11:54:47.149 DEBUG:tasks.check_counter:Getting stats from h 2024-05-24T11:54:47.149 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-24T11:54:47.406 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1042, 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 858, 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 305915, 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 514596, 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 8, 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 2, 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.106689260, 2024-05-24T11:54:48.361 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.044655235, 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.062772223, 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.005984746, 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 858, 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.072003012, 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000083919 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1820, 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000420842, 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000231 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.362 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 407, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 404, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1587906, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 198613, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.063832484, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.018144829, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.049393647, 2024-05-24T11:54:48.363 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.007413899, 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 403, 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.040438891, 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000100344 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 50, 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000014221, 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000284 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1583824, 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 186160 2024-05-24T11:54:48.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 31, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 304, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 7890, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 687019, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 16, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.020464721, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.009531334, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.007789645, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000185372, 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:48.365 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 300, 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.037972713, 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000126575 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 75, 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000020427, 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000272 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 630880 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-24T11:54:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 44, 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.001666254, 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000037869 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-05-24T11:54:48.367 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000014095, 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000007047 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 1, 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 1, 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002571372, 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002571372 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-05-24T11:54:48.368 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 11, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 11, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 0, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 11, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 11, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 1, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 1, 2024-05-24T11:54:48.369 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 2, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 2, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 1, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 1, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-05-24T11:54:48.370 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 1037, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 1804802542, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 6, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:48.371 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 1, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 1, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-05-24T11:54:48.372 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 2, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:48.373 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 3, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 3, 2024-05-24T11:54:48.374 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 1, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 1, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 4195723, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.004546874, 2024-05-24T11:54:48.375 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001515624 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 14, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 14, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 0, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 12, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 13, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 1, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 11, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 11, 2024-05-24T11:54:48.376 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 0, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 1, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 1, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 49880, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 198964 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 4, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 1, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 25, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 1, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.377 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002571372, 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002571372 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.378 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.380 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.381 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.382 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.383 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-05-24T11:54:48.384 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.385 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-05-24T11:54:48.387 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.388 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 2, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 0, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 1, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 1, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 973, 2024-05-24T11:54:48.389 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:48.390 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:48.391 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 238982, 2024-05-24T11:54:48.392 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 76, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 184, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:48.393 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 54560, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 216, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 40, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 11815, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 27, 2024-05-24T11:54:48.394 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 40, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 40, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.218995467, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.005474886 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 40, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 63 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 40, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 0, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 40, 2024-05-24T11:54:48.395 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 10, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 3, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 7, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 6, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:48.396 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 20, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-05-24T11:54:48.397 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-05-24T11:54:48.398 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 107, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 30, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 8, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 10, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 2, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 16, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 0, 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 1 2024-05-24T11:54:48.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-05-24T11:54:48.400 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 1476, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1746283, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 1476, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:48.401 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1476, 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1746283, 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-24T11:54:48.402 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 40, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 11815, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 40, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 11815, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 40, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 40, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 40, 2024-05-24T11:54:48.404 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 40, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:48.405 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55a437ba60a0": { 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-05-24T11:54:48.406 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 3, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1419, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 3, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1419, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:48.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:48.408 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-24T11:54:48.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:48 smithi053 ceph-mon[47448]: pgmap v527: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-24T11:54:48.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:48 smithi137 ceph-mon[47907]: pgmap v527: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-24T11:54:48.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:48 smithi039 ceph-mon[42919]: pgmap v527: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-24T11:54:48.739 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-24T11:54:48.739 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-24T11:54:48.740 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-24T11:54:48.740 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-24T11:54:48.740 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-24T11:54:48.740 DEBUG:tasks.check_counter:Getting stats from k 2024-05-24T11:54:48.740 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-24T11:54:48.994 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.b/config 2024-05-24T11:54:50.027 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-05-24T11:54:50.027 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:50.027 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 0, 2024-05-24T11:54:50.028 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 2, 2024-05-24T11:54:50.028 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 0, 2024-05-24T11:54:50.028 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 0, 2024-05-24T11:54:50.028 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2024-05-24T11:54:50.028 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2024-05-24T11:54:50.028 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.001828803, 2024-05-24T11:54:50.029 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000000000, 2024-05-24T11:54:50.029 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000638024, 2024-05-24T11:54:50.029 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-24T11:54:50.029 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:50.029 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:50.029 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:50.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:50.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.030 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:50.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:50.030 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:50.030 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:50.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.031 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-24T11:54:50.031 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-05-24T11:54:50.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.031 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:50.031 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 309, 2024-05-24T11:54:50.032 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 297, 2024-05-24T11:54:50.032 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1450336, 2024-05-24T11:54:50.032 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 169280, 2024-05-24T11:54:50.032 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2024-05-24T11:54:50.032 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2024-05-24T11:54:50.032 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.050490887, 2024-05-24T11:54:50.033 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.012747907, 2024-05-24T11:54:50.033 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.037605363, 2024-05-24T11:54:50.033 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.007377819, 2024-05-24T11:54:50.033 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:50.033 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 297, 2024-05-24T11:54:50.034 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.030443939, 2024-05-24T11:54:50.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000102504 2024-05-24T11:54:50.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.034 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:50.034 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:50.034 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:50.035 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:50.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.035 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1450336, 2024-05-24T11:54:50.035 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 169280 2024-05-24T11:54:50.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.036 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:50.036 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 3, 2024-05-24T11:54:50.036 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 226, 2024-05-24T11:54:50.036 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 480, 2024-05-24T11:54:50.036 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 268400, 2024-05-24T11:54:50.036 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-05-24T11:54:50.037 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-05-24T11:54:50.037 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.013734083, 2024-05-24T11:54:50.037 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.008658660, 2024-05-24T11:54:50.037 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.002338950, 2024-05-24T11:54:50.037 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-24T11:54:50.037 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:50.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 223, 2024-05-24T11:54:50.038 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.023531489, 2024-05-24T11:54:50.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000105522 2024-05-24T11:54:50.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.038 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:50.038 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:50.039 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:50.039 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:50.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.039 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-24T11:54:50.039 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 268400 2024-05-24T11:54:50.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.040 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-05-24T11:54:50.040 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2024-05-24T11:54:50.040 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-05-24T11:54:50.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.040 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-05-24T11:54:50.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:50.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:50.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:50.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:50.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:50.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:50.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:50.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:50.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:50.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:50.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:50.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:50.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:50.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:50.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:50.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:50.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:50.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:50.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:50.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:50.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:50.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:50.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:50.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:50.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:50.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:50.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:50.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-05-24T11:54:50.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:50.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:50.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:50.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:50.047 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 149745, 2024-05-24T11:54:50.047 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 31, 2024-05-24T11:54:50.047 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:50.047 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:50.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-05-24T11:54:50.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-05-24T11:54:50.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:50.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:50.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:50.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:50.049 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2024-05-24T11:54:50.049 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2024-05-24T11:54:50.049 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:50.049 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:50.049 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:50.049 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-05-24T11:54:50.050 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2024-05-24T11:54:50.050 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2024-05-24T11:54:50.050 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:50.050 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-05-24T11:54:50.050 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:50.050 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-05-24T11:54:50.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-05-24T11:54:50.051 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:50.051 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-05-24T11:54:50.051 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-05-24T11:54:50.051 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-05-24T11:54:50.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 308, 2024-05-24T11:54:50.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1404356, 2024-05-24T11:54:50.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:50.052 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 308, 2024-05-24T11:54:50.052 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-05-24T11:54:50.052 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-05-24T11:54:50.053 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 308, 2024-05-24T11:54:50.053 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1404356, 2024-05-24T11:54:50.053 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-05-24T11:54:50.053 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-05-24T11:54:50.053 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-05-24T11:54:50.053 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-05-24T11:54:50.054 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:50.054 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-05-24T11:54:50.054 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-05-24T11:54:50.397 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-24T11:54:50.398 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-24T11:54:50.398 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-24T11:54:50.398 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-24T11:54:50.398 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-24T11:54:50.398 DEBUG:tasks.check_counter:Getting stats from c 2024-05-24T11:54:50.398 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-24T11:54:50.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:50 smithi053 ceph-mon[47448]: pgmap v528: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-05-24T11:54:50.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:50 smithi137 ceph-mon[47907]: pgmap v528: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-05-24T11:54:50.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:50 smithi039 ceph-mon[42919]: pgmap v528: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-05-24T11:54:50.648 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:54:51.591 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-05-24T11:54:51.591 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:51.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 417, 2024-05-24T11:54:51.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 418, 2024-05-24T11:54:51.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1585036, 2024-05-24T11:54:51.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 233994, 2024-05-24T11:54:51.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 13, 2024-05-24T11:54:51.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.057009441, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.016341016, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.042323713, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.006503255, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 415, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.031080300, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000074892 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 58, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000013593, 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000234 2024-05-24T11:54:51.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1578000, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 178672 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 712, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 824, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 227876, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 627664, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 10, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.072164323, 2024-05-24T11:54:51.593 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.026037970, 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.041375209, 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.005886344, 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 823, 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.052413954, 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000063686 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1468, 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000313538, 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000213 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.594 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3936, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 130640 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 25, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 225, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 8506, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 533539, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 9, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.012160601, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.006515504, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.003905100, 2024-05-24T11:54:51.595 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000372393, 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 224, 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.029017566, 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000129542 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 11, 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000002428, 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000220 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 5008, 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 521680 2024-05-24T11:54:51.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 48, 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001751262, 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000036484 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:51.597 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000014549, 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000007274 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 9, 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 4, 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4, 2024-05-24T11:54:51.598 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000862147, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000215536 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 5, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 11, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 10, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 10, 2024-05-24T11:54:51.599 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 10, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 1, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 1, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 1, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 12, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 8, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 1, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-05-24T11:54:51.600 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 1037, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 1804802542, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 6, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:51.601 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:51.602 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 2 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 1, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:51.603 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-05-24T11:54:51.604 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 5, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 5, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 1, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 1, 2024-05-24T11:54:51.605 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4196257, 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5, 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.011686625, 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002337325 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-05-24T11:54:51.606 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 13, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 13, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 0, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 11, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 12, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 1, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 10, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 10, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 0, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 1, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 2, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 1, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 49904, 2024-05-24T11:54:51.607 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 198964 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 9, 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 9, 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 28, 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.608 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4, 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000862147, 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000215536 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.609 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:51.610 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-05-24T11:54:51.611 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-05-24T11:54:51.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-05-24T11:54:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.617 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.618 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.620 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 3, 2024-05-24T11:54:51.621 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2024-05-24T11:54:51.622 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 2, 2024-05-24T11:54:51.622 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-05-24T11:54:51.622 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 3, 2024-05-24T11:54:51.622 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 1, 2024-05-24T11:54:51.622 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 976, 2024-05-24T11:54:51.622 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:51.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.622 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-05-24T11:54:51.622 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:51.625 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 240793, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 87, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 88, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 1, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:51.626 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 184, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 50416, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 201, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-05-24T11:54:51.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 41, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 12255, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 30, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 41, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 41, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.223996130, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.005463320 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.628 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 41, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 63 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 41, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 0, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 41, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 0, 2024-05-24T11:54:51.629 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 5, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 7, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 6, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 0, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-05-24T11:54:51.630 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-05-24T11:54:51.631 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 107, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-05-24T11:54:51.632 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 30, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 8, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 11, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 3, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 14, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 1 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-05-24T11:54:51.633 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:51.634 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 1150, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1694674, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 1150, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1150, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1694674, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.635 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 41, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 12255, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 41, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 12255, 2024-05-24T11:54:51.636 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:51.637 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 41, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 41, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 41, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 41, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-05-24T11:54:51.638 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.639 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x555e24d800a0": { 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 5, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1953, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 5, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 5, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1953, 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:51.640 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:51.641 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:51.641 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:51.641 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:51.641 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:51.641 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-05-24T11:54:51.945 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-24T11:54:51.946 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-24T11:54:51.946 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-24T11:54:51.946 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-24T11:54:51.946 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-24T11:54:51.946 DEBUG:tasks.check_counter:Getting stats from f 2024-05-24T11:54:51.946 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-24T11:54:52.168 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:54:52.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:52 smithi053 ceph-mon[47448]: pgmap v529: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-05-24T11:54:52.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:52 smithi137 ceph-mon[47907]: pgmap v529: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-05-24T11:54:52.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:52 smithi039 ceph-mon[42919]: pgmap v529: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-05-24T11:54:53.073 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-05-24T11:54:53.073 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:53.073 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 138, 2024-05-24T11:54:53.073 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 135, 2024-05-24T11:54:53.073 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 202048, 2024-05-24T11:54:53.073 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 47133, 2024-05-24T11:54:53.073 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.015678892, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.004316184, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.006377252, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.003783986, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 135, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.016145266, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000119594 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.074 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 166320, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 17584 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 125, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 146, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 9659423, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 157270, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.014290792, 2024-05-24T11:54:53.075 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.003405521, 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.007302296, 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.002686605, 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 144, 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.010778155, 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000074848 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.076 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 118640 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 127, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 129, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 12644806, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 39504, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.015348844, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.002345828, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.009619359, 2024-05-24T11:54:53.077 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.002636865, 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 127, 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.005453730, 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000042942 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-05-24T11:54:53.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 575, 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.015478866, 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000026919 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:53.079 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.080 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 418, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 156, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 262, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 200, 2024-05-24T11:54:53.081 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 2, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:53.082 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-05-24T11:54:53.083 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-05-24T11:54:53.083 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-05-24T11:54:53.083 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-05-24T11:54:53.083 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-05-24T11:54:53.083 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-05-24T11:54:53.083 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-05-24T11:54:53.083 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:53.083 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:53.083 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 1037, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 1804802542, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 6, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-05-24T11:54:53.084 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-05-24T11:54:53.085 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-05-24T11:54:53.086 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 2646, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-05-24T11:54:53.087 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 2, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 276184116, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 278804240, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 278796430, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.088 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 2646 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 224, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 252, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 28, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 58, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 59, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 1, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 418, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 418, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 0, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-05-24T11:54:53.089 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 49476, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 207132 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.091 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.092 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:53.093 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.095 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:53.096 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.097 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.099 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.100 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-05-24T11:54:53.101 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 0, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 2501, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:53.102 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:53.103 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:53.104 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 196519, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 49, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-05-24T11:54:53.105 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 184, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 1177364, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 46008, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:53.106 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-05-24T11:54:53.107 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:53.107 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-05-24T11:54:53.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.107 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-05-24T11:54:53.107 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-05-24T11:54:53.107 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-05-24T11:54:53.107 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 346, 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 26, 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 4, 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 346, 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 346, 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.284995205, 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000823685 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-05-24T11:54:53.108 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 346, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 347 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 346, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 346, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 188, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 155, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-05-24T11:54:53.109 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 2, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-05-24T11:54:53.110 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-05-24T11:54:53.111 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 107, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 2, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 6, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 6, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-05-24T11:54:53.112 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-05-24T11:54:53.113 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 386, 2024-05-24T11:54:53.114 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 22466645, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 386, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 386, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 22466645, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.115 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 346, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 22237390, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 346, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 22237390, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.116 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 346, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 346, 2024-05-24T11:54:53.117 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 346, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 346, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:53.118 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:53.119 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x56139eb9c5a0": { 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:53.120 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:53.121 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:53.121 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:53.121 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:53.121 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-05-24T11:54:53.471 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-24T11:54:53.471 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-24T11:54:53.471 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-24T11:54:53.471 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-24T11:54:53.471 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-24T11:54:53.471 DEBUG:tasks.check_counter:Getting stats from i 2024-05-24T11:54:53.471 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-24T11:54:53.691 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:54:54.483 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:54 smithi137 ceph-mon[47907]: pgmap v530: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-24T11:54:54.487 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:54 smithi053 ceph-mon[47448]: pgmap v530: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-24T11:54:54.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:54 smithi039 ceph-mon[42919]: pgmap v530: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 35, 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 41, 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 9159, 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 15957, 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 12, 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.007195955, 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000666829, 2024-05-24T11:54:54.682 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.004081588, 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000759545, 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 37, 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.019227619, 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000519665 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 23, 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000002905, 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000126 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 3600 2024-05-24T11:54:54.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1800, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1355, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1983468, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 716210, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 8, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.141258595, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.048920266, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.100420514, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.012580377, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1356, 2024-05-24T11:54:54.684 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.080317944, 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000059231 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2313, 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000456976, 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000197 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1585600, 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 299728 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 37, 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 288, 2024-05-24T11:54:54.685 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 9702, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 576662, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 13, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.016723114, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.007868756, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.006349751, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000339069, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 286, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.020493040, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000071653 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.686 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 81, 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000025072, 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000309 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 512064 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-05-24T11:54:54.687 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 58, 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001754382, 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000030247 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000014295, 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000007147 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-05-24T11:54:54.688 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 6, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 5, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.004237514, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000847502 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 1, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 11, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-05-24T11:54:54.689 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 12, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 12, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 12, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 1, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 2, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 2, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 2, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 12, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 7, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 1, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 2, 2024-05-24T11:54:54.690 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 1037, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 1804802542, 2024-05-24T11:54:54.691 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 6, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 1, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:54.692 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 1, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 3, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:54.693 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-05-24T11:54:54.694 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 6, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 6, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 1, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 1, 2024-05-24T11:54:54.695 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4196398, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.036122034, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.006020339 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 15, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 15, 2024-05-24T11:54:54.696 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 0, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 13, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 14, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 1, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 12, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 13, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 1, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 2, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 4, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 2, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 48516, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 198964 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-05-24T11:54:54.697 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 13, 2024-05-24T11:54:54.698 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-05-24T11:54:54.698 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 6, 2024-05-24T11:54:54.698 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 20, 2024-05-24T11:54:54.698 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 2, 2024-05-24T11:54:54.698 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.003684974, 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001228324 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.699 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-05-24T11:54:54.700 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000552540, 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000276270 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-05-24T11:54:54.701 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.702 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.702 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.702 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:54.702 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.702 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.702 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.702 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.704 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.706 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-05-24T11:54:54.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.709 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.710 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.710 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-05-24T11:54:54.710 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.710 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.711 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.711 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:54.711 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:54.711 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:54.711 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.711 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 3, 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 2, 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 4, 2024-05-24T11:54:54.712 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 2, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1008, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:54.713 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:54.714 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:54.715 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 247927, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 84, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 184, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:54.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 59128, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 240, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 45, 2024-05-24T11:54:54.717 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 12735, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 27, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 45, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 45, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.263995462, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.005866565 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 45, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 70 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 45, 2024-05-24T11:54:54.718 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 0, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 45, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 0, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 6, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 7, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 6, 2024-05-24T11:54:54.719 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:54.720 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-05-24T11:54:54.721 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 107, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 30, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 9, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 3, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 20, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 1 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.722 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-05-24T11:54:54.723 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 1868, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1811530, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 1868, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1868, 2024-05-24T11:54:54.724 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1811530, 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-05-24T11:54:54.725 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 45, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 12735, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 45, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 12735, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:54.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 45, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 45, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 45, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 45, 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:54.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:54.728 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55829f22c0a0": { 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:54.729 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 6, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 2094, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 6, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 6, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2094, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:54.730 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-05-24T11:54:55.046 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-24T11:54:55.046 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-24T11:54:55.046 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-24T11:54:55.046 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-24T11:54:55.046 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-24T11:54:55.046 DEBUG:tasks.check_counter:Getting stats from l 2024-05-24T11:54:55.046 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b9b067bc6d30ec00e391f0fa43f50affe981ba2f shell --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-24T11:54:55.268 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/mon.c/config 2024-05-24T11:54:56.232 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-05-24T11:54:56.232 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-05-24T11:54:56.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1086, 2024-05-24T11:54:56.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1088, 2024-05-24T11:54:56.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 286542, 2024-05-24T11:54:56.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 337810, 2024-05-24T11:54:56.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 7, 2024-05-24T11:54:56.232 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.073437383, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.021938131, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.019742524, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.023580535, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1086, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.060291305, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000055516 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.233 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1019, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1242, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 252452, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 862696, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 7, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.083980323, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.025513605, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.018543729, 2024-05-24T11:54:56.234 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.031232712, 2024-05-24T11:54:56.235 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:56.235 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1240, 2024-05-24T11:54:56.235 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.070091242, 2024-05-24T11:54:56.235 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000056525 2024-05-24T11:54:56.235 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.235 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:56.235 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.235 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.235 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 547648 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1438, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1412, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1934512, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 512008, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 3, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.145593484, 2024-05-24T11:54:56.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.043135642, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.062601813, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.039223007, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1411, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.095918777, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000067979 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1571632, 2024-05-24T11:54:56.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 176128 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5236, 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.126763486, 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000024209 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:56.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-05-24T11:54:56.239 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 0, 2024-05-24T11:54:56.240 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-05-24T11:54:56.241 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 0, 2024-05-24T11:54:56.241 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-05-24T11:54:56.241 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-05-24T11:54:56.241 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-05-24T11:54:56.241 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 1, 2024-05-24T11:54:56.241 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-05-24T11:54:56.241 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-05-24T11:54:56.241 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-05-24T11:54:56.242 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-05-24T11:54:56.243 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-05-24T11:54:56.244 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-05-24T11:54:56.245 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 3, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-05-24T11:54:56.246 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-05-24T11:54:56.247 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 1, 2024-05-24T11:54:56.247 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-05-24T11:54:56.247 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-05-24T11:54:56.247 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-05-24T11:54:56.247 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4195723, 2024-05-24T11:54:56.247 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 4195330, 2024-05-24T11:54:56.247 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-05-24T11:54:56.247 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.247 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 3 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 3, 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 3, 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 0, 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 1, 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 1, 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 0, 2024-05-24T11:54:56.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 47300, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 207156 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-05-24T11:54:56.249 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-05-24T11:54:56.250 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.251 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.252 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-05-24T11:54:56.254 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-05-24T11:54:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.256 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.256 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.256 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-05-24T11:54:56.256 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.256 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.256 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.257 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.258 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.260 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.261 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-05-24T11:54:56.262 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-05-24T11:54:56.263 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2024-05-24T11:54:56.263 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 2, 2024-05-24T11:54:56.263 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-05-24T11:54:56.263 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-05-24T11:54:56.263 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-05-24T11:54:56.263 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 2506, 2024-05-24T11:54:56.263 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-05-24T11:54:56.264 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-05-24T11:54:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-05-24T11:54:56.266 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 190194, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 54, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8064, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 184, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-05-24T11:54:56.267 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 8320, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 13, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 3179, 2024-05-24T11:54:56.268 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 21, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 3, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 3179, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3179, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2.270961492, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000714363 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3179, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 3180 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 3179, 2024-05-24T11:54:56.269 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 3179, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 2091, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 1086, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-05-24T11:54:56.270 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-05-24T11:54:56.271 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-05-24T11:54:56.272 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 107, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 30, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 5, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 6, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 1, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.273 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-05-24T11:54:56.274 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 3539, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 2133920, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 3539, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3539, 2024-05-24T11:54:56.275 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2133920, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 3179, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 1424, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3179, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1424, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-05-24T11:54:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 3179, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 3179, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3179, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 3179, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55a29e4dc500": { 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-05-24T11:54:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-05-24T11:54:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-05-24T11:54:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-05-24T11:54:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-05-24T11:54:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-05-24T11:54:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-05-24T11:54:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-05-24T11:54:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-05-24T11:54:56.281 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-05-24T11:54:56.494 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:56 smithi137 ceph-mon[47907]: pgmap v531: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-05-24T11:54:56.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:56 smithi039 ceph-mon[42919]: pgmap v531: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-05-24T11:54:56.596 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-24T11:54:56.596 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-24T11:54:56.596 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-24T11:54:56.596 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-24T11:54:56.596 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-24T11:54:56.596 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-05-24T11:54:56.597 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-24T11:54:56.608 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-24T11:54:56.609 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:54:56.609 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-24T11:54:56.641 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:54:56.641 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-24T11:54:56.702 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:54:56.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:56 smithi053 ceph-mon[47448]: pgmap v531: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-05-24T11:54:56.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.876+0000 7f8d78415700 1 -- 172.21.15.39:0/3974100849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700fff80 msgr2=0x7f8d70100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:56.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.876+0000 7f8d78415700 1 --2- 172.21.15.39:0/3974100849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700fff80 0x7f8d70100360 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009990 tx=0x7f8d6c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.876+0000 7f8d78415700 1 -- 172.21.15.39:0/3974100849 shutdown_connections 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.876+0000 7f8d78415700 1 --2- 172.21.15.39:0/3974100849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d701050a0 0x7f8d7010cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.876+0000 7f8d78415700 1 --2- 172.21.15.39:0/3974100849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700fff80 0x7f8d70100360 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.876+0000 7f8d78415700 1 --2- 172.21.15.39:0/3974100849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d70100930 0x7f8d70104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.876+0000 7f8d78415700 1 -- 172.21.15.39:0/3974100849 >> 172.21.15.39:0/3974100849 conn(0x7f8d700fb840 msgr2=0x7f8d700fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.876+0000 7f8d78415700 1 -- 172.21.15.39:0/3974100849 shutdown_connections 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.876+0000 7f8d78415700 1 -- 172.21.15.39:0/3974100849 wait complete. 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d78415700 1 Processor -- start 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d78415700 1 -- start start 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d78415700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d700fff80 0x7f8d701972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d78415700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d701050a0 0x7f8d70197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d78415700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d7019bbc0 0x7f8d7019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d78415700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7019c640 con 0x7f8d701050a0 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d78415700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7019c7b0 con 0x7f8d700fff80 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d78415700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7019c970 con 0x7f8d7019bbc0 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d759b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d701050a0 0x7f8d70197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d759b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d701050a0 0x7f8d70197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33658/0 (socket says 172.21.15.39:33658) 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d759b0700 1 -- 172.21.15.39:0/236481465 learned_addr learned my addr 172.21.15.39:0/236481465 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d769b2700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d7019bbc0 0x7f8d7019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.877+0000 7f8d761b1700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d700fff80 0x7f8d701972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d759b0700 1 -- 172.21.15.39:0/236481465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d7019bbc0 msgr2=0x7f8d7019c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d759b0700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d7019bbc0 0x7f8d7019c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d759b0700 1 -- 172.21.15.39:0/236481465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d700fff80 msgr2=0x7f8d701972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:56.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d759b0700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d700fff80 0x7f8d701972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:56.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d759b0700 1 -- 172.21.15.39:0/236481465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c0095f0 con 0x7f8d701050a0 2024-05-24T11:54:56.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d759b0700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d701050a0 0x7f8d70197820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d950 tx=0x7f8d6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:56.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d637fe700 1 -- 172.21.15.39:0/236481465 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64004220 con 0x7f8d701050a0 2024-05-24T11:54:56.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d701a3ed0 con 0x7f8d701050a0 2024-05-24T11:54:56.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d637fe700 1 -- 172.21.15.39:0/236481465 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d64007620 con 0x7f8d701050a0 2024-05-24T11:54:56.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d637fe700 1 -- 172.21.15.39:0/236481465 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64005560 con 0x7f8d701050a0 2024-05-24T11:54:56.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.878+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d701a4350 con 0x7f8d701050a0 2024-05-24T11:54:56.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.879+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d70101450 con 0x7f8d701050a0 2024-05-24T11:54:56.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.880+0000 7f8d637fe700 1 -- 172.21.15.39:0/236481465 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8d64004750 con 0x7f8d701050a0 2024-05-24T11:54:56.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.881+0000 7f8d637fe700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d5c0a1290 0x7f8d5c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:56.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.881+0000 7f8d637fe700 1 -- 172.21.15.39:0/236481465 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8d640cca10 con 0x7f8d701050a0 2024-05-24T11:54:56.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.881+0000 7f8d761b1700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d5c0a1290 0x7f8d5c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:56.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.881+0000 7f8d761b1700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d5c0a1290 0x7f8d5c0a3750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00b5c0 tx=0x7f8d6c005fb0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:56.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:56.885+0000 7f8d637fe700 1 -- 172.21.15.39:0/236481465 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d64095e70 con 0x7f8d701050a0 2024-05-24T11:54:57.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.093+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d70067380 con 0x7f8d701050a0 2024-05-24T11:54:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.094+0000 7f8d637fe700 1 -- 172.21.15.39:0/236481465 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+22875 (secure 0 0 0) 0x7f8d64016090 con 0x7f8d701050a0 2024-05-24T11:54:57.099 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:57.099 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":107,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:54:19.813504+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/338214439":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/870992183":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/3248772673":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/3290494740":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/333059399":"2024-05-25T11:38:12.183530+0000","172.21.15.39:6801/2090515453":"2024-05-25T11:38:12.183530+0000","172.21.15.39:6800/2090515453":"2024-05-25T11:38:12.183530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d5c0a1290 msgr2=0x7f8d5c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d5c0a1290 0x7f8d5c0a3750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00b5c0 tx=0x7f8d6c005fb0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d701050a0 msgr2=0x7f8d70197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d701050a0 0x7f8d70197820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d950 tx=0x7f8d6400dc60 comp rx=0 tx=0).stop 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 shutdown_connections 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d7019bbc0 0x7f8d7019c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f8d5c0a1290 0x7f8d5c0a3750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d700fff80 0x7f8d701972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 --2- 172.21.15.39:0/236481465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d701050a0 0x7f8d70197820 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 >> 172.21.15.39:0/236481465 conn(0x7f8d700fb840 msgr2=0x7f8d700fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 shutdown_connections 2024-05-24T11:54:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.097+0000 7f8d78415700 1 -- 172.21.15.39:0/236481465 wait complete. 2024-05-24T11:54:57.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-24T11:36:46.509049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-24T11:36:47.866055+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:54:57.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-24T11:54:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 -- 172.21.15.39:0/1850139650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc068c90 msgr2=0x7f48fc105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 --2- 172.21.15.39:0/1850139650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc068c90 0x7f48fc105b60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0099c0 tx=0x7f48ec009cd0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 -- 172.21.15.39:0/1850139650 shutdown_connections 2024-05-24T11:54:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 --2- 172.21.15.39:0/1850139650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc106290 0x7f48fc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 --2- 172.21.15.39:0/1850139650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48fc0682e0 0x7f48fc0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 --2- 172.21.15.39:0/1850139650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc068c90 0x7f48fc105b60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 -- 172.21.15.39:0/1850139650 >> 172.21.15.39:0/1850139650 conn(0x7f48fc076f40 msgr2=0x7f48fc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 -- 172.21.15.39:0/1850139650 shutdown_connections 2024-05-24T11:54:57.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 -- 172.21.15.39:0/1850139650 wait complete. 2024-05-24T11:54:57.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 Processor -- start 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.243+0000 7f4903be4700 1 -- start start 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4903be4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48fc0682e0 0x7f48fc1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4903be4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc106290 0x7f48fc199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4903be4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc19ddb0 0x7f48fc19e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4903be4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc19e830 con 0x7f48fc19ddb0 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4903be4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc19e9a0 con 0x7f48fc0682e0 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4903be4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc19eb60 con 0x7f48fc106290 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4902181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc19ddb0 0x7f48fc19e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4902181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc19ddb0 0x7f48fc19e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33674/0 (socket says 172.21.15.39:33674) 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4902181700 1 -- 172.21.15.39:0/1499966528 learned_addr learned my addr 172.21.15.39:0/1499966528 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f490117f700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc106290 0x7f48fc199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4901980700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48fc0682e0 0x7f48fc1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4902181700 1 -- 172.21.15.39:0/1499966528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc106290 msgr2=0x7f48fc199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4902181700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc106290 0x7f48fc199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4902181700 1 -- 172.21.15.39:0/1499966528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48fc0682e0 msgr2=0x7f48fc1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4902181700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48fc0682e0 0x7f48fc1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4902181700 1 -- 172.21.15.39:0/1499966528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec0095f0 con 0x7f48fc19ddb0 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f4902181700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc19ddb0 0x7f48fc19e230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f48f400ef30 tx=0x7f48f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f48f2ffd700 1 -- 172.21.15.39:0/1499966528 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4018070 con 0x7f48fc19ddb0 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.244+0000 7f48f2ffd700 1 -- 172.21.15.39:0/1499966528 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48f400f040 con 0x7f48fc19ddb0 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.245+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc1a6060 con 0x7f48fc19ddb0 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.245+0000 7f48f2ffd700 1 -- 172.21.15.39:0/1499966528 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f400cbc0 con 0x7f48fc19ddb0 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.245+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc1a6540 con 0x7f48fc19ddb0 2024-05-24T11:54:57.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.246+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc102640 con 0x7f48fc19ddb0 2024-05-24T11:54:57.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.249+0000 7f48f2ffd700 1 -- 172.21.15.39:0/1499966528 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f48f400cd80 con 0x7f48fc19ddb0 2024-05-24T11:54:57.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.250+0000 7f48f2ffd700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48e80a1310 0x7f48e80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.250+0000 7f4901980700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48e80a1310 0x7f48e80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.250+0000 7f48f2ffd700 1 -- 172.21.15.39:0/1499966528 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f48f40cccc0 con 0x7f48fc19ddb0 2024-05-24T11:54:57.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.250+0000 7f4901980700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48e80a1310 0x7f48e80a37d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009c60 tx=0x7f48f8009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:57.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.253+0000 7f48f2ffd700 1 -- 172.21.15.39:0/1499966528 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48f4096150 con 0x7f48fc19ddb0 2024-05-24T11:54:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.458+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48fc19a8a0 con 0x7f48fc19ddb0 2024-05-24T11:54:57.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.458+0000 7f48f2ffd700 1 -- 172.21.15.39:0/1499966528 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48f4099770 con 0x7f48fc19ddb0 2024-05-24T11:54:57.462 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:54:57.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48e80a1310 msgr2=0x7f48e80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48e80a1310 0x7f48e80a37d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009c60 tx=0x7f48f8009260 comp rx=0 tx=0).stop 2024-05-24T11:54:57.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc19ddb0 msgr2=0x7f48fc19e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc19ddb0 0x7f48fc19e230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f48f400ef30 tx=0x7f48f400c3a0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 shutdown_connections 2024-05-24T11:54:57.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f48e80a1310 0x7f48e80a37d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48fc19ddb0 0x7f48fc19e230 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48fc0682e0 0x7f48fc1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 --2- 172.21.15.39:0/1499966528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc106290 0x7f48fc199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 >> 172.21.15.39:0/1499966528 conn(0x7f48fc076f40 msgr2=0x7f48fc10c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:57.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 shutdown_connections 2024-05-24T11:54:57.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.460+0000 7f4903be4700 1 -- 172.21.15.39:0/1499966528 wait complete. 2024-05-24T11:54:57.476 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:57 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/236481465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:54:57.477 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-24T11:54:57.493 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:57 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/236481465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:54:57.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.602+0000 7f7f77684700 1 -- 172.21.15.39:0/1853756509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069dd0 msgr2=0x7f7f70105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.602+0000 7f7f77684700 1 --2- 172.21.15.39:0/1853756509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069dd0 0x7f7f70105b70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009990 tx=0x7f7f6c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.602+0000 7f7f77684700 1 -- 172.21.15.39:0/1853756509 shutdown_connections 2024-05-24T11:54:57.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.602+0000 7f7f77684700 1 --2- 172.21.15.39:0/1853756509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f701062a0 0x7f7f7010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.602+0000 7f7f77684700 1 --2- 172.21.15.39:0/1853756509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70069420 0x7f7f70069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.602+0000 7f7f77684700 1 --2- 172.21.15.39:0/1853756509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069dd0 0x7f7f70105b70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.602+0000 7f7f77684700 1 -- 172.21.15.39:0/1853756509 >> 172.21.15.39:0/1853756509 conn(0x7f7f70076ba0 msgr2=0x7f7f70076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.602+0000 7f7f77684700 1 -- 172.21.15.39:0/1853756509 shutdown_connections 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.602+0000 7f7f77684700 1 -- 172.21.15.39:0/1853756509 wait complete. 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f77684700 1 Processor -- start 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f77684700 1 -- start start 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f77684700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069420 0x7f7f701001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f77684700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f701062a0 0x7f7f70100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f77684700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70100c40 0x7f7f701a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f77684700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f701a5ae0 con 0x7f7f70069420 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f77684700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f701a5c50 con 0x7f7f70100c40 2024-05-24T11:54:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f77684700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f701a5e10 con 0x7f7f701062a0 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f75420700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069420 0x7f7f701001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f75420700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069420 0x7f7f701001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40096/0 (socket says 172.21.15.39:40096) 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f75420700 1 -- 172.21.15.39:0/1733201065 learned_addr learned my addr 172.21.15.39:0/1733201065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.603+0000 7f7f75c21700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70100c40 0x7f7f701a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f74c1f700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f701062a0 0x7f7f70100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f75420700 1 -- 172.21.15.39:0/1733201065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f701062a0 msgr2=0x7f7f70100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f75420700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f701062a0 0x7f7f70100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f75420700 1 -- 172.21.15.39:0/1733201065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70100c40 msgr2=0x7f7f701a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f75420700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70100c40 0x7f7f701a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f75420700 1 -- 172.21.15.39:0/1733201065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0095f0 con 0x7f7f70069420 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f75420700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069420 0x7f7f701001c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d980 tx=0x7f7f6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f667fc700 1 -- 172.21.15.39:0/1733201065 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60004220 con 0x7f7f70069420 2024-05-24T11:54:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f667fc700 1 -- 172.21.15.39:0/1733201065 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f60007640 con 0x7f7f70069420 2024-05-24T11:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f701a60f0 con 0x7f7f70069420 2024-05-24T11:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f667fc700 1 -- 172.21.15.39:0/1733201065 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600055e0 con 0x7f7f70069420 2024-05-24T11:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.604+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f701a6600 con 0x7f7f70069420 2024-05-24T11:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.605+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f70109570 con 0x7f7f70069420 2024-05-24T11:54:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.606+0000 7f7f667fc700 1 -- 172.21.15.39:0/1733201065 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7f60004750 con 0x7f7f70069420 2024-05-24T11:54:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.607+0000 7f7f667fc700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.607+0000 7f7f74c1f700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.607+0000 7f7f667fc700 1 -- 172.21.15.39:0/1733201065 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7f600cd030 con 0x7f7f70069420 2024-05-24T11:54:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.607+0000 7f7f74c1f700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f701018f0 tx=0x7f7f6c005940 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:57.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.610+0000 7f7f667fc700 1 -- 172.21.15.39:0/1733201065 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7f60096430 con 0x7f7f70069420 2024-05-24T11:54:57.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:57 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/236481465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:54:57.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.819+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f70067460 con 0x7f7f70069420 2024-05-24T11:54:57.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.819+0000 7f7f667fc700 1 -- 172.21.15.39:0/1733201065 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f600160a0 con 0x7f7f70069420 2024-05-24T11:54:57.823 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-05-24T11:54:57.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7f5c0a1290 msgr2=0x7f7f5c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f701018f0 tx=0x7f7f6c005940 comp rx=0 tx=0).stop 2024-05-24T11:54:57.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069420 msgr2=0x7f7f701001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069420 0x7f7f701001c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d980 tx=0x7f7f6000dc90 comp rx=0 tx=0).stop 2024-05-24T11:54:57.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 shutdown_connections 2024-05-24T11:54:57.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7f5c0a1290 0x7f7f5c0a3750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f70100c40 0x7f7f701a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f70069420 0x7f7f701001c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 --2- 172.21.15.39:0/1733201065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f701062a0 0x7f7f70100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 >> 172.21.15.39:0/1733201065 conn(0x7f7f70076ba0 msgr2=0x7f7f70107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:57.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.821+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 shutdown_connections 2024-05-24T11:54:57.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.822+0000 7f7f77684700 1 -- 172.21.15.39:0/1733201065 wait complete. 2024-05-24T11:54:57.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-24T11:54:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.964+0000 7f1871c1b700 1 -- 172.21.15.39:0/3356390088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c1062a0 msgr2=0x7f186c10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.964+0000 7f1871c1b700 1 --2- 172.21.15.39:0/3356390088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c1062a0 0x7f186c10de20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f186400b0a0 tx=0x7f186400b3b0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.964+0000 7f1871c1b700 1 -- 172.21.15.39:0/3356390088 shutdown_connections 2024-05-24T11:54:57.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.964+0000 7f1871c1b700 1 --2- 172.21.15.39:0/3356390088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186c069420 0x7f186c069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.964+0000 7f1871c1b700 1 --2- 172.21.15.39:0/3356390088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c1062a0 0x7f186c10de20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.964+0000 7f1871c1b700 1 --2- 172.21.15.39:0/3356390088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186c069dd0 0x7f186c105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.964+0000 7f1871c1b700 1 -- 172.21.15.39:0/3356390088 >> 172.21.15.39:0/3356390088 conn(0x7f186c076ba0 msgr2=0x7f186c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:57.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.964+0000 7f1871c1b700 1 -- 172.21.15.39:0/3356390088 shutdown_connections 2024-05-24T11:54:57.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.964+0000 7f1871c1b700 1 -- 172.21.15.39:0/3356390088 wait complete. 2024-05-24T11:54:57.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1871c1b700 1 Processor -- start 2024-05-24T11:54:57.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1871c1b700 1 -- start start 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1871c1b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c069420 0x7f186c1001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1871c1b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186c1062a0 0x7f186c100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1871c1b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186c100c40 0x7f186c1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1871c1b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1a5bf0 con 0x7f186c069420 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1871c1b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1a5d60 con 0x7f186c1062a0 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1871c1b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1a5f20 con 0x7f186c100c40 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f186b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c069420 0x7f186c1001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f186b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c069420 0x7f186c1001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40126/0 (socket says 172.21.15.39:40126) 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f186b7fe700 1 -- 172.21.15.39:0/1264094689 learned_addr learned my addr 172.21.15.39:0/1264094689 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1862ffd700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186c1062a0 0x7f186c100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f186bfff700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186c100c40 0x7f186c1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f186b7fe700 1 -- 172.21.15.39:0/1264094689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186c100c40 msgr2=0x7f186c1a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f186b7fe700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186c100c40 0x7f186c1a5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f186b7fe700 1 -- 172.21.15.39:0/1264094689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186c1062a0 msgr2=0x7f186c100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f186b7fe700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186c1062a0 0x7f186c100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f186b7fe700 1 -- 172.21.15.39:0/1264094689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1864009d00 con 0x7f186c069420 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.965+0000 7f1862ffd700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186c1062a0 0x7f186c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.966+0000 7f186b7fe700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c069420 0x7f186c1001c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f185c00d980 tx=0x7f185c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.966+0000 7f18697fa700 1 -- 172.21.15.39:0/1264094689 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c009b40 con 0x7f186c069420 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.966+0000 7f18697fa700 1 -- 172.21.15.39:0/1264094689 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f185c00de60 con 0x7f186c069420 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.966+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c1a6200 con 0x7f186c069420 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.966+0000 7f18697fa700 1 -- 172.21.15.39:0/1264094689 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c0054e0 con 0x7f186c069420 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.966+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c1a66c0 con 0x7f186c069420 2024-05-24T11:54:57.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.967+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c109570 con 0x7f186c069420 2024-05-24T11:54:57.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.969+0000 7f18697fa700 1 -- 172.21.15.39:0/1264094689 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f185c004750 con 0x7f186c069420 2024-05-24T11:54:57.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.969+0000 7f18697fa700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f18580a1290 0x7f18580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:57.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.969+0000 7f1862ffd700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f18580a1290 0x7f18580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:57.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.969+0000 7f18697fa700 1 -- 172.21.15.39:0/1264094689 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f185c0cd070 con 0x7f186c069420 2024-05-24T11:54:57.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.970+0000 7f1862ffd700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f18580a1290 0x7f18580a3750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1854005e10 tx=0x7f1854005d60 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:57.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:57.973+0000 7f18697fa700 1 -- 172.21.15.39:0/1264094689 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f185c0964d0 con 0x7f186c069420 2024-05-24T11:54:58.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.178+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f186c1016a0 con 0x7f186c069420 2024-05-24T11:54:58.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.178+0000 7f18697fa700 1 -- 172.21.15.39:0/1264094689 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7f185c017070 con 0x7f186c069420 2024-05-24T11:54:58.182 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-05-24T11:54:58.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f18580a1290 msgr2=0x7f18580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f18580a1290 0x7f18580a3750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1854005e10 tx=0x7f1854005d60 comp rx=0 tx=0).stop 2024-05-24T11:54:58.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c069420 msgr2=0x7f186c1001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c069420 0x7f186c1001c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f185c00d980 tx=0x7f185c00dd40 comp rx=0 tx=0).stop 2024-05-24T11:54:58.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 shutdown_connections 2024-05-24T11:54:58.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f186c100c40 0x7f186c1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f18580a1290 0x7f18580a3750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f186c069420 0x7f186c1001c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 --2- 172.21.15.39:0/1264094689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186c1062a0 0x7f186c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 >> 172.21.15.39:0/1264094689 conn(0x7f186c076ba0 msgr2=0x7f186c0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:58.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 shutdown_connections 2024-05-24T11:54:58.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.180+0000 7f1871c1b700 1 -- 172.21.15.39:0/1264094689 wait complete. 2024-05-24T11:54:58.197 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-24T11:54:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.323+0000 7fd1c976e700 1 -- 172.21.15.39:0/1777552970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4100060 msgr2=0x7fd1c4100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.323+0000 7fd1c976e700 1 --2- 172.21.15.39:0/1777552970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4100060 0x7fd1c4100440 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009990 tx=0x7fd1b4009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.323+0000 7fd1c976e700 1 -- 172.21.15.39:0/1777552970 shutdown_connections 2024-05-24T11:54:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.323+0000 7fd1c976e700 1 --2- 172.21.15.39:0/1777552970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c4105180 0x7fd1c410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.323+0000 7fd1c976e700 1 --2- 172.21.15.39:0/1777552970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4100060 0x7fd1c4100440 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.323+0000 7fd1c976e700 1 --2- 172.21.15.39:0/1777552970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1c4100a10 0x7fd1c4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.323+0000 7fd1c976e700 1 -- 172.21.15.39:0/1777552970 >> 172.21.15.39:0/1777552970 conn(0x7fd1c4076f40 msgr2=0x7fd1c4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 -- 172.21.15.39:0/1777552970 shutdown_connections 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 -- 172.21.15.39:0/1777552970 wait complete. 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 Processor -- start 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 -- start start 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1c4100060 0x7fd1c4101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c4105180 0x7fd1c4102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4102570 0x7fd1c41a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c41a5aa0 con 0x7fd1c4102570 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c41a5c10 con 0x7fd1c4105180 2024-05-24T11:54:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.324+0000 7fd1c976e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c41a5dd0 con 0x7fd1c4100060 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4102570 0x7fd1c41a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4102570 0x7fd1c41a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40132/0 (socket says 172.21.15.39:40132) 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c37fe700 1 -- 172.21.15.39:0/2752018338 learned_addr learned my addr 172.21.15.39:0/2752018338 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c2ffd700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1c4100060 0x7fd1c4101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1bbfff700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c4105180 0x7fd1c4102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c37fe700 1 -- 172.21.15.39:0/2752018338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1c4100060 msgr2=0x7fd1c4101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c37fe700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1c4100060 0x7fd1c4101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c37fe700 1 -- 172.21.15.39:0/2752018338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c4105180 msgr2=0x7fd1c4102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c37fe700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c4105180 0x7fd1c4102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c37fe700 1 -- 172.21.15.39:0/2752018338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b40095f0 con 0x7fd1c4102570 2024-05-24T11:54:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c37fe700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4102570 0x7fd1c41a54d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00ef30 tx=0x7fd1bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:58.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c0ff9700 1 -- 172.21.15.39:0/2752018338 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc018070 con 0x7fd1c4102570 2024-05-24T11:54:58.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c0ff9700 1 -- 172.21.15.39:0/2752018338 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1bc00f040 con 0x7fd1c4102570 2024-05-24T11:54:58.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c41a6050 con 0x7fd1c4102570 2024-05-24T11:54:58.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c0ff9700 1 -- 172.21.15.39:0/2752018338 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00cc70 con 0x7fd1c4102570 2024-05-24T11:54:58.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.325+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c41a6560 con 0x7fd1c4102570 2024-05-24T11:54:58.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.326+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c4109400 con 0x7fd1c4102570 2024-05-24T11:54:58.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.327+0000 7fd1c0ff9700 1 -- 172.21.15.39:0/2752018338 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd1bc003a00 con 0x7fd1c4102570 2024-05-24T11:54:58.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.328+0000 7fd1c0ff9700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd1b00a1290 0x7fd1b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:58.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.328+0000 7fd1c2ffd700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd1b00a1290 0x7fd1b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:58.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.328+0000 7fd1c0ff9700 1 -- 172.21.15.39:0/2752018338 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd1bc0ccd90 con 0x7fd1c4102570 2024-05-24T11:54:58.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.328+0000 7fd1c2ffd700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd1b00a1290 0x7fd1b00a3750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b2e0 tx=0x7fd1b4005040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:58.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.331+0000 7fd1c0ff9700 1 -- 172.21.15.39:0/2752018338 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd1bc096280 con 0x7fd1c4102570 2024-05-24T11:54:58.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.556+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd1c4101530 con 0x7fd1c4102570 2024-05-24T11:54:58.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.556+0000 7fd1c0ff9700 1 -- 172.21.15.39:0/2752018338 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v107) v1 ==== 81+0+1001 (secure 0 0 0) 0x7fd1bc003cb0 con 0x7fd1c4102570 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2243717102 2024-05-25T11:31:20.894336+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3960296835 2024-05-25T11:31:41.814659+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3563042541 2024-05-25T11:31:41.814659+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2910856381 2024-05-25T11:31:20.894336+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1790202809 2024-05-25T11:31:20.894336+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1012810399 2024-05-25T11:31:20.894336+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3960296835 2024-05-25T11:31:41.814659+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1790202809 2024-05-25T11:31:20.894336+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1559144405 2024-05-25T11:31:41.814659+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3982984387 2024-05-25T11:31:41.814659+0000 2024-05-24T11:54:58.561 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/338214439 2024-05-25T11:38:12.183530+0000 2024-05-24T11:54:58.562 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/870992183 2024-05-25T11:38:12.183530+0000 2024-05-24T11:54:58.562 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3248772673 2024-05-25T11:38:12.183530+0000 2024-05-24T11:54:58.562 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3290494740 2024-05-25T11:38:12.183530+0000 2024-05-24T11:54:58.562 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/333059399 2024-05-25T11:38:12.183530+0000 2024-05-24T11:54:58.562 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/2090515453 2024-05-25T11:38:12.183530+0000 2024-05-24T11:54:58.562 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/2090515453 2024-05-25T11:38:12.183530+0000 2024-05-24T11:54:58.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd1b00a1290 msgr2=0x7fd1b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd1b00a1290 0x7fd1b00a3750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400b2e0 tx=0x7fd1b4005040 comp rx=0 tx=0).stop 2024-05-24T11:54:58.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4102570 msgr2=0x7fd1c41a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4102570 0x7fd1c41a54d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00ef30 tx=0x7fd1bc00c3a0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 shutdown_connections 2024-05-24T11:54:58.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1c4100060 0x7fd1c4101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fd1b00a1290 0x7fd1b00a3750 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1c4102570 0x7fd1c41a54d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 --2- 172.21.15.39:0/2752018338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c4105180 0x7fd1c4102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 >> 172.21.15.39:0/2752018338 conn(0x7fd1c4076f40 msgr2=0x7fd1c40699d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:58.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 shutdown_connections 2024-05-24T11:54:58.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.559+0000 7fd1c976e700 1 -- 172.21.15.39:0/2752018338 wait complete. 2024-05-24T11:54:58.564 INFO:teuthology.orchestra.run.smithi039.stderr:listed 17 entries 2024-05-24T11:54:58.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:58 smithi039 ceph-mon[42919]: pgmap v532: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8 op/s 2024-05-24T11:54:58.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:58 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1499966528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:54:58.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:58 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1733201065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:54:58.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:58 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1264094689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:54:58.575 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-24T11:54:58.576 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-24T11:54:58.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-24T11:54:58.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:58 smithi053 ceph-mon[47448]: pgmap v532: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8 op/s 2024-05-24T11:54:58.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:58 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1499966528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:54:58.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:58 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1733201065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:54:58.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:58 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1264094689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:54:58.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:58 smithi137 ceph-mon[47907]: pgmap v532: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8 op/s 2024-05-24T11:54:58.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:58 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1499966528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:54:58.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:58 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1733201065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:54:58.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:58 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1264094689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:54:58.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.758+0000 7f3834872700 1 -- 172.21.15.39:0/31904187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3830101fa0 msgr2=0x7f3830102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.758+0000 7f3834872700 1 --2- 172.21.15.39:0/31904187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3830101fa0 0x7f3830102380 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3820009990 tx=0x7f3820009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.758+0000 7f3834872700 1 -- 172.21.15.39:0/31904187 shutdown_connections 2024-05-24T11:54:58.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.758+0000 7f3834872700 1 --2- 172.21.15.39:0/31904187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f383010b3c0 0x7f383010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.758+0000 7f3834872700 1 --2- 172.21.15.39:0/31904187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3830101fa0 0x7f3830102380 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.758+0000 7f3834872700 1 --2- 172.21.15.39:0/31904187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38301028c0 0x7f383010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.758+0000 7f3834872700 1 -- 172.21.15.39:0/31904187 >> 172.21.15.39:0/31904187 conn(0x7f38300fb7e0 msgr2=0x7f38300fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:58.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.758+0000 7f3834872700 1 -- 172.21.15.39:0/31904187 shutdown_connections 2024-05-24T11:54:58.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.758+0000 7f3834872700 1 -- 172.21.15.39:0/31904187 wait complete. 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f3834872700 1 Processor -- start 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f3834872700 1 -- start start 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f3834872700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3830101fa0 0x7f38300ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f3834872700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38301028c0 0x7f3830100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f3834872700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383010b3c0 0x7f3830100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f3834872700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38301a5590 con 0x7f383010b3c0 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f3834872700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38301a5700 con 0x7f38301028c0 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f3834872700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38301a58c0 con 0x7f3830101fa0 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383010b3c0 0x7f3830100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382dd9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38301028c0 0x7f3830100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383010b3c0 0x7f3830100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40156/0 (socket says 172.21.15.39:40156) 2024-05-24T11:54:58.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382ed9d700 1 -- 172.21.15.39:0/4217386575 learned_addr learned my addr 172.21.15.39:0/4217386575 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382e59c700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3830101fa0 0x7f38300ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382ed9d700 1 -- 172.21.15.39:0/4217386575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3830101fa0 msgr2=0x7f38300ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382ed9d700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3830101fa0 0x7f38300ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382ed9d700 1 -- 172.21.15.39:0/4217386575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38301028c0 msgr2=0x7f3830100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382ed9d700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38301028c0 0x7f3830100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.759+0000 7f382ed9d700 1 -- 172.21.15.39:0/4217386575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38200095f0 con 0x7f383010b3c0 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.760+0000 7f382ed9d700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383010b3c0 0x7f3830100670 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3828007ee0 tx=0x7f382800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.760+0000 7f381f7fe700 1 -- 172.21.15.39:0/4217386575 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828017070 con 0x7f383010b3c0 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.760+0000 7f381f7fe700 1 -- 172.21.15.39:0/4217386575 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f382800f040 con 0x7f383010b3c0 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.760+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38301a5ba0 con 0x7f383010b3c0 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.760+0000 7f381f7fe700 1 -- 172.21.15.39:0/4217386575 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828005570 con 0x7f383010b3c0 2024-05-24T11:54:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.760+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38301a6060 con 0x7f383010b3c0 2024-05-24T11:54:58.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.761+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3830106450 con 0x7f383010b3c0 2024-05-24T11:54:58.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.762+0000 7f381f7fe700 1 -- 172.21.15.39:0/4217386575 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f382800db20 con 0x7f383010b3c0 2024-05-24T11:54:58.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.763+0000 7f381f7fe700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f38180a1290 0x7f38180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.763+0000 7f382e59c700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f38180a1290 0x7f38180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.763+0000 7f381f7fe700 1 -- 172.21.15.39:0/4217386575 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f38280cca60 con 0x7f383010b3c0 2024-05-24T11:54:58.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.763+0000 7f382e59c700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f38180a1290 0x7f38180a3750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3820006010 tx=0x7f3820005960 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.767+0000 7f381f7fe700 1 -- 172.21.15.39:0/4217386575 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3828095ec0 con 0x7f383010b3c0 2024-05-24T11:54:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.973+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38300688d0 con 0x7f383010b3c0 2024-05-24T11:54:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.975+0000 7f381f7fe700 1 -- 172.21.15.39:0/4217386575 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+22875 (secure 0 0 0) 0x7f38280b9030 con 0x7f383010b3c0 2024-05-24T11:54:58.979 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-24T11:54:58.979 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":107,"fsid":"f68008c4-19c0-11ef-bc9a-c7b262605968","created":"2024-05-24T11:30:55.204289+0000","modified":"2024-05-24T11:54:19.813504+0000","last_up_change":"2024-05-24T11:36:14.585548+0000","last_in_change":"2024-05-24T11:35:58.967618+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-24T11:33:34.333450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-24T11:36:46.509049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-24T11:36:47.866055+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"010b8819-8872-412c-b489-b1b1389d3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6803","nonce":618291952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6805","nonce":618291952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6809","nonce":618291952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":618291952},{"type":"v1","addr":"172.21.15.39:6807","nonce":618291952}]},"public_addr":"172.21.15.39:6803/618291952","cluster_addr":"172.21.15.39:6805/618291952","heartbeat_back_addr":"172.21.15.39:6809/618291952","heartbeat_front_addr":"172.21.15.39:6807/618291952","state":["exists","up"]},{"osd":1,"uuid":"9572166c-feb0-4545-b576-9909a66bcdee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6801","nonce":3467330431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6803","nonce":3467330431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6807","nonce":3467330431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3467330431},{"type":"v1","addr":"172.21.15.53:6805","nonce":3467330431}]},"public_addr":"172.21.15.53:6801/3467330431","cluster_addr":"172.21.15.53:6803/3467330431","heartbeat_back_addr":"172.21.15.53:6807/3467330431","heartbeat_front_addr":"172.21.15.53:6805/3467330431","state":["exists","up"]},{"osd":2,"uuid":"7e133e51-8deb-40e3-b5ee-813fd6dfb344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6801","nonce":3821366387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6803","nonce":3821366387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6807","nonce":3821366387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3821366387},{"type":"v1","addr":"172.21.15.137:6805","nonce":3821366387}]},"public_addr":"172.21.15.137:6801/3821366387","cluster_addr":"172.21.15.137:6803/3821366387","heartbeat_back_addr":"172.21.15.137:6807/3821366387","heartbeat_front_addr":"172.21.15.137:6805/3821366387","state":["exists","up"]},{"osd":3,"uuid":"29168261-54e3-4898-8f65-9b35e0e4a247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6811","nonce":445163923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6813","nonce":445163923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6817","nonce":445163923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":445163923},{"type":"v1","addr":"172.21.15.39:6815","nonce":445163923}]},"public_addr":"172.21.15.39:6811/445163923","cluster_addr":"172.21.15.39:6813/445163923","heartbeat_back_addr":"172.21.15.39:6817/445163923","heartbeat_front_addr":"172.21.15.39:6815/445163923","state":["exists","up"]},{"osd":4,"uuid":"cc68e143-c059-4fae-9d40-326fbd44dd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6809","nonce":2421884743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6811","nonce":2421884743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6815","nonce":2421884743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2421884743},{"type":"v1","addr":"172.21.15.53:6813","nonce":2421884743}]},"public_addr":"172.21.15.53:6809/2421884743","cluster_addr":"172.21.15.53:6811/2421884743","heartbeat_back_addr":"172.21.15.53:6815/2421884743","heartbeat_front_addr":"172.21.15.53:6813/2421884743","state":["exists","up"]},{"osd":5,"uuid":"d9f89542-51c6-4d02-a245-c6fb39f4d4e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6809","nonce":173299621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6811","nonce":173299621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6815","nonce":173299621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":173299621},{"type":"v1","addr":"172.21.15.137:6813","nonce":173299621}]},"public_addr":"172.21.15.137:6809/173299621","cluster_addr":"172.21.15.137:6811/173299621","heartbeat_back_addr":"172.21.15.137:6815/173299621","heartbeat_front_addr":"172.21.15.137:6813/173299621","state":["exists","up"]},{"osd":6,"uuid":"eb26433d-c0d3-4cbe-ba66-2b1a27944123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6819","nonce":2437170915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6821","nonce":2437170915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6825","nonce":2437170915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2437170915},{"type":"v1","addr":"172.21.15.39:6823","nonce":2437170915}]},"public_addr":"172.21.15.39:6819/2437170915","cluster_addr":"172.21.15.39:6821/2437170915","heartbeat_back_addr":"172.21.15.39:6825/2437170915","heartbeat_front_addr":"172.21.15.39:6823/2437170915","state":["exists","up"]},{"osd":7,"uuid":"ab3690ac-bcb8-4bee-93b1-d2c729919262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6817","nonce":1522585706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6819","nonce":1522585706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6823","nonce":1522585706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1522585706},{"type":"v1","addr":"172.21.15.53:6821","nonce":1522585706}]},"public_addr":"172.21.15.53:6817/1522585706","cluster_addr":"172.21.15.53:6819/1522585706","heartbeat_back_addr":"172.21.15.53:6823/1522585706","heartbeat_front_addr":"172.21.15.53:6821/1522585706","state":["exists","up"]},{"osd":8,"uuid":"7ef14bbb-dbe7-497f-b170-89772a0b6ec0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6817","nonce":753285325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6819","nonce":753285325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6823","nonce":753285325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":753285325},{"type":"v1","addr":"172.21.15.137:6821","nonce":753285325}]},"public_addr":"172.21.15.137:6817/753285325","cluster_addr":"172.21.15.137:6819/753285325","heartbeat_back_addr":"172.21.15.137:6823/753285325","heartbeat_front_addr":"172.21.15.137:6821/753285325","state":["exists","up"]},{"osd":9,"uuid":"32dc829d-7871-4c18-890f-82f263600944","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6827","nonce":1424013773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6829","nonce":1424013773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6833","nonce":1424013773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1424013773},{"type":"v1","addr":"172.21.15.39:6831","nonce":1424013773}]},"public_addr":"172.21.15.39:6827/1424013773","cluster_addr":"172.21.15.39:6829/1424013773","heartbeat_back_addr":"172.21.15.39:6833/1424013773","heartbeat_front_addr":"172.21.15.39:6831/1424013773","state":["exists","up"]},{"osd":10,"uuid":"f1b20b6f-c19f-4c58-aafa-feb7cc027de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6825","nonce":3257679839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6827","nonce":3257679839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6831","nonce":3257679839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":3257679839},{"type":"v1","addr":"172.21.15.53:6829","nonce":3257679839}]},"public_addr":"172.21.15.53:6825/3257679839","cluster_addr":"172.21.15.53:6827/3257679839","heartbeat_back_addr":"172.21.15.53:6831/3257679839","heartbeat_front_addr":"172.21.15.53:6829/3257679839","state":["exists","up"]},{"osd":11,"uuid":"78bc93a0-953b-478a-9d7e-25a4005fd6a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6825","nonce":2768517457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6827","nonce":2768517457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6831","nonce":2768517457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":2768517457},{"type":"v1","addr":"172.21.15.137:6829","nonce":2768517457}]},"public_addr":"172.21.15.137:6825/2768517457","cluster_addr":"172.21.15.137:6827/2768517457","heartbeat_back_addr":"172.21.15.137:6831/2768517457","heartbeat_front_addr":"172.21.15.137:6829/2768517457","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:32:55.308997+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:13.565102+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:30.794896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:33:49.881620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:07.707733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:24.550716+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:34:43.793429+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:01.867366+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:19.156160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:38.320398+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:35:56.393472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-24T11:36:13.330656+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/2243717102":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6801/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/3563042541":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/2910856381":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6801/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1012810399":"2024-05-25T11:31:20.894336+0000","172.21.15.39:6800/3960296835":"2024-05-25T11:31:41.814659+0000","172.21.15.39:6800/1790202809":"2024-05-25T11:31:20.894336+0000","172.21.15.39:0/1559144405":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/3982984387":"2024-05-25T11:31:41.814659+0000","172.21.15.39:0/338214439":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/870992183":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/3248772673":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/3290494740":"2024-05-25T11:38:12.183530+0000","172.21.15.39:0/333059399":"2024-05-25T11:38:12.183530+0000","172.21.15.39:6801/2090515453":"2024-05-25T11:38:12.183530+0000","172.21.15.39:6800/2090515453":"2024-05-25T11:38:12.183530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-24T11:54:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f38180a1290 msgr2=0x7f38180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f38180a1290 0x7f38180a3750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3820006010 tx=0x7f3820005960 comp rx=0 tx=0).stop 2024-05-24T11:54:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383010b3c0 msgr2=0x7f3830100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383010b3c0 0x7f3830100670 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3828007ee0 tx=0x7f382800d3b0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 shutdown_connections 2024-05-24T11:54:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3830101fa0 0x7f38300ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f38180a1290 0x7f38180a3750 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383010b3c0 0x7f3830100670 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 --2- 172.21.15.39:0/4217386575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38301028c0 0x7f3830100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 >> 172.21.15.39:0/4217386575 conn(0x7f38300fb7e0 msgr2=0x7f38300fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 shutdown_connections 2024-05-24T11:54:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:58.977+0000 7f3834872700 1 -- 172.21.15.39:0/4217386575 wait complete. 2024-05-24T11:54:58.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-24T11:33:34.333450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-24T11:36:46.509049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-24T11:36:47.866055+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-24T11:54:58.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-24T11:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.119+0000 7f8041373700 1 -- 172.21.15.39:0/3101524456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c100a10 msgr2=0x7f803c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.119+0000 7f8041373700 1 --2- 172.21.15.39:0/3101524456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c100a10 0x7f803c104a50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8024009990 tx=0x7f8024009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.119+0000 7f8041373700 1 -- 172.21.15.39:0/3101524456 shutdown_connections 2024-05-24T11:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.119+0000 7f8041373700 1 --2- 172.21.15.39:0/3101524456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f803c105180 0x7f803c10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.119+0000 7f8041373700 1 --2- 172.21.15.39:0/3101524456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f803c100060 0x7f803c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.119+0000 7f8041373700 1 --2- 172.21.15.39:0/3101524456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c100a10 0x7f803c104a50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.119+0000 7f8041373700 1 -- 172.21.15.39:0/3101524456 >> 172.21.15.39:0/3101524456 conn(0x7f803c076fa0 msgr2=0x7f803c0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.119+0000 7f8041373700 1 -- 172.21.15.39:0/3101524456 shutdown_connections 2024-05-24T11:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.119+0000 7f8041373700 1 -- 172.21.15.39:0/3101524456 wait complete. 2024-05-24T11:54:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.120+0000 7f8041373700 1 Processor -- start 2024-05-24T11:54:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.120+0000 7f8041373700 1 -- start start 2024-05-24T11:54:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f8041373700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f803c100060 0x7f803c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f8041373700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c105180 0x7f803c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f8041373700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f803c198be0 0x7f803c1a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f8041373700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1a38c0 con 0x7f803c105180 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f8041373700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1a3a30 con 0x7f803c198be0 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f8041373700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1a3bf0 con 0x7f803c100060 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c105180 0x7f803c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c105180 0x7f803c1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40172/0 (socket says 172.21.15.39:40172) 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803a7fc700 1 -- 172.21.15.39:0/774048696 learned_addr learned my addr 172.21.15.39:0/774048696 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803affd700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f803c100060 0x7f803c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803b7fe700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f803c198be0 0x7f803c1a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803a7fc700 1 -- 172.21.15.39:0/774048696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f803c100060 msgr2=0x7f803c198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803a7fc700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f803c100060 0x7f803c198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803a7fc700 1 -- 172.21.15.39:0/774048696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f803c198be0 msgr2=0x7f803c1a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803a7fc700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f803c198be0 0x7f803c1a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803a7fc700 1 -- 172.21.15.39:0/774048696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80240095f0 con 0x7f803c105180 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803b7fe700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f803c198be0 0x7f803c1a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.121+0000 7f803a7fc700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c105180 0x7f803c1986a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f80240036c0 tx=0x7f8024009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.122+0000 7f8033fff700 1 -- 172.21.15.39:0/774048696 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802401d070 con 0x7f803c105180 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.122+0000 7f8033fff700 1 -- 172.21.15.39:0/774048696 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f802400fb00 con 0x7f803c105180 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.122+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c1a3e70 con 0x7f803c105180 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.122+0000 7f8033fff700 1 -- 172.21.15.39:0/774048696 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024022b80 con 0x7f803c105180 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.122+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c1a4230 con 0x7f803c105180 2024-05-24T11:54:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.123+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c108450 con 0x7f803c105180 2024-05-24T11:54:59.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.124+0000 7f8033fff700 1 -- 172.21.15.39:0/774048696 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f802400fc70 con 0x7f803c105180 2024-05-24T11:54:59.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.125+0000 7f8033fff700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f80280a1290 0x7f80280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.125+0000 7f803affd700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f80280a1290 0x7f80280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.125+0000 7f8033fff700 1 -- 172.21.15.39:0/774048696 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f80240ce910 con 0x7f803c105180 2024-05-24T11:54:59.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.125+0000 7f803affd700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f80280a1290 0x7f80280a3750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f802c005950 tx=0x7f802c00b410 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.128+0000 7f8033fff700 1 -- 172.21.15.39:0/774048696 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8024097e00 con 0x7f803c105180 2024-05-24T11:54:59.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.335+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f803c069aa0 con 0x7f803c105180 2024-05-24T11:54:59.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.335+0000 7f8033fff700 1 -- 172.21.15.39:0/774048696 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80240270a0 con 0x7f803c105180 2024-05-24T11:54:59.339 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-24T11:54:59.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f80280a1290 msgr2=0x7f80280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f80280a1290 0x7f80280a3750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f802c005950 tx=0x7f802c00b410 comp rx=0 tx=0).stop 2024-05-24T11:54:59.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c105180 msgr2=0x7f803c1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c105180 0x7f803c1986a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f80240036c0 tx=0x7f8024009580 comp rx=0 tx=0).stop 2024-05-24T11:54:59.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 shutdown_connections 2024-05-24T11:54:59.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f803c100060 0x7f803c198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f80280a1290 0x7f80280a3750 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f803c198be0 0x7f803c1a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 --2- 172.21.15.39:0/774048696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c105180 0x7f803c1986a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 >> 172.21.15.39:0/774048696 conn(0x7f803c076fa0 msgr2=0x7f803c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:59.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 shutdown_connections 2024-05-24T11:54:59.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.337+0000 7f8041373700 1 -- 172.21.15.39:0/774048696 wait complete. 2024-05-24T11:54:59.353 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-24T11:54:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.479+0000 7f2c86927700 1 -- 172.21.15.39:0/2193396108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80101b10 msgr2=0x7f2c80105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.479+0000 7f2c86927700 1 --2- 172.21.15.39:0/2193396108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80101b10 0x7f2c80105b50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009990 tx=0x7f2c74009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.479+0000 7f2c86927700 1 -- 172.21.15.39:0/2193396108 shutdown_connections 2024-05-24T11:54:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.479+0000 7f2c86927700 1 --2- 172.21.15.39:0/2193396108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c80106280 0x7f2c8010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.479+0000 7f2c86927700 1 --2- 172.21.15.39:0/2193396108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c80101160 0x7f2c80101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.479+0000 7f2c86927700 1 --2- 172.21.15.39:0/2193396108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80101b10 0x7f2c80105b50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.479+0000 7f2c86927700 1 -- 172.21.15.39:0/2193396108 >> 172.21.15.39:0/2193396108 conn(0x7f2c800fc9e0 msgr2=0x7f2c800fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.479+0000 7f2c86927700 1 -- 172.21.15.39:0/2193396108 shutdown_connections 2024-05-24T11:54:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.479+0000 7f2c86927700 1 -- 172.21.15.39:0/2193396108 wait complete. 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.480+0000 7f2c86927700 1 Processor -- start 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.480+0000 7f2c86927700 1 -- start start 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.480+0000 7f2c86927700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c80101160 0x7f2c80197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c86927700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80106280 0x7f2c80197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c86927700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8019bc00 0x7f2c8019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c86927700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019c680 con 0x7f2c80106280 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c86927700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019c7f0 con 0x7f2c80101160 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c86927700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019c9b0 con 0x7f2c8019bc00 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80106280 0x7f2c80197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80106280 0x7f2c80197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40184/0 (socket says 172.21.15.39:40184) 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7f7fe700 1 -- 172.21.15.39:0/1464780530 learned_addr learned my addr 172.21.15.39:0/1464780530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c84ec4700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8019bc00 0x7f2c8019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7ffff700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c80101160 0x7f2c80197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7f7fe700 1 -- 172.21.15.39:0/1464780530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8019bc00 msgr2=0x7f2c8019c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7f7fe700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8019bc00 0x7f2c8019c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7f7fe700 1 -- 172.21.15.39:0/1464780530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c80101160 msgr2=0x7f2c80197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7f7fe700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c80101160 0x7f2c80197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7f7fe700 1 -- 172.21.15.39:0/1464780530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740095f0 con 0x7f2c80106280 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7ffff700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c80101160 0x7f2c80197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7f7fe700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80106280 0x7f2c80197860 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2c740036c0 tx=0x7f2c74009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7d7fa700 1 -- 172.21.15.39:0/1464780530 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7401d070 con 0x7f2c80106280 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7d7fa700 1 -- 172.21.15.39:0/1464780530 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c7400fb00 con 0x7f2c80106280 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c801a3eb0 con 0x7f2c80106280 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c7d7fa700 1 -- 172.21.15.39:0/1464780530 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c74022b80 con 0x7f2c80106280 2024-05-24T11:54:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.481+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c801a4270 con 0x7f2c80106280 2024-05-24T11:54:59.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.482+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c80109550 con 0x7f2c80106280 2024-05-24T11:54:59.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.483+0000 7f2c7d7fa700 1 -- 172.21.15.39:0/1464780530 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2c7400fc70 con 0x7f2c80106280 2024-05-24T11:54:59.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.484+0000 7f2c7d7fa700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2c680a1290 0x7f2c680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.484+0000 7f2c7d7fa700 1 -- 172.21.15.39:0/1464780530 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2c740ce1e0 con 0x7f2c80106280 2024-05-24T11:54:59.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.484+0000 7f2c7ffff700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2c680a1290 0x7f2c680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.484+0000 7f2c7ffff700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2c680a1290 0x7f2c680a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2c70005fd0 tx=0x7f2c70005ee0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:59.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.488+0000 7f2c7d7fa700 1 -- 172.21.15.39:0/1464780530 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2c740977f0 con 0x7f2c80106280 2024-05-24T11:54:59.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:59 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/2752018338' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-24T11:54:59.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:54:59 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/4217386575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:54:59.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.697+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c80198620 con 0x7f2c80106280 2024-05-24T11:54:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.697+0000 7f2c7d7fa700 1 -- 172.21.15.39:0/1464780530 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c74027020 con 0x7f2c80106280 2024-05-24T11:54:59.701 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-05-24T11:54:59.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2c680a1290 msgr2=0x7f2c680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2c680a1290 0x7f2c680a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2c70005fd0 tx=0x7f2c70005ee0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80106280 msgr2=0x7f2c80197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80106280 0x7f2c80197860 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2c740036c0 tx=0x7f2c74009580 comp rx=0 tx=0).stop 2024-05-24T11:54:59.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 shutdown_connections 2024-05-24T11:54:59.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c8019bc00 0x7f2c8019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f2c680a1290 0x7f2c680a3750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c80101160 0x7f2c80197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 --2- 172.21.15.39:0/1464780530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c80106280 0x7f2c80197860 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 >> 172.21.15.39:0/1464780530 conn(0x7f2c800fc9e0 msgr2=0x7f2c800fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:59.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 shutdown_connections 2024-05-24T11:54:59.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.699+0000 7f2c86927700 1 -- 172.21.15.39:0/1464780530 wait complete. 2024-05-24T11:54:59.716 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-24T11:54:59.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:59 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/2752018338' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-24T11:54:59.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:54:59 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/4217386575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:54:59.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:59 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/2752018338' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-24T11:54:59.744 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:54:59 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/4217386575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-24T11:54:59.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.843+0000 7fcd19b3e700 1 -- 172.21.15.39:0/688649234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd14106280 msgr2=0x7fcd1410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.843+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/688649234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd14106280 0x7fcd1410de00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009990 tx=0x7fcd0c009ca0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.843+0000 7fcd19b3e700 1 -- 172.21.15.39:0/688649234 shutdown_connections 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.843+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/688649234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd14106280 0x7fcd1410de00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.843+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/688649234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd14101160 0x7fcd14101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.843+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/688649234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd14101b10 0x7fcd14105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.843+0000 7fcd19b3e700 1 -- 172.21.15.39:0/688649234 >> 172.21.15.39:0/688649234 conn(0x7fcd140fc9e0 msgr2=0x7fcd140fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.843+0000 7fcd19b3e700 1 -- 172.21.15.39:0/688649234 shutdown_connections 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.843+0000 7fcd19b3e700 1 -- 172.21.15.39:0/688649234 wait complete. 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd19b3e700 1 Processor -- start 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd19b3e700 1 -- start start 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd19b3e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd14101160 0x7fcd141972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd19b3e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd14106280 0x7fcd14197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd19b3e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd1419bba0 0x7fcd1419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd19b3e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1419c620 con 0x7fcd1419bba0 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd19b3e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1419c790 con 0x7fcd14106280 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd19b3e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1419c950 con 0x7fcd14101160 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd13fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd1419bba0 0x7fcd1419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd13fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd1419bba0 0x7fcd1419c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40204/0 (socket says 172.21.15.39:40204) 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd13fff700 1 -- 172.21.15.39:0/3345578136 learned_addr learned my addr 172.21.15.39:0/3345578136 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd137fe700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd14101160 0x7fcd141972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd12ffd700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd14106280 0x7fcd14197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd13fff700 1 -- 172.21.15.39:0/3345578136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd14101160 msgr2=0x7fcd141972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd13fff700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd14101160 0x7fcd141972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd13fff700 1 -- 172.21.15.39:0/3345578136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd14106280 msgr2=0x7fcd14197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd13fff700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd14106280 0x7fcd14197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd13fff700 1 -- 172.21.15.39:0/3345578136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0095f0 con 0x7fcd1419bba0 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd137fe700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd14101160 0x7fcd141972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.844+0000 7fcd12ffd700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd14106280 0x7fcd14197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.845+0000 7fcd13fff700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd1419bba0 0x7fcd1419c020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c004760 tx=0x7fcd0c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.845+0000 7fcd10ff9700 1 -- 172.21.15.39:0/3345578136 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c01d070 con 0x7fcd1419bba0 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.845+0000 7fcd10ff9700 1 -- 172.21.15.39:0/3345578136 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd0c004ca0 con 0x7fcd1419bba0 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.845+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd141a3e50 con 0x7fcd1419bba0 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.845+0000 7fcd10ff9700 1 -- 172.21.15.39:0/3345578136 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c00f950 con 0x7fcd1419bba0 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.845+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd141a4270 con 0x7fcd1419bba0 2024-05-24T11:54:59.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.846+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd141094f0 con 0x7fcd1419bba0 2024-05-24T11:54:59.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.847+0000 7fcd10ff9700 1 -- 172.21.15.39:0/3345578136 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcd0c0050c0 con 0x7fcd1419bba0 2024-05-24T11:54:59.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.848+0000 7fcd10ff9700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fccfc0a1290 0x7fccfc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:54:59.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.848+0000 7fcd10ff9700 1 -- 172.21.15.39:0/3345578136 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcd0c0cdf30 con 0x7fcd1419bba0 2024-05-24T11:54:59.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.848+0000 7fcd137fe700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fccfc0a1290 0x7fccfc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:54:59.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.848+0000 7fcd137fe700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fccfc0a1290 0x7fccfc0a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009dd0 tx=0x7fcd04009040 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:54:59.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:54:59.851+0000 7fcd10ff9700 1 -- 172.21.15.39:0/3345578136 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcd0c097420 con 0x7fcd1419bba0 2024-05-24T11:55:00.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.059+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd141985c0 con 0x7fcd1419bba0 2024-05-24T11:55:00.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.059+0000 7fcd10ff9700 1 -- 172.21.15.39:0/3345578136 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd0c09aa40 con 0x7fcd1419bba0 2024-05-24T11:55:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-05-24T11:55:00.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fccfc0a1290 msgr2=0x7fccfc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:55:00.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fccfc0a1290 0x7fccfc0a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009dd0 tx=0x7fcd04009040 comp rx=0 tx=0).stop 2024-05-24T11:55:00.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd1419bba0 msgr2=0x7fcd1419c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:55:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd1419bba0 0x7fcd1419c020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c004760 tx=0x7fcd0c009580 comp rx=0 tx=0).stop 2024-05-24T11:55:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 shutdown_connections 2024-05-24T11:55:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd14101160 0x7fcd141972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7fccfc0a1290 0x7fccfc0a3750 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd1419bba0 0x7fcd1419c020 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 --2- 172.21.15.39:0/3345578136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd14106280 0x7fcd14197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 >> 172.21.15.39:0/3345578136 conn(0x7fcd140fc9e0 msgr2=0x7fcd140fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:55:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 shutdown_connections 2024-05-24T11:55:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.062+0000 7fcd19b3e700 1 -- 172.21.15.39:0/3345578136 wait complete. 2024-05-24T11:55:00.079 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-24T11:55:00.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.203+0000 7f7b0b995700 1 -- 172.21.15.39:0/3149988376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b041062a0 msgr2=0x7f7b0410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:55:00.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.203+0000 7f7b0b995700 1 --2- 172.21.15.39:0/3149988376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b041062a0 0x7f7b0410de20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00b0a0 tx=0x7f7afc00b3b0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.203+0000 7f7b0b995700 1 -- 172.21.15.39:0/3149988376 shutdown_connections 2024-05-24T11:55:00.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.203+0000 7f7b0b995700 1 --2- 172.21.15.39:0/3149988376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b041062a0 0x7f7b0410de20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.203+0000 7f7b0b995700 1 --2- 172.21.15.39:0/3149988376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b04069400 0x7f7b040697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.203+0000 7f7b0b995700 1 --2- 172.21.15.39:0/3149988376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b04069db0 0x7f7b04105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.203+0000 7f7b0b995700 1 -- 172.21.15.39:0/3149988376 >> 172.21.15.39:0/3149988376 conn(0x7f7b04076ba0 msgr2=0x7f7b04076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:55:00.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.203+0000 7f7b0b995700 1 -- 172.21.15.39:0/3149988376 shutdown_connections 2024-05-24T11:55:00.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.203+0000 7f7b0b995700 1 -- 172.21.15.39:0/3149988376 wait complete. 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b0b995700 1 Processor -- start 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b0b995700 1 -- start start 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b0b995700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b04069400 0x7f7b041001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b0b995700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b041062a0 0x7f7b04100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b0b995700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b04100c40 0x7f7b041a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b0b995700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b041a5c80 con 0x7f7b04100c40 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b0b995700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b041a5df0 con 0x7f7b041062a0 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b0b995700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b041a5fb0 con 0x7f7b04069400 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b04100c40 0x7f7b041a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09f32700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b04100c40 0x7f7b041a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40226/0 (socket says 172.21.15.39:40226) 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09f32700 1 -- 172.21.15.39:0/958703128 learned_addr learned my addr 172.21.15.39:0/958703128 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b08f30700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b041062a0 0x7f7b04100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:55:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09731700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b04069400 0x7f7b041001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09f32700 1 -- 172.21.15.39:0/958703128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b04069400 msgr2=0x7f7b041001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09f32700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b04069400 0x7f7b041001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09f32700 1 -- 172.21.15.39:0/958703128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b041062a0 msgr2=0x7f7b04100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09f32700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b041062a0 0x7f7b04100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09f32700 1 -- 172.21.15.39:0/958703128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc009d00 con 0x7f7b04100c40 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09731700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b04069400 0x7f7b041001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.204+0000 7f7b09f32700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b04100c40 0x7f7b041a5640 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7afc006130 tx=0x7f7afc0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.205+0000 7f7afa7fc700 1 -- 172.21.15.39:0/958703128 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00e040 con 0x7f7b04100c40 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.205+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b041a6230 con 0x7f7b04100c40 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.205+0000 7f7afa7fc700 1 -- 172.21.15.39:0/958703128 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7afc00bc50 con 0x7f7b04100c40 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.205+0000 7f7afa7fc700 1 -- 172.21.15.39:0/958703128 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0052a0 con 0x7f7b04100c40 2024-05-24T11:55:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.205+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b041a66b0 con 0x7f7b04100c40 2024-05-24T11:55:00.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.205+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04109570 con 0x7f7b04100c40 2024-05-24T11:55:00.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.207+0000 7f7afa7fc700 1 -- 172.21.15.39:0/958703128 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7afc01b450 con 0x7f7b04100c40 2024-05-24T11:55:00.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.208+0000 7f7afa7fc700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7af00a1290 0x7f7af00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-24T11:55:00.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.208+0000 7f7b09731700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7af00a1290 0x7f7af00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-24T11:55:00.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.208+0000 7f7afa7fc700 1 -- 172.21.15.39:0/958703128 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7afc0ce840 con 0x7f7b04100c40 2024-05-24T11:55:00.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.208+0000 7f7b09731700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7af00a1290 0x7f7af00a3750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009cb0 tx=0x7f7b00009260 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-24T11:55:00.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.211+0000 7f7afa7fc700 1 -- 172.21.15.39:0/958703128 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7afc097d30 con 0x7f7b04100c40 2024-05-24T11:55:00.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.436+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7b04102650 con 0x7f7b04100c40 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.437+0000 7f7afa7fc700 1 -- 172.21.15.39:0/958703128 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v107) v1 ==== 81+0+1001 (secure 0 0 0) 0x7f7afc09b350 con 0x7f7b04100c40 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2243717102 2024-05-25T11:31:20.894336+0000 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3960296835 2024-05-25T11:31:41.814659+0000 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3563042541 2024-05-25T11:31:41.814659+0000 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2910856381 2024-05-25T11:31:20.894336+0000 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1790202809 2024-05-25T11:31:20.894336+0000 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1012810399 2024-05-25T11:31:20.894336+0000 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3960296835 2024-05-25T11:31:41.814659+0000 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1790202809 2024-05-25T11:31:20.894336+0000 2024-05-24T11:55:00.441 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1559144405 2024-05-25T11:31:41.814659+0000 2024-05-24T11:55:00.442 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3982984387 2024-05-25T11:31:41.814659+0000 2024-05-24T11:55:00.442 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/338214439 2024-05-25T11:38:12.183530+0000 2024-05-24T11:55:00.442 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/870992183 2024-05-25T11:38:12.183530+0000 2024-05-24T11:55:00.442 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3248772673 2024-05-25T11:38:12.183530+0000 2024-05-24T11:55:00.442 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3290494740 2024-05-25T11:38:12.183530+0000 2024-05-24T11:55:00.442 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/333059399 2024-05-25T11:38:12.183530+0000 2024-05-24T11:55:00.442 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/2090515453 2024-05-25T11:38:12.183530+0000 2024-05-24T11:55:00.442 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/2090515453 2024-05-25T11:38:12.183530+0000 2024-05-24T11:55:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7af00a1290 msgr2=0x7f7af00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:55:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7af00a1290 0x7f7af00a3750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009cb0 tx=0x7f7b00009260 comp rx=0 tx=0).stop 2024-05-24T11:55:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b04100c40 msgr2=0x7f7b041a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-24T11:55:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b04100c40 0x7f7b041a5640 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7afc006130 tx=0x7f7afc0095b0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 shutdown_connections 2024-05-24T11:55:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b04069400 0x7f7b041001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:6800/1882625876,v1:172.21.15.39:6801/1882625876] conn(0x7f7af00a1290 0x7f7af00a3750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b04100c40 0x7f7b041a5640 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 --2- 172.21.15.39:0/958703128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b041062a0 0x7f7b04100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-24T11:55:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 >> 172.21.15.39:0/958703128 conn(0x7f7b04076ba0 msgr2=0x7f7b04107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-24T11:55:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 shutdown_connections 2024-05-24T11:55:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-24T11:55:00.439+0000 7f7b0b995700 1 -- 172.21.15.39:0/958703128 wait complete. 2024-05-24T11:55:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:listed 17 entries 2024-05-24T11:55:00.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:00 smithi039 ceph-mon[42919]: pgmap v533: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 op/s 2024-05-24T11:55:00.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:00 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/774048696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:55:00.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:00 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/1464780530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:55:00.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:00 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/3345578136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:55:00.456 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi039.front.sepia.ceph.com... 2024-05-24T11:55:00.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:55:00.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo fusermount -u /tmp/cephfs 2024-05-24T11:55:00.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:ceph-fuse[107557]: fuse finished with error 0 and tester_r 0 2024-05-24T11:55:00.644 INFO:teuthology.orchestra.run:waiting for 300 2024-05-24T11:55:00.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:00 smithi053 ceph-mon[47448]: pgmap v533: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 op/s 2024-05-24T11:55:00.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:00 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/774048696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:55:00.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:00 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/1464780530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:55:00.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:00 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/3345578136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:55:00.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:00 smithi137 ceph-mon[47907]: pgmap v533: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 op/s 2024-05-24T11:55:00.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:00 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/774048696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-24T11:55:00.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:00 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/1464780530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-24T11:55:00.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:00 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/3345578136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-24T11:55:01.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:01 smithi039 ceph-mon[42919]: from='client.? 172.21.15.39:0/958703128' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-24T11:55:01.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:01 smithi053 ceph-mon[47448]: from='client.? 172.21.15.39:0/958703128' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-24T11:55:01.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:01 smithi137 ceph-mon[47907]: from='client.? 172.21.15.39:0/958703128' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-24T11:55:02.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:02 smithi053 ceph-mon[47448]: pgmap v534: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-24T11:55:02.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:02 smithi137 ceph-mon[47907]: pgmap v534: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-24T11:55:02.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:02 smithi039 ceph-mon[42919]: pgmap v534: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-24T11:55:04.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:04 smithi053 ceph-mon[47448]: pgmap v535: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-24T11:55:04.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:04 smithi053 ceph-mon[47448]: osdmap e108: 12 total, 12 up, 12 in 2024-05-24T11:55:04.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:04 smithi137 ceph-mon[47907]: pgmap v535: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-24T11:55:04.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:04 smithi137 ceph-mon[47907]: osdmap e108: 12 total, 12 up, 12 in 2024-05-24T11:55:04.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:04 smithi039 ceph-mon[42919]: pgmap v535: 97 pgs: 97 active+clean; 5.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-24T11:55:04.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:04 smithi039 ceph-mon[42919]: osdmap e108: 12 total, 12 up, 12 in 2024-05-24T11:55:06.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:06 smithi053 ceph-mon[47448]: pgmap v537: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 32 op/s 2024-05-24T11:55:06.804 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T11:55:06.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:55:06.804 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-24T11:55:06.805 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:06 smithi137 ceph-mon[47907]: pgmap v537: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 32 op/s 2024-05-24T11:55:06.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:06 smithi039 ceph-mon[42919]: pgmap v537: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 5.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 32 op/s 2024-05-24T11:55:06.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:55:06.851 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-05-24T11:55:06.947 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-24T11:55:06.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:55:06.948 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-24T11:55:06.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-24T11:55:06.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-05-24T11:55:07.087 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-24T11:55:07.099 INFO:tasks.cephadm:Teardown begin 2024-05-24T11:55:07.099 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa/tasks/cephadm.py", line 1750, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-05-24T11:55:07.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-24T11:55:07.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-24T11:55:07.183 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-24T11:55:07.213 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-24T11:55:07.214 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-24T11:55:07.237 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-24T11:55:07.259 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-24T11:55:07.276 INFO:tasks.cephadm:Stopping all daemons... 2024-05-24T11:55:07.277 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-24T11:55:07.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.a 2024-05-24T11:55:07.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:07 smithi039 systemd[1]: Stopping Ceph mon.a for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:07.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-a[42896]: 2024-05-24T11:55:07.518+0000 7f16714f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:07.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-a[42896]: 2024-05-24T11:55:07.518+0000 7f16714f7700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-24T11:55:08.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 24 11:55:08 smithi039 bash[136580]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-a 2024-05-24T11:55:08.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-24T11:55:08.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:08.410 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-24T11:55:08.410 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-24T11:55:08.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.b 2024-05-24T11:55:08.737 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:08 smithi053 systemd[1]: Stopping Ceph mon.b for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:08.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-b[47425]: 2024-05-24T11:55:08.666+0000 7f1a2b7f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:08.738 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-b[47425]: 2024-05-24T11:55:08.666+0000 7f1a2b7f0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-24T11:55:09.237 INFO:journalctl@ceph.mon.b.smithi053.stdout:May 24 11:55:08 smithi053 bash[91167]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-b 2024-05-24T11:55:09.248 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-24T11:55:09.310 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:09.310 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-24T11:55:09.310 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-24T11:55:09.310 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.c 2024-05-24T11:55:09.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:09 smithi137 systemd[1]: Stopping Ceph mon.c for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:09.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-c[47883]: 2024-05-24T11:55:09.538+0000 7f64c64ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:09.743 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-c[47883]: 2024-05-24T11:55:09.538+0000 7f64c64ae700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-24T11:55:10.181 INFO:journalctl@ceph.mon.c.smithi137.stdout:May 24 11:55:09 smithi137 bash[92893]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mon-c 2024-05-24T11:55:10.188 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-24T11:55:10.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:10.237 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-24T11:55:10.238 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-24T11:55:10.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.x 2024-05-24T11:55:10.576 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:55:10 smithi039 systemd[1]: Stopping Ceph mgr.x for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:10.576 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:55:10 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x[43169]: 2024-05-24T11:55:10.484+0000 7efc45fad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:10.933 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 24 11:55:10 smithi039 bash[136726]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-x 2024-05-24T11:55:10.940 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-24T11:55:10.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:10.980 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-24T11:55:10.980 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-24T11:55:10.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.y 2024-05-24T11:55:11.373 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:55:11 smithi053 systemd[1]: Stopping Ceph mgr.y for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:11.702 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:55:11 smithi053 bash[91308]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-y 2024-05-24T11:55:11.702 INFO:journalctl@ceph.mgr.y.smithi053.stdout:May 24 11:55:11 smithi053 systemd[1]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-24T11:55:11.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-24T11:55:11.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:11.751 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-24T11:55:11.751 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-24T11:55:11.752 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.z 2024-05-24T11:55:12.138 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:55:11 smithi137 systemd[1]: Stopping Ceph mgr.z for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:12.444 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:55:12 smithi137 bash[93036]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mgr-z 2024-05-24T11:55:12.445 INFO:journalctl@ceph.mgr.z.smithi137.stdout:May 24 11:55:12 smithi137 systemd[1]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-24T11:55:12.451 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-24T11:55:12.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:12.487 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-24T11:55:12.487 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-24T11:55:12.487 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.0 2024-05-24T11:55:12.826 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:55:12 smithi039 systemd[1]: Stopping Ceph osd.0 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:12.827 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:55:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-0[60527]: 2024-05-24T11:55:12.734+0000 7f1bf4ef6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:12.827 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:55:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-0[60527]: 2024-05-24T11:55:12.734+0000 7f1bf4ef6700 -1 osd.0 108 *** Got signal Terminated *** 2024-05-24T11:55:12.827 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:55:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-0[60527]: 2024-05-24T11:55:12.734+0000 7f1bf4ef6700 -1 osd.0 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:55:18.576 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 24 11:55:18 smithi039 bash[136865]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-0 2024-05-24T11:55:19.740 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-24T11:55:19.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:19.783 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-24T11:55:19.783 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-24T11:55:19.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.1 2024-05-24T11:55:20.237 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:55:19 smithi053 systemd[1]: Stopping Ceph osd.1 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:20.237 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:55:20 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-1[52465]: 2024-05-24T11:55:20.052+0000 7fc30c648700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:20.238 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:55:20 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-1[52465]: 2024-05-24T11:55:20.052+0000 7fc30c648700 -1 osd.1 108 *** Got signal Terminated *** 2024-05-24T11:55:20.238 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:55:20 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-1[52465]: 2024-05-24T11:55:20.052+0000 7fc30c648700 -1 osd.1 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:55:25.987 INFO:journalctl@ceph.osd.1.smithi053.stdout:May 24 11:55:25 smithi053 bash[91447]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-1 2024-05-24T11:55:26.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-24T11:55:27.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:27.011 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-24T11:55:27.011 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-24T11:55:27.011 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.2 2024-05-24T11:55:27.493 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:55:27 smithi137 systemd[1]: Stopping Ceph osd.2 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:27.494 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:55:27 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-2[53734]: 2024-05-24T11:55:27.269+0000 7fdfc253f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:27.494 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:55:27 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-2[53734]: 2024-05-24T11:55:27.269+0000 7fdfc253f700 -1 osd.2 108 *** Got signal Terminated *** 2024-05-24T11:55:27.494 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:55:27 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-2[53734]: 2024-05-24T11:55:27.269+0000 7fdfc253f700 -1 osd.2 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:55:33.244 INFO:journalctl@ceph.osd.2.smithi137.stdout:May 24 11:55:32 smithi137 bash[93183]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-2 2024-05-24T11:55:34.043 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-24T11:55:34.084 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:34.084 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-24T11:55:34.084 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-24T11:55:34.084 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.3 2024-05-24T11:55:34.576 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 24 11:55:34 smithi039 systemd[1]: Stopping Ceph osd.3 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:34.576 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 24 11:55:34 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-3[69108]: 2024-05-24T11:55:34.331+0000 7f06c3048700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:34.576 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 24 11:55:34 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-3[69108]: 2024-05-24T11:55:34.331+0000 7f06c3048700 -1 osd.3 108 *** Got signal Terminated *** 2024-05-24T11:55:34.576 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 24 11:55:34 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-3[69108]: 2024-05-24T11:55:34.331+0000 7f06c3048700 -1 osd.3 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:55:39.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:39 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:39.255+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:39.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:39 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:39.409+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:39.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:39 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:39.443+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:40.326 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 24 11:55:40 smithi039 bash[137126]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-3 2024-05-24T11:55:40.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:40 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:40.214+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:40.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:40 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:40.359+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:40.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:40 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:40.488+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:40.827 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:40 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:40.589+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:41.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-24T11:55:41.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:41.271 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-24T11:55:41.271 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-24T11:55:41.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.4 2024-05-24T11:55:41.491 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:55:41 smithi053 systemd[1]: Stopping Ceph osd.4 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:41.491 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:41 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:41.181+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:41.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:41 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:41.291+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:41.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:41 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:41.312+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:41.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:41 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:41.472+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:41.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:41 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:41.614+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:41.987 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:55:41 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-4[61198]: 2024-05-24T11:55:41.545+0000 7f95caeae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:41.987 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:55:41 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-4[61198]: 2024-05-24T11:55:41.545+0000 7f95caeae700 -1 osd.4 108 *** Got signal Terminated *** 2024-05-24T11:55:41.987 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:55:41 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-4[61198]: 2024-05-24T11:55:41.545+0000 7f95caeae700 -1 osd.4 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:55:42.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:42 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:42.188+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:42.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:42 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:42.322+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:42.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:42 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:42.329+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:42.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:42 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:42.442+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:42.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:42 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:42.473+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:42.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:42 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:42.644+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:43.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:43 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:43.236+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:43.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:43 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:43.312+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:43.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:43 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:43.323+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:43.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:43 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:43.441+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:43.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:43 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:43.643+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:43.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:43 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:43.479+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:44.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:44 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:44.224+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:44.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:44 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:44.318+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:44.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:44 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:44.310+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:44.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:44 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:44.411+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:44.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:44 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:44.687+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:44.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:44 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:44.442+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:45.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:45 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:45.236+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:45.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:45 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:45.274+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:45.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:45 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:45.346+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:45.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:45 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:45.399+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:45.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:45 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:45.669+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:45.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:45 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:45.462+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:46.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:46 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:46.234+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:46.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:46 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:46.353+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:46.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:46 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:46.402+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:46.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:46 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:46.300+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:46.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:46 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:46.300+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:20.576092+0000 front 2024-05-24T11:55:20.576016+0000 (oldest deadline 2024-05-24T11:55:45.876183+0000) 2024-05-24T11:55:46.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:46 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:46.707+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:46.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:46 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:46.498+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:46.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:46 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:46.498+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:47.520 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:47 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:47.258+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:47.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:47 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:47.380+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:47.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:47 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:47.418+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:47.744 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:47 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:47.293+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:47.744 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:47 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:47.293+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:20.576092+0000 front 2024-05-24T11:55:20.576016+0000 (oldest deadline 2024-05-24T11:55:45.876183+0000) 2024-05-24T11:55:47.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:47 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:47.534+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:47.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:47 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:47.534+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:47.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:47 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:47.740+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:47.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:47 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:47.740+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:47.988 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:55:47 smithi053 bash[91713]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-4 2024-05-24T11:55:48.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:48 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:48.255+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:48.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:48 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:48.406+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:48.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:48 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:48.291+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:48.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:48 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:48.291+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:20.576092+0000 front 2024-05-24T11:55:20.576016+0000 (oldest deadline 2024-05-24T11:55:45.876183+0000) 2024-05-24T11:55:48.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:48 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:48.379+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:48.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:48 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:48.379+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:48.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:48 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:48.499+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:48.827 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:48 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:48.499+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:48.827 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:48 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:48.743+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:48.828 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:48 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:48.743+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:49.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-24T11:55:49.163 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:55:49 smithi053 systemd[1]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.4.service: Succeeded. 2024-05-24T11:55:49.163 INFO:journalctl@ceph.osd.4.smithi053.stdout:May 24 11:55:49 smithi053 systemd[1]: Stopped Ceph osd.4 for f68008c4-19c0-11ef-bc9a-c7b262605968. 2024-05-24T11:55:49.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:49.182 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-24T11:55:49.182 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-24T11:55:49.182 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.5 2024-05-24T11:55:49.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:49 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:49.241+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:49.493 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:49 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:49.386+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:49.493 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:49 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:49.403+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:49.494 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:49 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:49.403+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:49.494 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:49 smithi137 systemd[1]: Stopping Ceph osd.5 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:49.494 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:49 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:49.304+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:49.494 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:49 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:49.304+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:20.576092+0000 front 2024-05-24T11:55:20.576016+0000 (oldest deadline 2024-05-24T11:55:45.876183+0000) 2024-05-24T11:55:49.494 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:49 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:49.449+0000 7fbc7531f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:49.494 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:49 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:49.449+0000 7fbc7531f700 -1 osd.5 108 *** Got signal Terminated *** 2024-05-24T11:55:49.494 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:49 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:49.449+0000 7fbc7531f700 -1 osd.5 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:55:49.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:49 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:49.506+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:49.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:49 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:49.506+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:49.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:49 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:49.754+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:49.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:49 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:49.754+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:50.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:50 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:50.331+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:50.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:50 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:50.192+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:50.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:50 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:50.192+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:50.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:50 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:50.392+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:50.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:50 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:50.281+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:50.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:50 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:50.281+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:20.576092+0000 front 2024-05-24T11:55:20.576016+0000 (oldest deadline 2024-05-24T11:55:45.876183+0000) 2024-05-24T11:55:50.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:50 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:50.421+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:50.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:50 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:50.421+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:50.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:50 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:50.483+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:50.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:50 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:50.483+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:50.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:50 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:50.731+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:50.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:50 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:50.731+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:51.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:51 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:51.341+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:51.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:51 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:51.220+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:51.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:51 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:51.220+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:51.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:51 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:51.428+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:51.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:51 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:51.428+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:55:51.744 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:51 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:51.312+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:51.744 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:51 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:51.312+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:20.576092+0000 front 2024-05-24T11:55:20.576016+0000 (oldest deadline 2024-05-24T11:55:45.876183+0000) 2024-05-24T11:55:51.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:51 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:51.378+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:51.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:51 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:51.378+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:51.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:51 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:51.464+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:51.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:51 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:51.464+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:51.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:51 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:51.710+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:51.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:51 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:51.710+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:52.488 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:52 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:52.350+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:52.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:52 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:52.188+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:52.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:52 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:52.188+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:52.735 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:52 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:52.461+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:52.735 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:52 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:52.461+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:52.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:52 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:52.426+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:52.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:52 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:52.426+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:55:52.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:52 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:52.350+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:52.744 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:52 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:52.350+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:20.576092+0000 front 2024-05-24T11:55:20.576016+0000 (oldest deadline 2024-05-24T11:55:45.876183+0000) 2024-05-24T11:55:52.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:52 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:52.426+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:52.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:52 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:52.426+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:53.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:52 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:52.732+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:53.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:52 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:52.732+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:53.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:53 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:53.305+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:53.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:53 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:53.201+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:53.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:53 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:53.201+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:53.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:53 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:53.386+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:53.743 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:53 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:53.386+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:20.576092+0000 front 2024-05-24T11:55:20.576016+0000 (oldest deadline 2024-05-24T11:55:45.876183+0000) 2024-05-24T11:55:53.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:53 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:53.414+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:53.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:53 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:53.414+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:53.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:53 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:53.425+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:53.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:53 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:53.425+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:55:53.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:53 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:53.501+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:53.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:53 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:53.501+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:53.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:53 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:53.713+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:53.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:53 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:53.713+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:54.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:54 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:54.334+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:54.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:54 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:54.236+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:54.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:54 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:54.236+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:54.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:54 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:54.470+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:54.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:54 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:54.470+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:55:54.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:54 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:54.470+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:55:54.744 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:54 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:54.362+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:16.475565+0000 front 2024-05-24T11:55:16.475629+0000 (oldest deadline 2024-05-24T11:55:40.575640+0000) 2024-05-24T11:55:54.744 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:54 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5[62336]: 2024-05-24T11:55:54.362+0000 7fbc6e10e700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:20.576092+0000 front 2024-05-24T11:55:20.576016+0000 (oldest deadline 2024-05-24T11:55:45.876183+0000) 2024-05-24T11:55:54.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:54 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:54.407+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:54.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:54 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:54.407+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:54.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:54.510+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:54.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:54.510+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:54.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:54.510+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.148040+0000 front 2024-05-24T11:55:29.148062+0000 (oldest deadline 2024-05-24T11:55:53.848037+0000) 2024-05-24T11:55:54.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:54.716+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:54.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:54 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:54.716+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:55.441 INFO:journalctl@ceph.osd.5.smithi137.stdout:May 24 11:55:55 smithi137 bash[93451]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-5 2024-05-24T11:55:55.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:55 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:55.310+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:55.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:55 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:55.252+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:55.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:55 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:55.252+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:55.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:55 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:55.446+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:55.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:55 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:55.446+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:55.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:55 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:55.437+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:55.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:55 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:55.437+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:55:55.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:55 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:55.437+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:55:55.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:55 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:55.495+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:55.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:55 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:55.495+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:55.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:55 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:55.495+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.148040+0000 front 2024-05-24T11:55:29.148062+0000 (oldest deadline 2024-05-24T11:55:53.848037+0000) 2024-05-24T11:55:55.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:55 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:55.742+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:55.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:55 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:55.742+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:56.365 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-24T11:55:56.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:55:56.406 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-24T11:55:56.406 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-24T11:55:56.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.6 2024-05-24T11:55:56.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:56 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:56.274+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:56.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:56 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:56.274+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:55:56.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:56 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:56.203+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:56.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:56 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:56.203+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:56.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:56 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:56.203+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:55:56.734 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:56 smithi039 systemd[1]: Stopping Ceph osd.6 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:55:56.735 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:56.539+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:56.735 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:56.539+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:56.735 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:56.539+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.148040+0000 front 2024-05-24T11:55:29.148062+0000 (oldest deadline 2024-05-24T11:55:53.848037+0000) 2024-05-24T11:55:56.735 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:56.643+0000 7faa6c3b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:55:56.735 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:56.643+0000 7faa6c3b0700 -1 osd.6 108 *** Got signal Terminated *** 2024-05-24T11:55:56.735 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:56.643+0000 7faa6c3b0700 -1 osd.6 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:55:56.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:56 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:56.423+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:56.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:56 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:56.423+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:56.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:56 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:56.486+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:56.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:56 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:56.486+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:55:56.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:56 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:56.486+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:55:57.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:56.730+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:57.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:56 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:56.730+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:57.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:57 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:57.234+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:57.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:57 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:57.234+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:55:57.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:57 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:57.156+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:57.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:57 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:57.156+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:57.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:57 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:57.156+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:55:57.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:57 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:57.427+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:57.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:57 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:57.427+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:57.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:57 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:57.530+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:57.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:57 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:57.530+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:55:57.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:57 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:57.530+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:55:57.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:57 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:57.554+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:57.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:57 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:57.554+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:57.826 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:57 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:57.554+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.148040+0000 front 2024-05-24T11:55:29.148062+0000 (oldest deadline 2024-05-24T11:55:53.848037+0000) 2024-05-24T11:55:57.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:57 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:57.735+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:57.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:57 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:57.735+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:57.826 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:57 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:57.735+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:55:58.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:58 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:58.277+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:58.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:58 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:58.277+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:55:58.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:58 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:58.205+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:58.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:58 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:58.205+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:58.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:58 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:58.205+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:55:58.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:58 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:58.425+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:58.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:58 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:58.425+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:58.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:58 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:58.502+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:58.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:58 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:58.502+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:55:58.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:58 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:58.502+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:55:59.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:58 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:58.604+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:55:59.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:58 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:58.604+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:55:59.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:58 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:58.604+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.148040+0000 front 2024-05-24T11:55:29.148062+0000 (oldest deadline 2024-05-24T11:55:53.848037+0000) 2024-05-24T11:55:59.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:58 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:58.737+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:55:59.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:58 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:58.737+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:55:59.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:58 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:58.737+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:55:59.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:59 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:59.244+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:55:59.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:55:59 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:55:59.244+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:55:59.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:59 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:59.240+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:55:59.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:59 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:59.240+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:55:59.737 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:55:59 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:55:59.240+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:55:59.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:59 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:59.376+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:55:59.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:59 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:59.376+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:55:59.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:55:59 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:55:59.376+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:55:59.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:59 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:59.515+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:55:59.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:59 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:59.515+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:55:59.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:55:59 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:55:59.515+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:00.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:59 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:59.621+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:56:00.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:59 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:59.621+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:56:00.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:55:59 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:55:59.621+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.148040+0000 front 2024-05-24T11:55:29.148062+0000 (oldest deadline 2024-05-24T11:55:53.848037+0000) 2024-05-24T11:56:00.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:59 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:59.770+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:00.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:59 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:59.770+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:00.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:55:59 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:55:59.770+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:00.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:00 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:00.273+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:56:00.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:00 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:00.273+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:56:00.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:00 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:00.221+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:00.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:00 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:00.221+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:00.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:00 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:00.221+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:00.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:00 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:00.221+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:00.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:00 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:00.387+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:00.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:00 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:00.387+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:00.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:00 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:00.387+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:00.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:00 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:00.527+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:00.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:00 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:00.527+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:00.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:00 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:00.527+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:01.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:56:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:56:00.618+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:56:01.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:56:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:56:00.618+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:56:01.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:56:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:56:00.618+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.148040+0000 front 2024-05-24T11:55:29.148062+0000 (oldest deadline 2024-05-24T11:55:53.848037+0000) 2024-05-24T11:56:01.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:56:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:56:00.618+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:34.949226+0000 front 2024-05-24T11:55:34.949253+0000 (oldest deadline 2024-05-24T11:56:00.249271+0000) 2024-05-24T11:56:01.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:00.747+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:01.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:00.747+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:01.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:00 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:00.747+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:01.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:01 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:01.321+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:56:01.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:01 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:01.321+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:56:01.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:01 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:01.204+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:01.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:01 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:01.204+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:01.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:01 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:01.204+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:01.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:01 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:01.204+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:01.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:01 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:01.413+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:01.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:01 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:01.413+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:01.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:01 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:01.413+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:01.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:01 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:01.560+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:01.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:01 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:01.560+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:01.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:01 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:01.560+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:02.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:56:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:56:01.587+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:14.545689+0000 front 2024-05-24T11:55:14.545735+0000 (oldest deadline 2024-05-24T11:55:39.245816+0000) 2024-05-24T11:56:02.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:56:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:56:01.587+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.747189+0000 front 2024-05-24T11:55:22.746986+0000 (oldest deadline 2024-05-24T11:55:45.646909+0000) 2024-05-24T11:56:02.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:56:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:56:01.587+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.148040+0000 front 2024-05-24T11:55:29.148062+0000 (oldest deadline 2024-05-24T11:55:53.848037+0000) 2024-05-24T11:56:02.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:56:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6[77270]: 2024-05-24T11:56:01.587+0000 7faa6519f700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:34.949226+0000 front 2024-05-24T11:55:34.949253+0000 (oldest deadline 2024-05-24T11:56:00.249271+0000) 2024-05-24T11:56:02.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:01.783+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:02.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:01.783+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:02.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:01 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:01.783+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:02.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:02 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:02.347+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:56:02.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:02 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:02.347+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:56:02.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:02 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:02.170+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:02.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:02 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:02.170+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:02.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:02 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:02.170+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:02.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:02 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:02.170+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:02.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:02 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:02.375+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:02.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:02 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:02.375+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:02.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:02 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:02.375+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:02.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:02 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:02.588+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:02.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:02 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:02.588+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:02.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:02 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:02.588+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:02.781 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 24 11:56:02 smithi039 bash[137384]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-6 2024-05-24T11:56:03.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:02.778+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:03.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:02.778+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:03.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:02.778+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:03.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:02 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:02.778+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:03.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:03.313+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:56:03.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:03.313+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:56:03.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:03.313+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:37.724203+0000 front 2024-05-24T11:55:37.724167+0000 (oldest deadline 2024-05-24T11:56:02.424104+0000) 2024-05-24T11:56:03.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:03.179+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:03.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:03.179+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:03.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:03.179+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:03.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:03.179+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:03.584 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-24T11:56:03.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:03.626 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-24T11:56:03.626 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-24T11:56:03.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.7 2024-05-24T11:56:03.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:03 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:03.363+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:03.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:03 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:03.363+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:03.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:03 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:03.363+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:03.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:03 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:03.363+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:03.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:03 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:03.556+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:03.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:03 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:03.556+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:03.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:03 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:03.556+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:03.988 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:03 smithi053 systemd[1]: Stopping Ceph osd.7 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:03.988 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:03.875+0000 7fec7c91f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:03.988 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:03.875+0000 7fec7c91f700 -1 osd.7 108 *** Got signal Terminated *** 2024-05-24T11:56:03.988 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:03.875+0000 7fec7c91f700 -1 osd.7 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:56:04.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:03 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:03.747+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:04.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:03 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:03.747+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:04.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:03 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:03.747+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:04.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:03 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:03.747+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:04.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:04 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:04.347+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:56:04.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:04 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:04.347+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:56:04.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:04 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:04.347+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:37.724203+0000 front 2024-05-24T11:55:37.724167+0000 (oldest deadline 2024-05-24T11:56:02.424104+0000) 2024-05-24T11:56:04.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:04 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:04.195+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:04.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:04 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:04.195+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:04.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:04 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:04.195+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:04.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:04 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:04.195+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:04.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:04 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:04.342+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:04.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:04 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:04.342+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:04.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:04 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:04.342+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:04.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:04 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:04.342+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:04.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:04 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:04.534+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:04.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:04 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:04.534+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:04.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:04 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:04.534+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:05.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:04 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:04.790+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:05.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:04 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:04.790+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:05.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:04 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:04.790+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:05.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:04 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:04.790+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:05.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:05.371+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:56:05.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:05.371+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:56:05.487 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:05.371+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:37.724203+0000 front 2024-05-24T11:55:37.724167+0000 (oldest deadline 2024-05-24T11:56:02.424104+0000) 2024-05-24T11:56:05.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:05.152+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:05.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:05.152+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:05.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:05.152+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:05.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:05 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:05.152+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:05.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:05.363+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:05.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:05.363+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:05.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:05.363+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:05.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:05.363+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:05.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:05.545+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:05.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:05.545+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:05.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:05 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:05.545+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:06.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:05.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:06.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:05.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:06.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:05.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:06.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:05 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:05.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:06.393 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:06.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:06.393 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:06.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:06.393 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:06.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:06.393 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:06.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:06.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:06.390+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:56:06.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:06.390+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:56:06.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:06.390+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:37.724203+0000 front 2024-05-24T11:55:37.724167+0000 (oldest deadline 2024-05-24T11:56:02.424104+0000) 2024-05-24T11:56:06.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:06.533+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:06.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:06.533+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:06.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:06.533+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:06.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:06.356+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:06.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:06.356+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:06.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:06.356+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:06.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:06 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:06.356+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:07.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:06 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:06.814+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:07.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:06 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:06.814+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:07.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:06 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:06.814+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:07.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:06 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:06.814+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:07.372 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:07.066+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:07.372 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:07.066+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:07.372 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:07.066+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:07.372 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:07.066+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:07.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:07.369+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:56:07.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:07.369+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:56:07.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:07 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:07.369+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:37.724203+0000 front 2024-05-24T11:55:37.724167+0000 (oldest deadline 2024-05-24T11:56:02.424104+0000) 2024-05-24T11:56:07.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:07.506+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:07.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:07.506+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:07.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:07.506+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:07.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:07.348+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:07.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:07.348+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:07.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:07.348+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:07.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:07 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:07.348+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:08.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:07.796+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:08.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:07.796+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:08.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:07.796+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:08.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:07 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:07.796+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:08.408 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:08.098+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:08.409 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:08.098+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:08.409 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:08.098+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:08.409 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:08.098+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:08.409 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:08.098+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:08.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:08.405+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:23.821228+0000 front 2024-05-24T11:55:23.821271+0000 (oldest deadline 2024-05-24T11:55:49.621866+0000) 2024-05-24T11:56:08.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:08.405+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:29.622404+0000 front 2024-05-24T11:55:29.622619+0000 (oldest deadline 2024-05-24T11:55:56.023105+0000) 2024-05-24T11:56:08.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:08 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7[69805]: 2024-05-24T11:56:08.405+0000 7fec79716700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:37.724203+0000 front 2024-05-24T11:55:37.724167+0000 (oldest deadline 2024-05-24T11:56:02.424104+0000) 2024-05-24T11:56:08.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:08 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:08.526+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:08.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:08 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:08.526+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:08.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:08 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:08.526+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:08.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:08 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:08.388+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:08.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:08 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:08.388+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:08.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:08 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:08.388+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:08.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:08 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:08.388+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:09.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:08 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:08.800+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:09.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:08 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:08.800+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:09.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:08 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:08.800+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:09.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:08 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:08.800+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:09.457 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:09 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:09.124+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:09.457 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:09 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:09.124+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:09.457 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:09 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:09.124+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:09.457 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:09 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:09.124+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:09.458 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:09 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:09.124+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:09.737 INFO:journalctl@ceph.osd.7.smithi053.stdout:May 24 11:56:09 smithi053 bash[91972]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-7 2024-05-24T11:56:09.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:09.372+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:09.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:09.372+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:09.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:09.372+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:09.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:09.372+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:09.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:09.372+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:42.643407+0000 front 2024-05-24T11:55:42.643454+0000 (oldest deadline 2024-05-24T11:56:08.543347+0000) 2024-05-24T11:56:09.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:09.542+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:09.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:09.542+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:09.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:09.542+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:09.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:09.542+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:10.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:09 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:09.842+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:10.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:09 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:09.842+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:10.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:09 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:09.842+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:10.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:09 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:09.842+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:10.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:09 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:09.842+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:10.413 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:10 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:10.076+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:10.413 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:10 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:10.076+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:10.413 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:10 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:10.076+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:10.413 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:10 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:10.076+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:10.413 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:10 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:10.076+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:10.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:10.335+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:10.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:10.335+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:10.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:10.335+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:10.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:10.335+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:10.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:10.335+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:42.643407+0000 front 2024-05-24T11:55:42.643454+0000 (oldest deadline 2024-05-24T11:56:08.543347+0000) 2024-05-24T11:56:10.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:10.569+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:10.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:10.569+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:10.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:10.569+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:10.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:10 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:10.569+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:10.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-24T11:56:10.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:10.837 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-24T11:56:10.837 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-24T11:56:10.837 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.8 2024-05-24T11:56:11.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:10 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:10.814+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:11.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:10 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:10.814+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:11.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:10 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:10.814+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:11.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:10 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:10.814+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:11.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:10 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:10.814+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:11.243 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:10 smithi137 systemd[1]: Stopping Ceph osd.8 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:11.243 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:11.061+0000 7f20e8d9b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:11.243 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:11.061+0000 7f20e8d9b700 -1 osd.8 108 *** Got signal Terminated *** 2024-05-24T11:56:11.243 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:11.061+0000 7f20e8d9b700 -1 osd.8 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:56:11.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:11 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:11.094+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:11.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:11 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:11.094+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:11.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:11 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:11.094+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:11.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:11 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:11.094+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:11.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:11 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:11.094+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:11.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:11.341+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:11.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:11.341+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:11.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:11.341+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:11.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:11.341+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:11.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:11.341+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:42.643407+0000 front 2024-05-24T11:55:42.643454+0000 (oldest deadline 2024-05-24T11:56:08.543347+0000) 2024-05-24T11:56:11.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:11.568+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:11.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:11.568+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:11.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:11.568+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:11.745 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:11 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:11.568+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:12.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:11 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:11.826+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:12.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:11 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:11.826+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:12.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:11 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:11.826+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:12.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:11 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:11.826+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:12.326 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:11 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:11.826+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:12.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:12.097+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:12.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:12.097+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:12.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:12.097+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:12.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:12.097+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:12.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:12 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:12.097+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:12.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:12.348+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:12.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:12.348+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:12.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:12.348+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:12.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:12.348+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:12.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:12.348+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:42.643407+0000 front 2024-05-24T11:55:42.643454+0000 (oldest deadline 2024-05-24T11:56:08.543347+0000) 2024-05-24T11:56:12.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:12.545+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:12.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:12.545+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:12.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:12.545+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:12.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:12 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:12.545+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:13.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:12.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:13.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:12.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:13.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:12.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:13.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:12.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:13.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:12 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:12.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:13.489 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:13 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:13.096+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:13.490 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:13 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:13.096+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:13.490 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:13 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:13.096+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:13.490 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:13 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:13.096+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:13.490 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:13 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:13.096+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:13.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:13.374+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:13.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:13.374+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:13.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:13.374+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:13.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:13.374+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:13.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:13.374+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:42.643407+0000 front 2024-05-24T11:55:42.643454+0000 (oldest deadline 2024-05-24T11:56:08.543347+0000) 2024-05-24T11:56:13.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:13.566+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:13.745 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:13.566+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:13.745 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:13.566+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:13.745 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:13 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:13.566+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:14.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:13 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:13.743+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:14.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:13 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:13.743+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:14.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:13 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:13.743+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:14.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:13 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:13.743+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:14.077 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:13 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:13.743+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:14.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:14 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:14.136+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:14.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:14 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:14.136+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:14.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:14 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:14.136+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:14.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:14 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:14.136+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:14.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:14 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:14.136+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:14.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:14 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:14.525+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:14.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:14 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:14.525+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:14.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:14 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:14.525+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:14.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:14 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:14.525+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:14.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:14 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:14.424+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:14.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:14 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:14.424+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:14.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:14 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:14.424+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:14.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:14 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:14.424+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:14.744 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:14 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:14.424+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:42.643407+0000 front 2024-05-24T11:55:42.643454+0000 (oldest deadline 2024-05-24T11:56:08.543347+0000) 2024-05-24T11:56:15.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:14 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:14.763+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:15.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:14 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:14.763+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:15.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:14 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:14.763+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:15.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:14 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:14.763+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:15.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:14 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:14.763+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:15.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:15 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:15.166+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:15.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:15 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:15.166+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:15.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:15 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:15.166+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:15.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:15 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:15.166+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:15.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:15 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:15.166+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:15.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:15.419+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:17.038504+0000 front 2024-05-24T11:55:17.038489+0000 (oldest deadline 2024-05-24T11:55:39.338527+0000) 2024-05-24T11:56:15.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:15.419+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.639578+0000 front 2024-05-24T11:55:24.639632+0000 (oldest deadline 2024-05-24T11:55:48.040366+0000) 2024-05-24T11:56:15.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:15.419+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:31.541343+0000 front 2024-05-24T11:55:31.541365+0000 (oldest deadline 2024-05-24T11:55:58.542074+0000) 2024-05-24T11:56:15.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:15.419+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:38.542521+0000 front 2024-05-24T11:55:38.542584+0000 (oldest deadline 2024-05-24T11:56:02.642742+0000) 2024-05-24T11:56:15.743 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8[70966]: 2024-05-24T11:56:15.419+0000 7f20e5b92700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:42.643407+0000 front 2024-05-24T11:55:42.643454+0000 (oldest deadline 2024-05-24T11:56:08.543347+0000) 2024-05-24T11:56:15.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:15.568+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:15.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:15.568+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:15.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:15.568+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:15.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:15.568+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:15.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:15.568+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:16.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:15.757+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:16.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:15.757+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:16.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:15.757+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:16.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:15.757+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:16.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:15 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:15.757+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:16.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:16 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:16.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:16.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:16 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:16.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:16.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:16 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:16.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:16.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:16 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:16.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:16.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:16 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:16.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:16.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:16 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:16.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:16.910 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:16 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:16.567+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:16.911 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:16 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:16.567+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:16.911 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:16 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:16.567+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:16.911 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:16 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:16.567+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:16.911 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:16 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:16.567+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:17.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:16.767+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:17.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:16.767+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:17.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:16.767+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:17.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:16.767+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:17.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:16.767+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:17.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:16 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:16.767+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:53.751180+0000 front 2024-05-24T11:55:53.751200+0000 (oldest deadline 2024-05-24T11:56:16.651173+0000) 2024-05-24T11:56:17.243 INFO:journalctl@ceph.osd.8.smithi137.stdout:May 24 11:56:16 smithi137 bash[93708]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-8 2024-05-24T11:56:17.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:17 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:17.087+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:17.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:17 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:17.087+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:17.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:17 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:17.087+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:17.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:17 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:17.087+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:17.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:17 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:17.087+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:17.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:17 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:17.087+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:17.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:17 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:17.541+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:17.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:17 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:17.541+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:17.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:17 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:17.541+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:17.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:17 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:17.541+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:17.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:17 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:17.541+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:18.070 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-24T11:56:18.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:17 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:17.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:18.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:17 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:17.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:18.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:17 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:17.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:18.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:17 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:17.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:18.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:17 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:17.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:18.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:17 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:17.782+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:53.751180+0000 front 2024-05-24T11:55:53.751200+0000 (oldest deadline 2024-05-24T11:56:16.651173+0000) 2024-05-24T11:56:18.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:18.112 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-24T11:56:18.112 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-24T11:56:18.112 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.9 2024-05-24T11:56:18.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:18 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:18.057+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:18.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:18 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:18.057+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:18.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:18 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:18.057+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:18.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:18 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:18.057+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:18.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:18 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:18.057+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:18.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:18 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:18.057+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:18.576 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 systemd[1]: Stopping Ceph osd.9 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:18.576 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:18.349+0000 7f9fcef5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:18.576 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:18.349+0000 7f9fcef5e700 -1 osd.9 108 *** Got signal Terminated *** 2024-05-24T11:56:18.576 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:18.349+0000 7f9fcef5e700 -1 osd.9 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:56:18.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:18 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:18.525+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:18.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:18 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:18.525+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:18.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:18 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:18.525+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:18.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:18 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:18.525+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:18.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:18 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:18.525+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:19.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:18.789+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:19.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:18.789+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:19.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:18.789+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:19.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:18.789+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:19.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:18.789+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:19.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:18 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:18.789+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:53.751180+0000 front 2024-05-24T11:55:53.751200+0000 (oldest deadline 2024-05-24T11:56:16.651173+0000) 2024-05-24T11:56:19.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:19 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:19.070+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:19.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:19 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:19.070+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:19.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:19 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:19.070+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:19.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:19 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:19.070+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:19.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:19 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:19.070+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:19.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:19 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:19.070+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:19.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:19 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:19.515+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:19.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:19 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:19.515+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:19.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:19 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:19.515+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:19.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:19 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:19.515+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:19.993 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:19 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:19.515+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:20.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:19 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:19.765+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:20.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:19 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:19.765+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:20.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:19 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:19.765+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:20.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:19 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:19.765+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:20.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:19 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:19.765+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:20.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:19 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:19.765+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:53.751180+0000 front 2024-05-24T11:55:53.751200+0000 (oldest deadline 2024-05-24T11:56:16.651173+0000) 2024-05-24T11:56:20.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:20 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:20.100+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:20.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:20 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:20.100+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:20.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:20 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:20.100+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:20.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:20 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:20.100+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:20.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:20 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:20.100+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:20.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:20 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:20.100+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:20.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:20 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:20.483+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:20.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:20 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:20.483+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:20.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:20 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:20.483+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:20.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:20 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:20.483+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:20.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:20 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:20.483+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:21.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:20.720+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:21.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:20.720+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:21.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:20.720+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:21.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:20.720+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:21.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:20.720+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:21.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:20 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:20.720+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:53.751180+0000 front 2024-05-24T11:55:53.751200+0000 (oldest deadline 2024-05-24T11:56:16.651173+0000) 2024-05-24T11:56:21.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:21 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:21.055+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:21.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:21 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:21.055+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:21.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:21 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:21.055+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:21.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:21 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:21.055+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:21.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:21 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:21.055+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:21.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:21 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:21.055+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:21.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:21 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:21.459+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:21.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:21 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:21.459+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:21.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:21 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:21.459+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:21.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:21 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:21.459+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:21.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:21 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:21.459+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:22.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:21 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:21.680+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:22.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:21 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:21.680+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:22.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:21 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:21.680+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:22.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:21 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:21.680+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:22.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:21 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:21.680+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:22.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:21 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:21.680+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:53.751180+0000 front 2024-05-24T11:55:53.751200+0000 (oldest deadline 2024-05-24T11:56:16.651173+0000) 2024-05-24T11:56:22.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:22 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:22.083+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:22.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:22 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:22.083+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:22.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:22 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:22.083+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:22.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:22 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:22.083+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:22.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:22 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:22.083+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:22.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:22 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:22.083+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:22.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:22 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:22.440+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:22.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:22 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:22.440+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:22.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:22 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:22.440+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:22.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:22 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:22.440+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:22.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:22 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:22.440+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:23.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:22 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:22.636+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.945315+0000 front 2024-05-24T11:55:13.945427+0000 (oldest deadline 2024-05-24T11:55:39.845466+0000) 2024-05-24T11:56:23.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:22 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:22.636+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:22.146676+0000 front 2024-05-24T11:55:22.146666+0000 (oldest deadline 2024-05-24T11:55:47.446535+0000) 2024-05-24T11:56:23.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:22 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:22.636+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.147749+0000 front 2024-05-24T11:55:32.147866+0000 (oldest deadline 2024-05-24T11:55:56.847495+0000) 2024-05-24T11:56:23.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:22 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:22.636+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:36.847873+0000 front 2024-05-24T11:55:36.847931+0000 (oldest deadline 2024-05-24T11:56:02.148145+0000) 2024-05-24T11:56:23.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:22 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:22.636+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.449403+0000 front 2024-05-24T11:55:44.449441+0000 (oldest deadline 2024-05-24T11:56:09.749238+0000) 2024-05-24T11:56:23.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:22 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9[86059]: 2024-05-24T11:56:22.636+0000 7f9fc7d4d700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:53.751180+0000 front 2024-05-24T11:55:53.751200+0000 (oldest deadline 2024-05-24T11:56:16.651173+0000) 2024-05-24T11:56:23.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:23 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:23.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:23.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:23 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:23.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:23.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:23 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:23.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:23.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:23 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:23.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:23.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:23 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:23.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:23.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:23 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:23.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:23.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:23 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:23.111+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:01.143863+0000 front 2024-05-24T11:56:01.143903+0000 (oldest deadline 2024-05-24T11:56:22.843825+0000) 2024-05-24T11:56:23.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:23 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:23.419+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:23.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:23 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:23.419+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:23.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:23 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:23.419+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:23.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:23 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:23.419+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:23.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:23 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:23.419+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:24.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:24 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:24.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:24.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:24 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:24.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:24.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:24 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:24.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:24.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:24 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:24.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:24.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:24 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:24.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:24.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:24 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:24.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:24.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:24 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:24.119+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:01.143863+0000 front 2024-05-24T11:56:01.143903+0000 (oldest deadline 2024-05-24T11:56:22.843825+0000) 2024-05-24T11:56:24.576 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 24 11:56:24 smithi039 bash[137648]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-9 2024-05-24T11:56:24.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:24 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:24.430+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:24.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:24 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:24.430+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:24.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:24 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:24.430+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:24.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:24 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:24.430+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:24.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:24 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:24.430+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:24.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:24 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:24.430+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:25.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-24T11:56:25.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:25.480 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-24T11:56:25.481 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-24T11:56:25.481 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.10 2024-05-24T11:56:25.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.075+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:25.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.075+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:25.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.075+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:25.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.075+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:25.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.075+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:25.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.075+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:25.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.075+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:01.143863+0000 front 2024-05-24T11:56:01.143903+0000 (oldest deadline 2024-05-24T11:56:22.843825+0000) 2024-05-24T11:56:25.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:25.453+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:25.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:25.453+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:25.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:25.453+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:25.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:25.453+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:25.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:25.453+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:25.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:25.453+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:25.987 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 systemd[1]: Stopping Ceph osd.10 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:25.987 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.748+0000 7f4610212700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:25.987 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.748+0000 7f4610212700 -1 osd.10 108 *** Got signal Terminated *** 2024-05-24T11:56:25.987 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:25 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:25.748+0000 7f4610212700 -1 osd.10 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:56:26.490 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:26 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:26.033+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:26.490 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:26 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:26.033+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:26.491 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:26 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:26.033+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:26.491 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:26 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:26.033+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:26.491 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:26 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:26.033+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:26.491 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:26 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:26.033+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:26.491 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:26 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:26.033+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:01.143863+0000 front 2024-05-24T11:56:01.143903+0000 (oldest deadline 2024-05-24T11:56:22.843825+0000) 2024-05-24T11:56:26.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:26 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:26.405+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:26.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:26 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:26.405+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:26.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:26 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:26.405+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:26.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:26 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:26.405+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:26.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:26 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:26.405+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:26.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:26 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:26.405+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:27.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:27 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:27.039+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:27.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:27 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:27.039+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:27.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:27 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:27.039+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:27.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:27 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:27.039+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:27.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:27 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:27.039+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:27.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:27 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:27.039+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:27.488 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:27 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:27.039+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:01.143863+0000 front 2024-05-24T11:56:01.143903+0000 (oldest deadline 2024-05-24T11:56:22.843825+0000) 2024-05-24T11:56:27.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:27 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:27.394+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:27.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:27 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:27.394+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:27.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:27 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:27.394+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:27.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:27 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:27.394+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:27.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:27 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:27.394+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:27.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:27 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:27.394+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:28.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:28 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:28.014+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:28.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:28 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:28.014+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:28.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:28 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:28.014+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:28.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:28 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:28.015+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:28.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:28 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:28.015+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:28.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:28 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:28.015+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:28.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:28 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:28.015+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:01.143863+0000 front 2024-05-24T11:56:01.143903+0000 (oldest deadline 2024-05-24T11:56:22.843825+0000) 2024-05-24T11:56:28.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:28 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:28.423+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:28.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:28 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:28.423+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:28.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:28 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:28.423+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:28.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:28 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:28.423+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:28.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:28 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:28.423+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:28.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:28 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:28.423+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:29.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:29 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:29.034+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:29.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:29 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:29.034+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:29.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:29 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:29.034+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:29.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:29 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:29.034+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:29.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:29 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:29.034+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:29.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:29 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:29.034+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:29.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:29 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:29.034+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:01.143863+0000 front 2024-05-24T11:56:01.143903+0000 (oldest deadline 2024-05-24T11:56:22.843825+0000) 2024-05-24T11:56:29.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:29 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:29.403+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:29.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:29 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:29.403+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:29.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:29 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:29.403+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:29.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:29 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:29.403+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:29.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:29 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:29.403+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:29.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:29 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:29.403+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:30.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:30 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:30.082+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:15.537137+0000 front 2024-05-24T11:55:15.536992+0000 (oldest deadline 2024-05-24T11:55:38.436872+0000) 2024-05-24T11:56:30.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:30 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:30.082+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.139595+0000 front 2024-05-24T11:55:24.139550+0000 (oldest deadline 2024-05-24T11:55:50.039753+0000) 2024-05-24T11:56:30.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:30 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:30.082+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:30.040347+0000 front 2024-05-24T11:55:30.040223+0000 (oldest deadline 2024-05-24T11:55:55.340426+0000) 2024-05-24T11:56:30.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:30 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:30.082+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-24T11:55:35.340999+0000 front 2024-05-24T11:55:35.341122+0000 (oldest deadline 2024-05-24T11:56:00.040955+0000) 2024-05-24T11:56:30.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:30 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:30.082+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:44.141788+0000 front 2024-05-24T11:55:44.141905+0000 (oldest deadline 2024-05-24T11:56:07.641978+0000) 2024-05-24T11:56:30.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:30 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:30.082+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:51.742970+0000 front 2024-05-24T11:55:51.742917+0000 (oldest deadline 2024-05-24T11:56:15.242964+0000) 2024-05-24T11:56:30.487 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:30 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10[77814]: 2024-05-24T11:56:30.082+0000 7f4609001700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:01.143863+0000 front 2024-05-24T11:56:01.143903+0000 (oldest deadline 2024-05-24T11:56:22.843825+0000) 2024-05-24T11:56:30.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:30 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:30.387+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:30.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:30 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:30.387+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:30.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:30 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:30.387+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:30.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:30 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:30.387+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:30.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:30 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:30.387+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:30.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:30 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:30.387+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:31.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:31 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:31.343+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:31.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:31 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:31.343+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:31.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:31 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:31.343+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:31.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:31 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:31.343+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:31.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:31 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:31.343+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:31.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:31 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:31.343+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:31.987 INFO:journalctl@ceph.osd.10.smithi053.stdout:May 24 11:56:31 smithi053 bash[92229]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-10 2024-05-24T11:56:32.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:32 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:32.300+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:32.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:32 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:32.300+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:32.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:32 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:32.300+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:32.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:32 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:32.300+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:32.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:32 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:32.300+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:32.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:32 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:32.300+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:32.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-24T11:56:32.916 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:32.917 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-24T11:56:32.917 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-24T11:56:32.917 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.11 2024-05-24T11:56:33.247 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:32 smithi137 systemd[1]: Stopping Ceph osd.11 for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:33.247 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:33.150+0000 7fa59fc63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:33.247 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:33.150+0000 7fa59fc63700 -1 osd.11 108 *** Got signal Terminated *** 2024-05-24T11:56:33.247 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:33.150+0000 7fa59fc63700 -1 osd.11 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-24T11:56:33.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:33.326+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:33.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:33.326+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:33.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:33.326+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:33.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:33.326+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:33.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:33.326+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:33.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:33 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:33.326+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:34.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:34.339+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:34.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:34.339+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:34.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:34.339+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:34.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:34.339+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:34.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:34.339+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:34.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:34.339+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:34.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:34 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:34.339+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-24T11:56:08.363160+0000 front 2024-05-24T11:56:08.363128+0000 (oldest deadline 2024-05-24T11:56:34.262949+0000) 2024-05-24T11:56:35.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:35.332+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:35.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:35.332+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:35.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:35.332+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:35.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:35.332+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:35.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:35.332+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:35.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:35.332+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:35.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:35 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:35.332+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-24T11:56:08.363160+0000 front 2024-05-24T11:56:08.363128+0000 (oldest deadline 2024-05-24T11:56:34.262949+0000) 2024-05-24T11:56:36.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:36.307+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:36.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:36.307+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:36.743 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:36.307+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:36.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:36.307+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:36.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:36.307+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:36.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:36.307+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:36.744 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:36 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:36.307+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-24T11:56:08.363160+0000 front 2024-05-24T11:56:08.363128+0000 (oldest deadline 2024-05-24T11:56:34.262949+0000) 2024-05-24T11:56:37.746 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:37.324+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-24T11:55:13.053931+0000 front 2024-05-24T11:55:13.053833+0000 (oldest deadline 2024-05-24T11:55:41.854559+0000) 2024-05-24T11:56:37.747 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:37.324+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-05-24T11:55:24.755858+0000 front 2024-05-24T11:55:24.755912+0000 (oldest deadline 2024-05-24T11:55:50.655766+0000) 2024-05-24T11:56:37.747 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:37.324+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-05-24T11:55:32.257252+0000 front 2024-05-24T11:55:32.257332+0000 (oldest deadline 2024-05-24T11:55:53.957434+0000) 2024-05-24T11:56:37.747 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:37.324+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-05-24T11:55:45.659572+0000 front 2024-05-24T11:55:45.659429+0000 (oldest deadline 2024-05-24T11:56:08.559461+0000) 2024-05-24T11:56:37.747 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:37.324+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-05-24T11:55:49.060360+0000 front 2024-05-24T11:55:49.060398+0000 (oldest deadline 2024-05-24T11:56:14.960462+0000) 2024-05-24T11:56:37.747 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:37.324+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-24T11:56:00.261402+0000 front 2024-05-24T11:56:00.261374+0000 (oldest deadline 2024-05-24T11:56:23.761333+0000) 2024-05-24T11:56:37.747 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:37 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11[78978]: 2024-05-24T11:56:37.324+0000 7fa59ca5a700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-05-24T11:56:08.363160+0000 front 2024-05-24T11:56:08.363128+0000 (oldest deadline 2024-05-24T11:56:34.262949+0000) 2024-05-24T11:56:39.243 INFO:journalctl@ceph.osd.11.smithi137.stdout:May 24 11:56:38 smithi137 bash[93963]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-osd-11 2024-05-24T11:56:40.098 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-24T11:56:40.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:40.139 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-24T11:56:40.139 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-24T11:56:40.140 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.a 2024-05-24T11:56:40.507 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 24 11:56:40 smithi039 systemd[1]: Stopping Ceph mds.a for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:40.826 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 24 11:56:40 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-a[94600]: 2024-05-24T11:56:40.503+0000 7f6872688700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:40.826 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 24 11:56:40 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-a[94600]: 2024-05-24T11:56:40.503+0000 7f6872688700 -1 mds.a *** got signal Terminated *** 2024-05-24T11:56:44.989 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 24 11:56:44 smithi039 bash[137903]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-a 2024-05-24T11:56:44.995 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-24T11:56:45.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:45.034 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-24T11:56:45.034 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-24T11:56:45.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.d 2024-05-24T11:56:45.314 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 24 11:56:45 smithi039 systemd[1]: Stopping Ceph mds.d for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:45.575 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 24 11:56:45 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-d[93689]: 2024-05-24T11:56:45.311+0000 7f6599f37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:45.576 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 24 11:56:45 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-d[93689]: 2024-05-24T11:56:45.311+0000 7f6599f37700 -1 mds.d *** got signal Terminated *** 2024-05-24T11:56:49.023 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 24 11:56:48 smithi039 bash[138042]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-d 2024-05-24T11:56:49.028 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-24T11:56:49.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:49.063 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-24T11:56:49.063 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-24T11:56:49.063 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.g 2024-05-24T11:56:49.322 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 24 11:56:49 smithi039 systemd[1]: Stopping Ceph mds.g for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:49.575 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 24 11:56:49 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-g[94214]: 2024-05-24T11:56:49.318+0000 7efc8249f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:49.576 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 24 11:56:49 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-g[94214]: 2024-05-24T11:56:49.318+0000 7efc8249f700 -1 mds.g *** got signal Terminated *** 2024-05-24T11:56:52.229 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 24 11:56:51 smithi039 bash[138183]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-g 2024-05-24T11:56:52.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-24T11:56:52.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:52.269 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-24T11:56:52.270 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-24T11:56:52.270 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.j 2024-05-24T11:56:52.530 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 24 11:56:52 smithi039 systemd[1]: Stopping Ceph mds.j for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:52.825 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 24 11:56:52 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-j[93304]: 2024-05-24T11:56:52.526+0000 7f1587979700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:52.826 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 24 11:56:52 smithi039 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-j[93304]: 2024-05-24T11:56:52.526+0000 7f1587979700 -1 mds.j *** got signal Terminated *** 2024-05-24T11:56:55.076 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 24 11:56:54 smithi039 bash[138322]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-j 2024-05-24T11:56:55.098 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-24T11:56:55.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:56:55.133 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-24T11:56:55.133 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-24T11:56:55.134 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.b 2024-05-24T11:56:55.487 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:56:55 smithi053 systemd[1]: Stopping Ceph mds.b for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:56:55.487 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:56:55 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-b[84735]: 2024-05-24T11:56:55.385+0000 7f0de5fd6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:56:55.487 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:56:55 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-b[84735]: 2024-05-24T11:56:55.385+0000 7f0de5fd6700 -1 mds.b *** got signal Terminated *** 2024-05-24T11:57:00.237 INFO:journalctl@ceph.mds.b.smithi053.stdout:May 24 11:56:59 smithi053 bash[92495]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-b 2024-05-24T11:57:00.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-24T11:57:00.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:57:00.307 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-24T11:57:00.307 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-24T11:57:00.307 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.e 2024-05-24T11:57:00.638 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 24 11:57:00 smithi053 systemd[1]: Stopping Ceph mds.e for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:57:00.987 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 24 11:57:00 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-e[84045]: 2024-05-24T11:57:00.634+0000 7f8e4d757700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:57:00.987 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 24 11:57:00 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-e[84045]: 2024-05-24T11:57:00.634+0000 7f8e4d757700 -1 mds.e *** got signal Terminated *** 2024-05-24T11:57:02.862 INFO:journalctl@ceph.mds.e.smithi053.stdout:May 24 11:57:02 smithi053 bash[92635]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-e 2024-05-24T11:57:02.868 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-24T11:57:02.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:57:02.906 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-24T11:57:02.906 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-24T11:57:02.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.h 2024-05-24T11:57:03.196 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 24 11:57:02 smithi053 systemd[1]: Stopping Ceph mds.h for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:57:03.487 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 24 11:57:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-h[84394]: 2024-05-24T11:57:03.192+0000 7fd2a96d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:57:03.487 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 24 11:57:03 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-h[84394]: 2024-05-24T11:57:03.192+0000 7fd2a96d4700 -1 mds.h *** got signal Terminated *** 2024-05-24T11:57:06.308 INFO:journalctl@ceph.mds.h.smithi053.stdout:May 24 11:57:06 smithi053 bash[92774]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-h 2024-05-24T11:57:06.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-24T11:57:06.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:57:06.354 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-24T11:57:06.354 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-24T11:57:06.354 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.k 2024-05-24T11:57:06.637 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 24 11:57:06 smithi053 systemd[1]: Stopping Ceph mds.k for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:57:06.987 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 24 11:57:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-k[83703]: 2024-05-24T11:57:06.633+0000 7f22591df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:57:06.987 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 24 11:57:06 smithi053 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-k[83703]: 2024-05-24T11:57:06.633+0000 7f22591df700 -1 mds.k *** got signal Terminated *** 2024-05-24T11:57:09.426 INFO:journalctl@ceph.mds.k.smithi053.stdout:May 24 11:57:09 smithi053 bash[92914]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-k 2024-05-24T11:57:09.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-24T11:57:09.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:57:09.469 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-24T11:57:09.470 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-24T11:57:09.470 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.c 2024-05-24T11:57:09.993 INFO:journalctl@ceph.mds.c.smithi137.stdout:May 24 11:57:09 smithi137 systemd[1]: Stopping Ceph mds.c for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:57:09.993 INFO:journalctl@ceph.mds.c.smithi137.stdout:May 24 11:57:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-c[85226]: 2024-05-24T11:57:09.710+0000 7fadf9756700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:57:09.993 INFO:journalctl@ceph.mds.c.smithi137.stdout:May 24 11:57:09 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-c[85226]: 2024-05-24T11:57:09.710+0000 7fadf9756700 -1 mds.c *** got signal Terminated *** 2024-05-24T11:57:15.197 INFO:journalctl@ceph.mds.c.smithi137.stdout:May 24 11:57:14 smithi137 bash[94225]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-c 2024-05-24T11:57:15.204 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-24T11:57:15.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:57:15.242 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-24T11:57:15.242 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-24T11:57:15.242 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.f 2024-05-24T11:57:15.494 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:57:15 smithi137 systemd[1]: Stopping Ceph mds.f for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:57:15.993 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:57:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: 2024-05-24T11:57:15.502+0000 7f56982f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:57:15.993 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:57:15 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f[84885]: 2024-05-24T11:57:15.502+0000 7f56982f8700 -1 mds.f *** got signal Terminated *** 2024-05-24T11:57:19.065 INFO:journalctl@ceph.mds.f.smithi137.stdout:May 24 11:57:18 smithi137 bash[94364]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-f 2024-05-24T11:57:19.071 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-24T11:57:19.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:57:19.105 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-24T11:57:19.105 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-24T11:57:19.105 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.i 2024-05-24T11:57:19.353 INFO:journalctl@ceph.mds.i.smithi137.stdout:May 24 11:57:19 smithi137 systemd[1]: Stopping Ceph mds.i for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:57:19.743 INFO:journalctl@ceph.mds.i.smithi137.stdout:May 24 11:57:19 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-i[85568]: 2024-05-24T11:57:19.351+0000 7f7446cbd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:57:19.743 INFO:journalctl@ceph.mds.i.smithi137.stdout:May 24 11:57:19 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-i[85568]: 2024-05-24T11:57:19.351+0000 7f7446cbd700 -1 mds.i *** got signal Terminated *** 2024-05-24T11:57:25.153 INFO:journalctl@ceph.mds.i.smithi137.stdout:May 24 11:57:24 smithi137 bash[94504]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-i 2024-05-24T11:57:25.158 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-24T11:57:25.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:57:25.192 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-24T11:57:25.192 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-24T11:57:25.192 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.l 2024-05-24T11:57:25.479 INFO:journalctl@ceph.mds.l.smithi137.stdout:May 24 11:57:25 smithi137 systemd[1]: Stopping Ceph mds.l for f68008c4-19c0-11ef-bc9a-c7b262605968... 2024-05-24T11:57:25.743 INFO:journalctl@ceph.mds.l.smithi137.stdout:May 24 11:57:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-l[84542]: 2024-05-24T11:57:25.476+0000 7ff5996e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-24T11:57:25.743 INFO:journalctl@ceph.mds.l.smithi137.stdout:May 24 11:57:25 smithi137 ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-l[84542]: 2024-05-24T11:57:25.476+0000 7ff5996e0700 -1 mds.l *** got signal Terminated *** 2024-05-24T11:57:27.435 INFO:journalctl@ceph.mds.l.smithi137.stdout:May 24 11:57:27 smithi137 bash[94643]: ceph-f68008c4-19c0-11ef-bc9a-c7b262605968-mds-l 2024-05-24T11:57:27.440 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f68008c4-19c0-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-24T11:57:27.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-24T11:57:27.473 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-24T11:57:27.474 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-24T11:57:27.669 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:57:29.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-24T11:57:29.424 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:57:31.013 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-24T11:57:31.201 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T11:57:32.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-24T11:57:32.621 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-24T11:57:32.661 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-24T11:57:32.691 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-24T11:57:32.693 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/remote/smithi039/crash 2024-05-24T11:57:32.694 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/crash -- . 2024-05-24T11:57:32.763 INFO:teuthology.orchestra.run.smithi039.stderr:tar: /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-24T11:57:32.763 INFO:teuthology.orchestra.run.smithi039.stderr:tar: Error is not recoverable: exiting now 2024-05-24T11:57:32.765 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/remote/smithi053/crash 2024-05-24T11:57:32.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/crash -- . 2024-05-24T11:57:32.817 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-24T11:57:32.817 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-05-24T11:57:32.819 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/remote/smithi137/crash 2024-05-24T11:57:32.820 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/crash -- . 2024-05-24T11:57:32.877 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-24T11:57:32.877 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-05-24T11:57:32.878 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-24T11:57:32.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-05-24T11:57:32.915 INFO:teuthology.orchestra.run.smithi039.stderr:grep: /var/log/ceph/f68008c4-19c0-11ef-bc9a-c7b262605968/ceph.log: No such file or directory 2024-05-24T11:57:32.917 INFO:tasks.cephadm:Compressing logs... 2024-05-24T11:57:32.917 DEBUG:teuthology.orchestra.run.smithi039:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-24T11:57:32.963 DEBUG:teuthology.orchestra.run.smithi053:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-24T11:57:32.965 DEBUG:teuthology.orchestra.run.smithi137:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-24T11:57:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-24T11:57:32.995 INFO:teuthology.orchestra.run.smithi039.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-24T11:57:32.998 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-24T12:01:29.658 INFO:tasks.cephadm:Archiving logs... 2024-05-24T12:01:29.659 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/remote/smithi039/log 2024-05-24T12:01:29.660 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-24T12:01:54.314 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/remote/smithi053/log 2024-05-24T12:01:54.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-24T12:02:05.289 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/remote/smithi137/log 2024-05-24T12:02:05.290 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-24T12:02:14.371 INFO:tasks.cephadm:Removing cluster... 2024-05-24T12:02:14.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 --force 2024-05-24T12:02:14.567 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T12:02:14.892 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 --force 2024-05-24T12:02:15.089 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T12:02:15.456 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f68008c4-19c0-11ef-bc9a-c7b262605968 --force 2024-05-24T12:02:15.640 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: f68008c4-19c0-11ef-bc9a-c7b262605968 2024-05-24T12:02:15.968 INFO:tasks.cephadm:Removing cephadm ... 2024-05-24T12:02:15.968 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-24T12:02:15.990 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-24T12:02:16.013 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-24T12:02:16.030 INFO:tasks.cephadm:Teardown complete 2024-05-24T12:02:16.031 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-24T12:02:16.040 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/task/install/__init__.py", line 632, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa/tasks/check_counter.py", line 125, in end RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-05-24T12:02:16.040 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-24T12:02:16.041 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-24T12:02:16.043 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-24T12:02:16.056 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-24T12:02:16.178 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-24T12:02:16.188 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-24T12:02:16.188 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-24T12:02:16.191 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-24T12:02:16.194 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-24T12:02:16.208 INFO:teuthology.orchestra.run.smithi039.stderr:bash: ntpq: command not found 2024-05-24T12:02:16.210 INFO:teuthology.orchestra.run.smithi137.stderr:bash: ntpq: command not found 2024-05-24T12:02:16.212 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2024-05-24T12:02:16.214 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-24T12:02:16.214 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-05-24T12:02:16.215 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 +970us[ +995us] +/- 48ms 2024-05-24T12:02:16.215 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -1062us[-1062us] +/- 75ms 2024-05-24T12:02:16.215 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -369us[ -369us] +/- 71ms 2024-05-24T12:02:16.215 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T12:02:16.216 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-24T12:02:16.216 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-05-24T12:02:16.216 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +924us[ +932us] +/- 48ms 2024-05-24T12:02:16.216 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -1108us[-1100us] +/- 75ms 2024-05-24T12:02:16.216 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -401us[ -393us] +/- 71ms 2024-05-24T12:02:16.216 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T12:02:16.217 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-24T12:02:16.217 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-05-24T12:02:16.218 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +966us[+1026us] +/- 48ms 2024-05-24T12:02:16.218 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -1080us[-1080us] +/- 75ms 2024-05-24T12:02:16.218 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -375us[ -316us] +/- 71ms 2024-05-24T12:02:16.218 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-24T12:02:16.219 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-24T12:02:16.228 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-24T12:02:16.229 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-24T12:02:16.238 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-24T12:02:16.362 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-24T12:02:16.444 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-24T12:02:16.512 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-24T12:02:16.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T12:02:16.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-24T12:02:16.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T12:02:16.588 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-24T12:02:16.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T12:02:16.618 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-24T12:02:16.629 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-24T12:02:16.638 INFO:teuthology.task.internal:Duration was 2813.055173 seconds 2024-05-24T12:02:16.639 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-24T12:02:16.648 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-24T12:02:16.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-24T12:02:16.651 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-24T12:02:16.654 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-24T12:02:16.705 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-24T12:02:16.711 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-24T12:02:16.716 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-24T12:02:17.122 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-24T12:02:17.123 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-05-24T12:02:17.123 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-24T12:02:17.149 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-05-24T12:02:17.150 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-24T12:02:17.179 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-05-24T12:02:17.179 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-24T12:02:17.202 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-24T12:02:17.202 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-24T12:02:17.205 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-24T12:02:17.222 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-24T12:02:17.297 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-24T12:02:17.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-24T12:02:17.299 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-24T12:02:17.302 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-24T12:02:17.412 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-24T12:02:17.423 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-24T12:02:17.423 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-24T12:02:17.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-24T12:02:17.483 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-24T12:02:17.511 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-24T12:02:17.522 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-24T12:02:17.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-24T12:02:17.527 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-24T12:02:17.557 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-05-24T12:02:17.559 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-05-24T12:02:17.578 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-05-24T12:02:17.599 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-24T12:02:17.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T12:02:17.629 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-24T12:02:17.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T12:02:17.650 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-24T12:02:17.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-24T12:02:17.666 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-24T12:02:17.678 INFO:teuthology.task.internal:Transferring archived files... 2024-05-24T12:02:17.679 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/remote/smithi039 2024-05-24T12:02:17.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-24T12:02:17.723 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/remote/smithi053 2024-05-24T12:02:17.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-24T12:02:17.774 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718222/remote/smithi137 2024-05-24T12:02:17.774 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-24T12:02:17.815 INFO:teuthology.task.internal:Removing archive directory... 2024-05-24T12:02:17.815 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-24T12:02:17.817 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-24T12:02:17.820 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-24T12:02:17.866 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-24T12:02:17.878 INFO:teuthology.task.internal:Not uploading archives. 2024-05-24T12:02:17.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-24T12:02:17.889 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-24T12:02:17.889 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-24T12:02:17.892 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-24T12:02:17.894 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-24T12:02:17.906 INFO:teuthology.orchestra.run.smithi039.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 24 12:02 /home/ubuntu/cephtest 2024-05-24T12:02:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 24 12:02 /home/ubuntu/cephtest 2024-05-24T12:02:17.922 INFO:teuthology.orchestra.run.smithi137.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 24 12:02 /home/ubuntu/cephtest 2024-05-24T12:02:17.924 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-24T12:02:17.934 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-24T12:02:17.947 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-05-24T12:02:17.951 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/postgres}} duration: 2813.0551726818085 failure_reason: 'The following counters failed to be set on mds daemons: {''mds.imported'', ''mds.exported''}' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-05-24T12:02:17.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-24T12:02:18.045 INFO:teuthology.run:FAIL